Opened 8 years ago

Closed 5 years ago

Last modified 5 years ago

#2250 closed defect (fixed)

Issue with parMETIS 4.0.2

Reported by: AlexF Owned by:
Priority: normal Milestone: Iteration I2
Component: infrastructure Keywords:
Cc: jonc@…, jmpf@…, GaryM Estimated pair-hours remaining: 0
Pair-hours expended on ticket: 6 Initial estimate of effort: 1
Editable by public: yes

Description (last modified by jmpf@…)

Note parMETIS 4 is associated with METIS 5. (parMETIS 3 was associated with METIS 4.)

I tried to use Chaste with Parmetis 4.0.2 and ran into two problems:

  1. the type idxtype has been renamed to idx_t;
  2. the argument have changed in the method METIS_PartMeshNodal.

In the end I just reinstalled an earlier version of Parmetis to get things working, but thought I'd raise a ticket to highlight this problem in case at some point someone wants to fix it.

We also need to update the install guide (e.g. InstallParMetis) as reported by Dan:

the latest version is at http://glaros.dtc.umn.edu/gkhome/fetch/sw/parmetis/parmetis-4.0.2.tar.gz or 3.1 is http://glaros.dtc.umn.edu/gkhome/fetch/sw/parmetis/OLD/ParMetis-3.1.tar.gz

Change History (21)

comment:1 Changed 8 years ago by jonc@…

  • Editable by public set

comment:2 Changed 8 years ago by jonc@…

  • Description modified (diff)

comment:3 Changed 8 years ago by jonc@…

  • Pair-hours expended on ticket changed from 0 to 0.1

Updated InstallParMetis.

comment:4 Changed 8 years ago by jmpf@…

PETSc 3.3 downloads parMETIS 4.0.2 (see #2130)

comment:5 Changed 8 years ago by jmpf@…

  • Description modified (diff)
  • Pair-hours expended on ticket changed from 0.1 to 1
  • Summary changed from Issue with Parmetis 4.0.2 to Issue with parMETIS 4.0.2

Starting to work through the issues.

Index: mesh/src/common/NodePartitioner.cpp
===================================================================
--- mesh/src/common/NodePartitioner.cpp	(revision 17709)
+++ mesh/src/common/NodePartitioner.cpp	(working copy)
@@ -58,9 +58,11 @@
 extern void METIS_PartMeshNodal(int*, int*, int*, int*, int*, int*, int*, int*, int*);
 }
 #include <parmetis.h>
+#if (PARMETIS_MAJOR_VERSION >= 4) //ParMETIS 4.x and above
+#define idxtype idx_t
+#endif
 
 
-
 template <unsigned ELEMENT_DIM, unsigned SPACE_DIM>
 void NodePartitioner<ELEMENT_DIM, SPACE_DIM>::DumbPartitioning(AbstractMesh<ELEMENT_DIM, SPACE_DIM>& rMesh,
                                                                std::set<unsigned>& rNodesOwned)
@@ -149,11 +151,19 @@
         int numflag = 0; //0 means C-style numbering is assumed
         int nparts = PetscTools::GetNumProcs();
         int edgecut;
-        idxtype* epart = new idxtype[ne];
-        assert(epart != NULL);
 
         Timer::Reset();
+#if (PARMETIS_MAJOR_VERSION >= 4) //ParMETIS 4.x and above
+        // => METIS 5
+        assert(METIS_VER_MAJOR == 5);
+        real_t* epart = new real_t[ne];
+        assert(epart != NULL);
         METIS_PartMeshNodal(&ne, &nn, elmnts, &etype, &numflag, &nparts, &edgecut, epart, npart);//, wgetflag, vwgt);
+#else
+        idxtype* epart = new idxtype[ne];
+        assert(epart != NULL);
+        METIS_PartMeshNodal(&ne, &nn, elmnts, &etype, &numflag, &nparts, &edgecut, epart, npart);//, wgetflag, vwgt);
+#endif

Note that this diff doesn't work yet:

  • In METIS 5 (compatible with parMETIS 4) METIS_PartMeshNodal has idx_t rather than idxtype.
  • epart has changed from idx_t to real_t
  • METIS_PartMeshNodal has not only changed the type of its arguments -- the number of arguments has also changed
  • There doesn't appear to be clear guidance as to how to change to the new interfact

Issues with METIS_LIBRARY partitioning:

Suggestion: I will be easier (and more helpful to the users) to deprecate METIS_LIBRARY than to fix it for parMETIS 4/METIS 5

comment:6 Changed 8 years ago by jmpf@…

r17708 changes mixed-dimension partitioning.

Here's what to hit next!

heart/src/problem/HeartConfigDefaults.hpp:        <MeshPartitioning>metis</MeshPartitioning>

comment:7 Changed 8 years ago by Sara

See also #2263

comment:8 Changed 8 years ago by Sara

r17953 changes HeartConfigDefaults to use parMETIS partitioning

comment:9 Changed 8 years ago by Sara

  • Pair-hours expended on ticket changed from 1 to 2

comment:10 Changed 8 years ago by Sara

r17959 changed tolerance of TestElectrodes to run on 3 or more processors - we should probably visualise the results to make sure something strange is not going on.

comment:11 follow-up: Changed 8 years ago by Sara

In r18048: make code compile with new ParMETIS and METIS. Remove unneeded code from #2106.

Todo: code for METIS partition segfaults with new version.

comment:12 follow-up: Changed 8 years ago by jmpf@…

Small code fix in r18053

comment:13 in reply to: ↑ 12 Changed 8 years ago by jmpf@…

Replying to jmpf@…:

Small code fix in r18053

and in r18054

comment:14 in reply to: ↑ 11 Changed 8 years ago by jmpf@…

  • Pair-hours expended on ticket changed from 2 to 3

Replying to Sara:

Todo: code for METIS partition segfaults with new version.

Done in r18189. All parallel test pack now passes.

comment:15 Changed 8 years ago by jmpf@…

  • Estimated pair-hours remaining changed from 1 to 0
  • Pair-hours expended on ticket changed from 3 to 4
  • Resolution set to fixed
  • Status changed from new to closed

This is working now. Crontab added in r18204. See #2130 for any further developments.

comment:16 follow-up: Changed 8 years ago by GaryM

What colour should we make Metis 4.* on InstallGuides/DependencyVersions ?

comment:17 in reply to: ↑ 16 Changed 8 years ago by jmpf@…

Replying to GaryM:

What colour should we make Metis 4.* on InstallGuides/DependencyVersions ?

PETSc 3.3 and METIS 4 are both somewhere between orange and green. When we have decided on the new state of the crontab then they should get their new colours before the ticket gets closed.

comment:18 Changed 8 years ago by jonc@…

  • Pair-hours expended on ticket changed from 4 to 4.25

r18391 and r18390 get lofty to check it's using the requested ParMETIS, and update the crontab accordingly. Once the build runs (this weekend) we can update the wiki page.

comment:19 Changed 6 years ago by jmpf@…

  • Estimated pair-hours remaining changed from 0 to 2
  • Resolution fixed deleted
  • Status changed from closed to reopened

There is a todo on this ticket:

    ///\todo #2250 Direct calls to METIS are to be deprecated
    WARN_ONCE_ONLY("METIS_LIBRARY partitioning is deprecated and will be removed from later versions of Chaste");

This was prior to 3.2 so we can now complete the deprecation.

comment:20 Changed 5 years ago by jmpf@…

  • Estimated pair-hours remaining changed from 2 to 0
  • Pair-hours expended on ticket changed from 4.25 to 6
  • Resolution set to fixed
  • Status changed from reopened to closed

METIS partitioner deprecated in r26447 r26448 r26449.

comment:21 Changed 5 years ago by jmpf@…

Found a serial/parallel inconsistency: fixed in r26450.

Note: See TracTickets for help on using tickets.