wiki:InstallPetscAndMpi

PetSC and MPI installation

Download MPICH from ftp://ftp.mcs.anl.gov/pub/mpi/mpich.tar.gz to $HOME.

Note: if you are installing on a machine with g++ 4.3, download the patched MPI archive from https://chaste.cs.ox.ac.uk/cgi-bin/trac.cgi/attachment/wiki/InstallGuides/NonRootDeveloper/mpich-1.2.7p1.tar.gz.

Download Petsc from ftp://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-2.3.3-p15.tar.gz to $HOME.

cd 
tar -xzvf mpich.tar.gz
cd mpich-1.2.7p1
CCFLAGS=-fPIC CFLAGS=-fPIC  ./configure --prefix=${HOME}/mpi --with-comm=shared --with-device=ch_shmem --enable-sharedlib --disable-f77
make
cd examples/test/
make testing
cd ../..
make install
cd
rm -rf mpich-1.2.7p1/  mpich.tar.gz 


tar -xzvf petsc-2.3.3-p15.tar.gz 
rm petsc-2.3.3-p15.tar.gz 
cd petsc-2.3.3-p15/
export PETSC_DIR=`pwd`
./config/configure.py  --download-c-blas-lapack=1 --with-mpi-dir=${HOME}/mpi --with-x=false  -PETSC_ARCH=linux-gnu --with-clanguage=cxx --with-shared --with-fortran=0
make all
./config/configure.py  --download-c-blas-lapack=1 --with-mpi-dir=${HOME}/mpi --with-x=false   --with-debugging=0 -PETSC_ARCH=linux-gnu-opt --with-clanguage=cxx --with-shared --with-fortran=0
make all
./config/configure.py  --download-c-blas-lapack=1 --with-mpi-dir=${HOME}/mpi --with-x=false  -PETSC_ARCH=linux-gnu-profile --with-clanguage=cxx --with-shared --with-fortran=0 --CFLAGS=-pg -CXXFLAGS=-pg -LDFLAGS=-pg
make all

PETSc 3 instructions

The configuration of PETSc 3 treats PETSC_ARCH as an environmental variable, rather than as a command-line option:

wget http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.0.0-p8.tar.gz
tar xzvf petsc-3.0.0-p8.tar.gz 
rm petsc-3.0.0-p8.tar.gz
cd petsc-3.0.0-p8/
export PETSC_DIR=`pwd`

export PETSC_ARCH=linux-gnu
./config/configure.py  --download-c-blas-lapack=1 --with-mpi-dir=${HOME}/mpi --with-x=false --with-clanguage=cxx --with-shared --with-fortran=0
make all

export PETSC_ARCH=linux-gnu-opt
./config/configure.py  --download-c-blas-lapack=1 --with-mpi-dir=${HOME}/mpi --with-x=false   --with-debugging=0 --with-clanguage=cxx --with-shared --with-fortran=0
make all

export PETSC_ARCH=linux-gnu-profile 
./config/configure.py  --download-c-blas-lapack=1 --with-mpi-dir=${HOME}/mpi --with-x=false --with-clanguage=cxx --with-shared --with-fortran=0 --CFLAGS=-pg -CXXFLAGS=-pg -LDFLAGS=-pg
make all

PETSc 3.1

The above appears to work for PETSc 3.1 also. Specifically, petsc-3.1-p4

PETSc 3.2

Noticeable changes at PETSc 3.2

  • the download flag for Blas/Lapack has changed.
  • we can now get PETSc to download and install CVODE (Sundials).
  • the shared libraries flag has changed.
  • There are serious memory problems using MPICH-1 with 3.2 so we are switching to MPICH-2.
wget http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.2-p6.tar.gz
tar xzvf petsc-3.2-p6.tar.gz 
rm petsc-3.2-p6.tar.gz 
cd petsc-3.2-p6/
export PETSC_DIR=`pwd`

export PETSC_ARCH=linux-gnu
./config/configure.py  --download-f2cblaslapack=1 --download-mpich=1 --download-parmetis=1 --download-hdf5=1  --download-sundials --with-x=false --with-clanguage=cxx --with-shared-libraries --with-fortran=0
./config/builder.py #Equivalent to make all

export PETSC_ARCH=linux-gnu-opt
./config/configure.py  --download-f2cblaslapack=1 --download-mpich=1 --download-parmetis=1 --download-hdf5=1  --download-sundials --with-x=false --with-clanguage=cxx --with-shared-libraries --with-fortran=0 --with-debugging=0
./config/builder.py #Equivalent to make all

export PETSC_ARCH=linux-gnu-profile
./config/configure.py  --download-f2cblaslapack=1 --download-mpich=1 --download-parmetis=1 --download-hdf5=1  --download-sundials --with-x=false --with-clanguage=cxx --with-shared-libraries --with-fortran=0 --CFLAGS="-fno-omit-frame-pointer -pg" -CXXFLAGS="-fno-omit-frame-pointer -pg" -LDFLAGS=-pg
./config/builder.py #Equivalent to make all

Final steps

Next, add MPI programs to your path, by adding the following to ~/.bashrc:

# Add MPI programs to path
export PATH=$HOME/mpi/bin:$PATH

Finally, it used to be the case that to prevent compiler warnings when building Chaste, MPI needed to be patched. If the path to MPI's headers is specified in other_includepaths in your hostconfig file, this should not now occur. If it does, you can apply the patch file, mpi.patch, attached to this page. Download it (click on the link then click 'Original format' at the bottom to avoid getting html in the download), to ~/mpi.patch then do

cd ~/mpi
patch -p1 <../mpi.patch

MPI installation with Fortran/Fortran 90 compilers

Note: these instructions don't appear to work on Lucid!

First, make sure that you have gfortran installed:

sudo aptitude install gfortran

Download MPICH as above.

cd ~/mpich-1.2.7p1
F90=gfortran F77=g77 CCFLAGS=-fPIC CFLAGS=-fPIC  ./configure --prefix=${HOME}/mpi --with-comm=shared --with-device=ch_shmem --enable-sharedlib --enable-f90
make
make install

Now edit the mpif90 builder script! (Also, make sure that F90BASE="gfortran" is set - we have seen occasions when it is missed...)

 diff -u mpi/bin/mpif90-orig mpi/bin/mpif90
--- mpi/bin/mpif90-orig	2010-01-20 16:52:31.000000000 +0000
+++ mpi/bin/mpif90	2010-01-20 16:58:55.000000000 +0000
@@ -30,8 +30,8 @@
 F90BASE="gfortran"
 F90LINKERBASE="gfortran"
 LDFLAGSBASE=""
-BASE_FFLAGS=" "
-F90INC=""
+BASE_FFLAGS=" -fsecond-underscore"
+F90INC="-I"
 # f90modinc specifies how to add a directory to the search path for modules.
 # Some compilers (Intel ifc) do not support this concept, and instead need
 # a specific list of files that contain module names and directories.
@@ -43,7 +43,7 @@
 #
 # Linker flags
 F90_LDFLAGS=""
-BASE_LIB_LIST="   -lpthread  -lrt "
+BASE_LIB_LIST="   -lpthread  -lrt -lg2c"
 FLIB_LIST=""
 F90LIB_PATH_LEADER="-L"
 F90LIB_PATH="${libdir}"
Last modified 7 years ago Last modified on May 10, 2013, 4:37:36 PM

Attachments (1)

  • mpi.patch (4.9 KB) - added by jonathan.cooper@… 14 years ago. Patch to remove compiler warnings from MPI C++ interface

Download all attachments as: .zip