Cannot install mpi4py on CentOS 7

Question

I have CentOS 7 and I have installed mpicc (it works and compiles for openmpi in C). I also have python 2.7.5 and just installed pip. I'm running this command and get the following errors:

sudo pip install mpi4py

Collecting mpi4py   Using cached mpi4py-2.0.0.tar.gz Installing
> collected packages: mpi4py   Running setup.py install for mpi4py ...
> error
>     Complete output from command /usr/bin/python2 -u -c "import setuptools,
> tokenize;__file__='/tmp/pip-build-x5jD4O/mpi4py/setup.py';exec(compile(getattr(tokenize,
> 'open', open)(__file__).read().replace('\r\n', '\n'), __file__,
> 'exec'))" install --record /tmp/pip-mpMoZO-record/install-record.txt
> --single-version-externally-managed --compile:
>     running install
>     running build
>     running build_src
>     running build_py
>     creating build
>     creating build/lib.linux-x86_64-2.7
>     creating build/lib.linux-x86_64-2.7/mpi4py
>     copying src/__main__.py -> build/lib.linux-x86_64-2.7/mpi4py
>     copying src/__init__.py -> build/lib.linux-x86_64-2.7/mpi4py
>     creating build/lib.linux-x86_64-2.7/mpi4py/include
>     creating build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.MPI.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.MPI_api.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/__init__.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/libmpi.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/MPI.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/__init__.pyx -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi.pxi -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/include/mpi4py/mpi4py.i -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
>     copying src/MPI.pxd -> build/lib.linux-x86_64-2.7/mpi4py
>     copying src/libmpi.pxd -> build/lib.linux-x86_64-2.7/mpi4py
>     running build_clib
>     MPI configuration: [mpi] from 'mpi.cfg'
>     checking for library 'lmpe' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -llmpe -o _configtest
>     /bin/ld: cannot find -llmpe
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'mpe' dylib library
>     creating build/temp.linux-x86_64-2.7
>     creating build/temp.linux-x86_64-2.7/src
>     creating build/temp.linux-x86_64-2.7/src/lib-pmpi
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/mpe.o
>     creating build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/mpe.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libmpe.so
>     checking for library 'vt-mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt-mpi -o _configtest
>     /bin/ld: cannot find -lvt-mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     checking for library 'vt.mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt.mpi -o _configtest
>     /bin/ld: cannot find -lvt.mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'vt' dylib library
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt.o
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt.so
>     checking for library 'vt-mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt-mpi -o _configtest
>     /bin/ld: cannot find -lvt-mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     checking for library 'vt.mpi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt.mpi -o _configtest
>     /bin/ld: cannot find -lvt.mpi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'vt-mpi' dylib library
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt-mpi.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-mpi.o
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-mpi.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt-mpi.so
>     checking for library 'vt-hyb' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt-hyb -o _configtest
>     /bin/ld: cannot find -lvt-hyb
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     checking for library 'vt.ompi' ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
>     gcc -pthread _configtest.o -lvt.ompi -o _configtest
>     /bin/ld: cannot find -lvt.ompi
>     collect2: error: ld returned 1 exit status
>     failure.
>     removing: _configtest.c _configtest.o
>     building 'vt-hyb' dylib library
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt-hyb.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-hyb.o
>     gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-hyb.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt-hyb.so
>     running build_ext
>     MPI configuration: [mpi] from 'mpi.cfg'
>     checking for MPI compile and link ...
>     gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _configtest.c -o _configtest.o
>     _configtest.c:2:17: fatal error: mpi.h: No such file or directory
>      #include <mpi.h>
>                      ^
>     compilation terminated.
>     failure.
>     removing: _configtest.c _configtest.o
>     error: Cannot compile MPI programs. Check your configuration!!!

I tried every solution I found so far and none seemed to work, has anyone any idea about this problem, please? Thank you

Answer 1

it seems you missed to install the development files of OpenMPI on Centos, the line that is the key here is:

_configtest.c:2:17: fatal error: mpi.h: No such file or directory
  #include <mpi.h>

you should install the openmpi-devel (or equivalent) through yum and you should be good to reinstall the mpi4py module.

Answer 2

For anyone else who runs into this, I'm posting this answer just in case you have a similar problem to me. The answer from @francis was the key. On my Centos7.7 system I was able to install mpi4py using the CC environment variable suggested by @iakov-nakhimovski but it was not working and I was getting runtime errors. The problem turned out to be that other software installed on the system had its own python installation in the path which was interfering with the setup of mpi4py. The solution was to remove this other installation from the path and to load the openmpi module which was shipped with the openmpi-devel package (I think).

module load mpi/openmpi-x86_64

Then install mpi4py using pip (note you do not need to explicitly define CC=mpicc since all these tools get put into the path by the environment module). When running a python script which requires mpi4py you also need to load this module.

Answer 3

Run into the same issue and solved with:

yum install openmpi-devel
export CC=/usr/lib64/openmpi/bin/mpicc
pip install mpi4py

Answer 4

You could also try:

CentOS

yum install python-mpi4py
yum install python3-mpi4py

Debian

apt install python-mpi4py
apt install python3-mpi4py