Trouble Installing GalSim on OSX with Anaconda

Question

I have been attempting to install GalSim on OSX 10.9 Mavericks with Anaconda installed and set as the default python, but have been running into the following error:

Unable to build a python loadable module using the python executable:
/usr/bin/env python,
the library name libpython2.7.a,
and the libdir //anaconda/lib/python2.7/config.
If these are not the correct library names, you can tell scons the 
correct names to use with the flags EXTRA_LIB_PATH and/or EXTRA_LIBS.

When checking my config.log files, there are a few instances of Undefined symbols for architecture x86_64:, even though I made sure that the compiler being used was clang++, as recommended in the GalSim FAQ.

There are also numerous instances of the following:

/usr/bin/env python < .sconf_temp/conftest_73 > .sconf_temp/conftest_73.out
Fatal Python error: PyThreadState_Get: no current thread
sh: line 1: 17019 Abort trap: 6           /usr/bin/env python < ".sconf_temp/conftest_73" > ".sconf_temp/conftest_73.out"

I'm not sure what to do to remedy the situation. I've reinstalled Boost several times, using the ./b2 -a command for each time after the first. I've made sure that boost is referencing /anaconda/bin/python, and confirmed it by checking the project-config.jam files for each installation. I've used the commands

./bootstrap.sh
./b2 -a toolset=clang cxxflags="-stdlib=libc++" linkflags="-stdlib=libc++" install

as recommended in the GalSim FAQ. I am really not sure what else to try besides attempting to reinstall all the requisite packages all over again. Does anyone have any advice on what to do before I go to my last resort? Any help is appreciated.

Below is the output from my last scons run:

scons: Reading SConscript files ...
SCons is version 2.3.1 using python version 2.7.6
Python is from //anaconda/include/python2.7
Using the following (non-default) scons options:
   CXX = clang++
These can be edited directly in the file gs_scons.conf.
Type scons -h for a full list of available options.
Using python =  /usr/bin/env python
Using default PYPREFIX =  //anaconda/lib/python2.7/site-packages
Using compiler: /usr/bin/clang++
compiler version: 5.1
Determined that a good number of jobs = 2
Checking for C++ header file fftw3.h... yes
Checking for correct FFTW linkage... yes
Checking for boost header files... yes
Checking for C++ header file TMV.h... yes
Using TMV_LINK file: /usr/local/share/tmv/tmv-link
     -ltmv -lblas
Mac version is 10.9.3
XCode version is 5.1.1
Checking for correct TMV linkage... (this may take a little while)
Checking for correct TMV linkage... yes
Checking if we can build against Python... 
Unable to build a python loadable module using the python executable:
/usr/bin/env python,
the library name libpython2.7.a,
and the libdir //anaconda/lib/python2.7/config.
If these are not the correct library names, you can tell scons the 
correct names to use with the flags EXTRA_LIB_PATH and/or EXTRA_LIBS.

Please fix the above error(s) and rerun scons.
Note: you may want to look through the file INSTALL.md for advice.
Also, if you are having trouble, please check the INSTALL FAQ at 
   https://github.com/GalSim-developers/GalSim/wiki/Installation%20FAQ

Answer 1

Michael - I think your problem may be related to this issue: Boost.Python python linkage error

In short, it seems that boost will sometimes claim to be linking against anaconda python, but it will really link against a system python despite it all.

That page has a solution as well. It seems a bit kludgy, so you may want to see if someone else has an idea. But if not, then you could try that?

Answer 2

I believe the crux of the problem is that anaconda's python library doesn't have the install name set correctly. Here is what otool reports for that library on my system:

$ otool -L /anaconda/lib/libpython2.7.dylib 
/anaconda/lib/libpython2.7.dylib:
    libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 476.0.0)
    /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

On Macs (as opposed to other Unix/Linux varieties), the runtime loader looks at the install names of all the libraries to be loaded in the dylib or executable. Since anaconda doesn't have this set correctly here, when GalSim or boost compile against it, that library just has the file name with no directory. So the loader doesn't know where it is, goes looking in the normal places, and finds the system version first.

The answer pointed to by user2932864 basically changes the runtime search order to put the anaconda location before the system python, so the loader finds the anaconda version. However, this solution doesn't work so well if you want to have both python options available on your system. The better solution (IMO) is to fix the anaconda library file. To do this, simply type (assuming your anaconda installation is in /anaconda):

sudo install_name_tool -id /anaconda/lib/libpython2.7.dylib /anaconda/lib/libpython2.7.dylib 

After doing this, I was able to successfully install boost (1.53) with

./bootstrap.sh --prefix=$HOME/anaconda_install/ --with-python=/anaconda/bin/python2.7
./b2 link=shared 
./b2 link=shared install

But then boost has the same problem. They don't properly set the install name of libboost_python.dylib either. If this is the only boost installation on your system, then you are probably ok. But since I have quite a few different versions on mine, I then had to do

install_name_tool -id $HOME/anaconda_install/lib/libboost_python.dylib $HOME/anaconda_install/lib/libboost_python.dylib

Then I was able to install GalSim the normal way using the anaconda python:

scons PYTHON=/anaconda/bin/python PREFIX=$HOME/anaconda_install BOOST_DIR=$HOME/anaconda_install
sudo scons install

Answer 3

Yeah I believe this is related to our underlying RPATH/@dynlib@ logic which we're working on revamping here: https://github.com/conda/conda-build/pull/111. The Linux support is currently implemented, OS X and Windows is on the todo list.