How should I use cx_freeze with a Macports library?

Question

I'm currently using the Python 3.4 Mac OS X build from Python.org. I'm using a Python module that depends on a library that I built in Macports. The script does not run out-of-the-box:

Traceback (most recent call last):
  File "magnetx.py", line 6, in <module>
    import yara
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so, 2): Library not loaded: /usr/local/lib/libyara.3.dylib
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so
  Reason: image not found

I can fix this if I set an environment variable

export DYLD_FALLBACK_LIBRARY_PATH="/opt/local/lib:$DYLD_FALLBACK_LIBRARY_PATH"

Unfortunately, it does not satisfy cx_freeze. It keeps looking in /usr/local/lib, when it should be looking in /opt/local/lib.

copying /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so -> build/exe.macosx-10.6-intel-3.4/yara.so copying /usr/local/lib/libyara.3.dylib -> build/exe.macosx-10.6-intel-3.4/libyara.3.dylib error: [Errno 2] No such file or directory: '/usr/local/lib/libyara.3.dylib'

I could probably build Python in Macports, but that seems like it should be unnecessary. Any ideas on how to fix this?

Answer 1

On OS X, dependent libraries are referenced using absolute paths. The path that gets copied into your binary depends on the so-called "install name" of the library you link against at build time. In your case, the yara.so does not reference the library you would like it to load. Let's explore a couple of reasons why this could be the case, and a couple of ways to fix that:

• I've verified that libyara.dylib as installed by MacPorts (on my system) has an install name of /opt/local/lib/libyara.0.dylib. Sometimes, build systems that don't use a cross-platform library build tool and don't expect the peculiarities of OS X mess this up (and use relative paths or /usr/local/lib). If this was the case, it would be a bug in the software's build system, which could be manually fixed using install_name_tool(1)'s -id flag (before linking against the library).
• Your copy of yara.so may have been built against a different version of libyara.dylib that resides in /usr/local/lib. That would explain why your yara.so does not contain the correct absolute path to the MacPorts copy of libyara.dylib, but it would also prevent the error you're seeing from happening in the first place, unless you had a copy in /usr/local/lib at build time and deleted it later on. As you've already seen, you can instruct OS X' loader to also search different paths using the DYLD_* series of environment variables. My take on why this doesn't work for cx_freeze is that it doesn't pay attention to the DYLD_* series of variables.
• If you are sure that the copy of libyara.dylib yara.so expects to find in /usr/local/lib is binary-compatible with the one in /opt/local/lib, you can manually modify the library load commands in yara.so to point to the latter path using install_name_tool(1)'s -change old new parameter, e.g. install_name_tool -change /usr/local/lib/libyara.3.dylib /opt/local/lib/libyara.0.dylib /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/yara.so. This is essentially modifying the binary with the change the loader did for you when you set DYLD_FALLBACK_LIBRARY_PATH. Since the library major version numbers seem to be different, this may not be a safe assumption.
• If you don't know whether yara.so is compatible with MacPorts' build of libyara.0.dylib, you can and should recompile yara.so. If the re-compile went right, you should be able to check the library load commands using otool -L yara.so and see the paths beginning with /opt/local in there (provided that otool -D /opt/local/lib/libyara.0.dylib correctly points to itself).

Edit: I've just re-checked and noticed that my MacPorts build's library version number differs from the one your system expects. That sounds a lot like case number 2 to me.