Python line_profiler not finding module

Question

I recognize that this is an installation failure on my part, and I'm sorry to lay this uninteresting and inconsequential question at your feet, but for the life of me I can't manage to figure out what is going wrong and I've run out of ideas. I'm hoping someone will be able to quickly point out the obvious.

I am trying to profile a python script (using Kern's line_profiler), and the script needs to load the netCDF4 module. I have installed both line_profiler and netCDF4 with pip. Both are reported as present and updated when I queue pip for the list of installed packages.

Without using the profiler, my script runs without problems, meaning that the netCDF4 module is loaded properly. However, if I run 'kernprof -l -v myscript.py' from the "myscript" directory, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/kernprof", line 9, in <module>
   load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')()
  File "Library/Python/2.7/site-packages/kernprof.py", line 221, in main
   execfile(script_file, ns, ns)
  File "myscript.py", line 5, in <module>
   from netCDF4 import Dataset
ImportError: No module named netCDF4 

I am running Python from an installation at /opt/local/bin/python, which is listed first in my PATH.

So, in any case, if the default Python version that I have set is the same as that which appears first in my PATH, and that default version is able to access the netCDF4 module, why isn't line_profiler?

Answer 1

kernprof has a shebang that redirects to the default python install which doesn't have all the required modules.

You can force the use of your "complete" python install by doing:

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/‌​Resources/Python.app‌​/Contents/MacOS/Pyth‌​on /usr/local/bin/kernprof -l -v myscript.py

So shebang is ignored, and you run the profiler with the version of python containing all the required packages.