Name of Python dynamic library module different than file name

Question

In Python, the name of a module is usually the same as the name of the corresponding file, but I have a problem with a module for which it is not the case:

me@host:/usr/lib/python2.7/dist-packages/paraview$ ls vtkCommonCorePython*
vtkCommonCorePython.x86_64-linux-gnu.so

me@host:/usr/lib/python2.7/dist-packages/paraview$ python -c \
    "import vtkCommonCorePython; print(vtkCommonCorePython.__file__)"
vtkCommonCorePython.x86_64-linux-gnu.so

How does it work?

My problem is that I try to use this module with another Python (/opt/python/2.7.9/bin/python) and it does not find the module:

me@host:/usr/lib/python2.7/dist-packages/paraview$ module load python/2.7.9 
--- Loading module environment: python/2.7.9
------------------------------------------------
me@host:/usr/lib/python2.7/dist-packages/paraview$ python -c \
    "import vtkCommonCorePython; print(vtkCommonCorePython.__file__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named vtkCommonCorePython

Remark: Most of the files in /usr/lib/python2.7/lib-dynload also end with x86_64-linux-gnu.so while the files in /opt/python/2.7.9/lib/python2.7/lib-dynload just end with .so.

Update after cdarke's answer:

cdarke mentions the imp.get_suffixes function, which returns a tuple with suffixes and their meaning for Python:

with /opt/python/2.7.9/bin/python

[('.so', 'rb', 3), ('module.so', 'rb', 3), ('.py', 'U', 1), ('.pyc', 'rb', 2)]

and with /usr/lib/python

[('.x86_64-linux-gnu.so', 'rb', 3), ('.so', 'rb', 3), ('module.so', 'rb', 3), ('.py', 'U', 1), ('.pyc', 'rb', 2)]

This difference explains why I get the error but then it does not seem that these suffixes are set in site.py (as written by cdarke) and there is no function imp.set_suffixes.

So to solve the problem I need to know how are set these suffixes.

Update 2

So it seems that the suffixes are encoded into a binary Python file, meaning that there is no way to modify that after the building of Python. So the cleanest way to solve the problem would be to rebuild the opt Python and thus to understand which options to give to configure to obtain a Python multi-arch as the system Python under Debian Jessie.

Cdarke, thanks a lot for the help.

Answer 1

The .so files are also certainly written in C or C++. In the Python 2 API there is an entry point function where the module is named, so for a module called "example":

PyMODINIT_FUNC initexample(void)
{
    (void)Py_InitModule("example", ExampleMethods);
}

The second parameter to Py_InitModule is the name of a method array - basically a list of the C function names. The .so file is linked with the Python27 run-time library.

So how does Python know which suffixes to use? You can find these out using:

import imp
print imp.get_suffixes()

This will show the valid suffixes for modules on your system (the documentation for the imp module is instructive).

Edit:

After much digging through source code and the like I discovered the relevant lines in configure.ac. There are two macros, VERSION and SOVERSION that are used during the build. It appears to me that the differences are because the pythons have been built in a different way. I think the answer is to build the modules from source on each platform.

The original information I gave about site.py came from http://bioportal.weizmann.ac.il/course/python/PyMOTW/PyMOTW/docs/site/index.html, but I have not tested it.

There is an undocumented imputil.add_suffix() function, but that is clearly not designed to be called by the user.