How to import or install pre-built python extension module (C++) (i.e. library not compiled via setuptools)?

Question

I have a C++ project for which I am developing a Python interface. For now I am using pybind11 since it seems pretty neat, and has some nice tools for building the extension module with CMake, which is how the main C++ project is built.

Via CMake I managed to get a shared library containing the interface functions to build, however now that I have it I don't know how to tell Python that it exists and make it import-able. I don't want to reconfigure the whole build of the project to be launched via Python (i.e. as described here with setuptools) because it is a big project and I am just providing a Python interface to part of it. So it would be preferable if I could just build the shared library for Python along with the rest of the C++ code, and then just later on run "setup.py install" to do whatever else needs to be done to make the shared library visible to Python.

Is this possible? Or do I need to do some other sort of refactoring, like make the main project build some other pure C++ libraries, which I then just link into the Python extension module library which is built separately via setuptools?

Answer 1

If you need to install just one binary module you can create an simple installer just for that module. Let's assume that you have a binary module foo.so (or foo.pyd if you are working on Windows) that is already built with your cmake-generated build script. Then you can create a simple setup setup script:

from setuptools import setup

setup(
    name='foo',
    version='0.1.2.3',
    py_modules=['foo']
)

Then you need to add MANIFEST.in file to pick your binary module file:

include foo.so

So you need 3 files:

foo.so
MANIFEST.in
setup.py

Now you can do python setup.py install from your Python virtual environment, and your binary module will be installed in it. If you want to distribute your module, then it's better to install Python wheel package and create a .whl file: python setup.py bdist_wheel. Such "wheel" can later be installed with pip command. Note that binary modules must be installed on the same platform and Python version that was used to build those modules.