cmake to bazel - cppyy - shared libraries and includepath

Question

I wanted to use the macro cppyy_add_bindings from FindCppyy.cmake. I have to add this in a cpp/python using bazel project. How do I handle the library dependencies? Do I have to load all libraries explicitly in the py file, or is it possible to have bazel do this for me?

Answer 1

I'm not familiar with Bazel, but can perhaps shed some light of what cppyy needs/expects. In particular, FindCppyy.cmake was written with a specific type of use in mind that may not fit everyone.

What cppyy needs is for the linker symbols of C++ code used from Python to be available at run-time. How they are made available does not matter: as long as they are, it will work. Here are a couple of options:

• Link dependent libraries with another library that will be loaded. This is typical when using dictionaries (https://cppyy.readthedocs.io/en/latest/utilities.html#dictionaries): simply link all the relevant library with the dictionary shared library, then only load the dictionary, the rest will be pulled in by the dynamic linker. This is the most common method AFAIK, precisely b/c the dependencies are easy to figure out as the build tool already has them and already knows how to link.

• Add them to the .rootmap file (https://cppyy.readthedocs.io/en/latest/utilities.html#class-loader): when a class is not found, the .rootmap files (which should sit in some path reachable by LD_LIBRARY_PATH on Linux/Mac or PATH on Windows) are searched and if the class is found there, the libraries specified are all loaded. This would have my preference (b/c class-lazy, not project-lazy) and is often used together with a dictionary file as above.

• Use cppyy.load_library explicitly within a Python module. Personally, I do not prefer this unless the code is truly well compartimentalized, and/or the project is small (a handful of libraries at most), as this approach is not lazy.

• Use another means such as ctypes.CDLL. This will work, but ctypes.CDLL behaves very differently from one OS to the next, putting more work on the developer, so I would not recommend it. It is also not lazy.

The cppyy cmake macros mentioned assume that the goal is one python module per project, and that as laziness granularity, the project (as opposed to individual classes) is good enough. If that fits your purposes, that I'd go for the first option above: generate a dictionary from all headers for (each part of) the project, and get the list of libraries for it from Bazel, then when building the generated dictionary code, simply link with it. Finally, rely on auto-loading, or simply load_library that one dictionary shared library.

Now, if I take your question completely literally (don't know whether I should), then there may be an option for Bazel to load libraries explicitly into the process, e.g. startup? If so, that would work as long as the libraries are loaded with the RTLD_GLOBAL flag (Linux, Mac; this is common), or have the relevant symbols exported (Windows; also common, but requires more care).