Linking C++ object files embedded with Python

Question

I have a C++ constructor file (formatting_SQ.cpp) of a header file formatting_SQ.h which I want to link to other constructor files of header files (neat.cpp nnode.cpp link.cpp etc...-> neat.h nnode.h link.h) in order to have formatting_SQ.o.

Then, I want to link my main.cpp file with this formatting_SQ.o file. The problem is: formatting_SQ is embedded with python, and as far as my understanding goes, C++ embedded with Python needs the compiling flag -lpython3.6m on Linux: such flag requires a reference to a main() function, which I don't have in formatting_SQ.cpp because it's a constructor file meant to be an object file.

So I first tried to create object files for each constructor file and then link everything together at once:

g++ -c -O3 -Wall -fPIC -fopenmp -std=c++14 -lstdc++ `python3 -m pybind11 --includes` *.cpp 
g++ -o the_executable neat.o nnode.o link.o trait.o gene.o network.o innovation.o organism.o species.o genome.o population.o formatting_SQ.o main.o -fopenmp -O3 -Wall -fPIC `python3 -m pybind11 --includes` -lpython3.6m

Here comes my first question: Are these command right or is there eventually a compilation flag missing ? This gives me a segmentation fault as I try to execute ./the_executable.

Then, I tried to compile formatting_SQ.cpp independently with all other constructor files, but as expected, this doesn't work because there is no reference to main in formatting_SQ.cpp.

g++ -o temp_formatting neat.o nnode.o link.o trait.o gene.o network.o innovation.o organism.o species.o genome.o population.o formatting_SQ.o -fopenmp -O3 -Wall -fPIC `python3 -m pybind11 --includes` -lpython3.6m

So here comes my second question: how could I create a python embedded object file linking formatting_SQ.cpp with all other constructor files without having this undefined reference to main error ?

formatting_SQ.cpp

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <pybind11/stl.h>
#include <Python.h>
#include <omp.h>
#include "formatting_SQ.h"
#include "neat.h"
#include "network.h"
#include "link.h"
#include "nnode.h"
#include "trait.h"
#include "gene.h"
#include "genome.h"
#include "innovation.h"
#include "organism.h"
#include "species.h"
#include "population.h"

namespace py = pybind11;
py::module compile_data = py::module::import("initialize");

main.cpp

#include <pybind11/embed.h>
#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <pybind11/stl.h>
#include <Python.h>
#include "formatting_SQ.h"
#include <omp.h>
    
namespace py = pybind11;
        
int main(int argc, char** argv){
      
....

Answer 1

So after some long hours of research I can conclude that the compilation method is correct but BE EXTREMELY CAREFULL with where you declare your import modules from python, because this was the problem for me

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <pybind11/stl.h>
#include <Python.h>
#include <omp.h>
#include "formatting_SQ.h"
#include "neat.h"
#include "network.h"
#include "link.h"
#include "nnode.h"
#include "trait.h"
#include "gene.h"
#include "genome.h"
#include "innovation.h"
#include "organism.h"
#include "species.h"
#include "population.h"

namespace py = pybind11;
py::module compile_data = py::module::import("initialize"); DON'T DO THIS its wrong !!! 

You must declare your modules locally otherwise there be some conflicts in the namespace as the same module may be imported more than once and this causes the segmentation fault.