Why yolov8 starts parallel process automatically while start training, validation, and testing in C++

46 views Asked by At

I have implemented YOLOv8 for my application, it is working perfectly fine in python. this python file contains a class named "ProductDetection" so using this call I can do train and test on custom dataset. whenever this class is called in python file it is working properly.

now I want to do the same by calling this class "ProductDetection" in c++ code. for that I make wrapper.pyd and load it to c++ code.I have done all python setup for c++ implementation. in C++ code it is working till train step. at the time of training starts it multiple parallel processes stats and continuous till full cpu memory is used.

here is my python train function:


def Train(self, freeze=0, recipe_name="dummy"):
    with self.lock:
        recipe_set_path = os.path.join(self.recipe_path, str(recipe_name))
        self.model.train(data=self.data_path,
                         epochs=self.epochs,
                         time=0.5,
                         patience=10,
                         batch=self.batch_size,
                         imgsz=self.image_size,
                         cache=False,
                         save=True,  
                         resume=False,
                         name=recipe_set_path,
                         exist_ok=True,
                         device=self.device,
                         workers=0,
                         seed=42,
                         single_cls=True,
                         close_mosaic=0,
                         profile=True,
                         freeze=0,
                         plots=False)
        return

train function in c++:



bool TrainDET(std::string recipe_name, float freez_layer, bool cache, bool profile) {
#ifndef _DEBUG
    PyGILState_STATE gstate;
    gstate = PyGILState_Ensure();
    PyObject* pcache = cache ? Py_True : Py_False;
    PyObject* pprofile = profile ? Py_True : Py_False;

    PyObject* result = PyObject_CallMethod((PyObject*)pInstance, "Train", "(sfOO)", recipe_name.c_str(), freez_layer, pcache, pprofile);
    if (!result) {
        PyErr_Print();
        Py_DECREF(result);
        PyGILState_Release(gstate);
        return false;
    }
    Py_DECREF(result);
    PyGILState_Release(gstate);
#endif
    return true;
}

c++ main function:

int main() {

    Py_Initialize();

    PyEval_InitThreads(); // Initialize Python's Global Interpreter Lock (GIL) for single-threaded execution

    PyInitialize(true);

    GetPyInstance(4, 2, 640, 640, "mywrapper");

    std::string recipe_model_path = "C:\\Users\\Karan\\Documents\\Visual Studio 2015\\Projects\\testwrapper\\testwrapper\\DATA\\Models\\Blisbeat_BlisterTopA\\";

    SetDETPath(recipe_model_path, "C:\\Users\\Karan\\Documents\\Visual Studio 2015\\Projects\\testwrapper\\testwrapper\\DATA\\BASE_DATASET\\data.yaml", "DATA\\DL\\det_model.pt");

    LoadDLModel("dummy");

    TrainDET("Unknown", 0, false, true);
return 0;

in this first I initialize python in cpp, the get instance of python class. then use different function to train and test in my dataset at given path. problem start at "TrainDET("Unknown", 0, false, true);" in this step multiple parallel processes starts in taskmanager as shown in image above.

I tried to run c++ code in single thread by using putenv("OMP_NUM_THREADS=1"); putenv("OPENBLAS_NUM_THREADS=1"); putenv("MKL_NUM_THREADS=1");,tried gstate = PyGILState_Ensure(); to lock thrade, and also tried the same in python code by import threading. so that multiprocess can stop. still no improvement is there.

0

There are 0 answers