QT OpenCV multi threading to fetch images and display in GUI Main thread

1.5k views Asked by At

The Basic Idea

I am developing an application with 4 USB cameras, frames must be fetched from all these cameras and displayed in a QLabel at best possible fps.

Fetching frames from all cameras sequentially results in a low frame rate, even though my processor has only 20% utilization.

How I think Frame rate can be improved?

With use of Multiple Threads CPU utilization can be increased, and there could be 4 threads each fetching images from individual cameras, signaling and sending the images back to GUI thread(Main thread) that will in turn display images to appropriate label.

Current Design: Based on https://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/

The Thread Class inherits from QObject with methods:

class fetchCam1 : public QObject
{
    Q_OBJECT

public:
    fetchCam1(cv::VideoCapture cap);
    ~fetchCam1();

public slots:
    void process();
    cv::Mat returnCamFrame();

signals:
    void finished();
};

Its Implementation:

fetchCam1::fetchCam1(cv::VideoCapture cap)
{
    this->cap = cap;
}

cv::Mat fetchCam1::returnCam1Frame()
{
    return src;
}

void fetchCam1::process() {
    // allocate resources using new here
    cap.read(src);
    emit finished();
}

Initiating threads and worker roles in Main(GUI) thread:

    The captureS1..S4, Is the VideoCapture object, also fetchCam1* worker1-4, 
    QThread* thread1-4; are defined as global variables.

thread1 = new QThread;
worker1 = new fetchCam1(captureS1);
worker1->moveToThread(thread1);
connect(thread1, SIGNAL(started()), worker1, SLOT(process()));
connect(worker1, SIGNAL(finished()), this, SLOT(updateCam1()));
connect(worker1, SIGNAL(finished()), worker1, SLOT(process()));
thread1->start();

thread2 = new QThread;
worker2 = new fetchCam1(captureS2);
worker2->moveToThread(thread2);
connect(thread2, SIGNAL(started()), worker2, SLOT(process()));
connect(worker2, SIGNAL(finished()), this, SLOT(updateCam2()));
connect(worker2, SIGNAL(finished()), worker2, SLOT(process()));
thread2->start();

thread3 = new QThread;
worker3 = new fetchCam1(captureS3);
worker3->moveToThread(thread3);
connect(thread3, SIGNAL(started()), worker3, SLOT(process()));
connect(worker3, SIGNAL(finished()), this, SLOT(updateCam3()));
connect(worker3, SIGNAL(finished()), worker3, SLOT(process()));
thread3->start();

thread4 = new QThread;
worker4 = new fetchCam1(captureS4);
worker4->moveToThread(thread4);
connect(thread4, SIGNAL(started()), worker4, SLOT(process()));
connect(worker4, SIGNAL(finished()), this, SLOT(updateCam4()));
connect(worker4, SIGNAL(finished()), worker4, SLOT(process()));
thread4->start();

Each of which invoke:

void MainWindow::updateCam1()
{
    cv::Mat src = worker1->returnCamFrame();
    QPixmap pixg = convertFromMat(src);
    ui->label_4s1->setPixmap(pixg);
}
void MainWindow::updateCam2()
{
    cv::Mat src = worker1->returnCamFrame();
    QPixmap pixg = convertFromMat(src);
    ui->label_4s2->setPixmap(pixg);
}
void MainWindow::updateCam3()
{
    cv::Mat src = worker1->returnCamFrame();
    QPixmap pixg = convertFromMat(src);
    ui->label_4s3->setPixmap(pixg);
}
void MainWindow::updateCam4()
{
    cv::Mat src = worker1->returnCamFrame();
    QPixmap pixg = convertFromMat(src);
    ui->label_4s4->setPixmap(pixg);
}

Please help me identify what I am doing wrong? The above code does start and load first few frames, after which the program crashes with no errors whatsoever.

Also if only one thread is used to fetch images from one camera, the above code works great.

What would be the right approach to implement this? Thank you.

Update The Crash:

I believe that the frames for all 4 cameras are being fetched quite fast, and that's generating way too many signals for the Main Thread to handle, which causes an overflow causing the crash. I believe some kind of synchronization can help but how can I go about it?

0

There are 0 answers