Qt: Capturing images continuously from camera fills the memory

Question

I am trying to capture images continuously so I can send them using UDP. I am doing this to implement a live video streaming program.

The code below captures images continuously and assigns images to QGraphicsScene so I can test if images play like a video. But when I run the program my computer freezes after couple seconds even though I delete the pointers. How can I fix this problem ?

#include "mainwindow.h"
#include "ui_mainwindow.h"
#include <QThread>

MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);
scene = new QGraphicsScene(this);
ui->graphicsView->setScene(scene);
cam = new QCamera;
cam->setCaptureMode(QCamera::CaptureStillImage);

viewfinder = new QCameraViewfinder;
viewfinder->show();
QCameraImageCapture *cap = new QCameraImageCapture(cam);
cap->setCaptureDestination(QCameraImageCapture::CaptureToBuffer);

cam->setViewfinder(viewfinder);


QObject::connect(cap, &QCameraImageCapture::imageCaptured, [=] (int id, QImage img) {

     while(true){
        QByteArray *buf = new QByteArray;
        QBuffer *buffer=new QBuffer(buf);
        buffer->open(QIODevice::WriteOnly);
        img.save(buffer, "BMP");
        QPixmap *pixmap = new QPixmap();
        pixmap->loadFromData(buffer->buffer());
        scene->addPixmap(*pixmap);
        delete buf;
        delete buffer;
        delete pixmap;

        QThread::sleep(0.0416);
        cap->capture();

     }

});

QObject::connect(cap, &QCameraImageCapture::readyForCaptureChanged, [=] (bool state) {
   if(state == true) {
       cam->searchAndLock();
       cap->capture();
       cam->unlock();
   }
});
cam->start();
}

MainWindow::~MainWindow()
{
    delete ui;
}

Answer 1

Your implementation of your goal to "... capture images continuously to send them ..." via the slow QCameraImageCapture in a loop calls for trouble (lagging, UI blocking).

Better use the following subclass.

// subclass from QAbstractVideoSurface
class AltVideoSurface: public QAbstractVideoSurface{
    Q_OBJECT
public:
    AltVideoSurface(QObject * parentLabel=NULL) : QAbstractVideoSurface(parentLabel)
    {
        this->parentLabel = (QLabel*)parentLabel;
    }

    QLabel * parentLabel;

    QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType type) const {
        return QList<QVideoFrame::PixelFormat>() << QVideoFrame::Format_RGB24;
    }

    // will called on every new video frame
    bool present(const QVideoFrame& frame)
    {
        if ( frame.isValid() ) {
        
            // clone frame
            QVideoFrame cloneFrame(frame);
            // map frame's memory from video to cpu
            cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
            
            // have a full resolution image
            QImage imageFull(cloneFrame.bits(),
                             cloneFrame.width(),
                             cloneFrame.height(),
                             QVideoFrame::imageFormatFromPixelFormat(cloneFrame.pixelFormat()));
                             
            // release cpu mapping
            cloneFrame.unmap();
        
            // perhaps make an additional scaled image
            QImage imageSmall = imageFull.scaled( 640, 480, Qt::KeepAspectRatio);
            // render small image in UI viewlinder (here a QLabel)
            parentLabel->setPixmap(QPixmap::fromImage(imageSmall));
        
            return true;
        } 
        
        return false;
    }
};


mainwindow h-file
~~~~~~~~~~~~~~~~~
...
    QScopedPointer<QCamera> m_camera;
    AltVideoSurface*        m_surface;
...
    
    
mainwindow cpp-file 
~~~~~~~~~~~~~~~~~~~
Call the subclassed QAbstractVideoSurface within your camera init method.
...

    // init camera
    m_camera.reset(new QCamera(QCameraInfo::defaultCamera()));

    // set video mode
    m_camera->setCaptureMode(QCamera::CaptureVideo);
    
    // the subclassed QAbstractVideoSurface as view (here a QLabel) gets full resolution images
    m_surface = new AltVideoSurface(ui->labelView);
    
    // connect camera, viewfinder and subclassed surface
    m_camera->setViewfinder(m_surface);
    
... 

Answer 2

You should use imageAvailable() instead of imageCaptured signal.

Here is an example :

connect(cap, &QCameraImageCapture::imageAvailable, [=] (int id, QVideoFrame v ) {

    if (v.isValid()) {
        if(v.map(QAbstractVideoBuffer::ReadOnly)) {

            QByteArray bitsVideo( (char *) v.bits(), v.mappedBytes() );

            //call to your send raw data function (over UDP) : 
            //datagram will contain frame details e.g : [ width, hight, byteperline, format, rawdata ]

            sendDataOverUDP( v.width(), v.height(), 
                             v.bytesperLine(), 
                             QVideoFrame::imageFormatFromPixelFormat(v.pixelFormat()), 
                             bitsVideo );


        }
    }
});

The others side, the server or other clients will create the image from raw data received as follows:

    void onDataImageReceived( int width, int height, 
                          int bytePerLine, 
                          QImage::Format fmt, 
                          QByteArray bitsVideo )
{

    QImage img ((uchar *)bitsVideo.data(), width, height, bytesPerLine, fmt);
    //do something with img ...

}

Answer 3

I'm not familiar with QCamera and related classes but the lambda you connect the QCameraImageCapture::imageCaptured signal to doesn't look correct. That signal is emitted when a single frame is ready for preview. In your lambda, however, you have...

while(true){
    QByteArray *buf = new QByteArray;
    QBuffer *buffer=new QBuffer(buf);
    buffer->open(QIODevice::WriteOnly);
    img.save(buffer, "BMP");
    QPixmap *pixmap = new QPixmap();
    pixmap->loadFromData(buffer->buffer());
    scene->addPixmap(*pixmap);
    delete buf;
    delete buffer;
    delete pixmap;

    QThread::sleep(0.0416);
    cap->capture();
}

That while loop never exits and will block the Qt event processing loop. Note, also, that the code chunk...

QByteArray *buf = new QByteArray;
QBuffer *buffer=new QBuffer(buf);
buffer->open(QIODevice::WriteOnly);
img.save(buffer, "BMP");
QPixmap *pixmap = new QPixmap();
pixmap->loadFromData(buffer->buffer());
scene->addPixmap(*pixmap);
delete buf;
delete buffer;
delete pixmap;

is overkill and (unless I'm mistaken) basically amounts to...

scene->addPixmap(QPixmap::fromImage(img));

So I think your lambda should be more like (untested)...

[=](int id, QImage img)
{
    scene->addPixmap(QPixmap::fromImage(img));
}