Slow performance of QPainter when initialized by QImage object

885 views Asked by At

A bit of context:
I am writing a media player where I have to draw a big point cloud and some more geometric shapes on any given video. The complexity is also that we don't have a defined format of video and it can come in any format. (fps, resolution)
A limitation is that we need to use QT specific components and any external library would be under discussion if there is no option in QT.

I did implemented a QAbstractVideoSurface to read frames and then edit them with dynamic data. I have two specific issues with the later mentioned code:

  1. The point where image is loaded into QPainter, it becomes very slow depending upon the resolution of the image. At low resolution the performance is just as good as real-time but at high resolution its really bad.

First of all, I know this is a known issue and widely discussed but none of the answers I found are suitable for me as all of them end up in some case specific solution.

  1. Even in the cases where i get video drawn at almost real-time, the rectangles I am drawing are missing edges frequently when displayed as video. A pure dump of images show every pixel properly plotted.

I must accept that I really did not knew anything about media processing until I had this task and since everything is to happening on the fly, I didn't had time to read much of text. However, if you point me to the correct document, I would go through it from begging to end. :)

Now below is my code for present method of QAbstractVideoSurface implementation:

bool VideoFrameGrabber::present(const QVideoFrame &frame)
{
    if (frame.isValid())
    {
        QVideoFrame cloneFrame(frame);
        cloneFrame.map(QAbstractVideoBuffer::ReadOnly);

        qDebug()<<"1 "<<QTime::currentTime().msecsSinceStartOfDay(); 
        const QImage image(cloneFrame.bits(),
                           cloneFrame.width(),
                           cloneFrame.height(),
                           QVideoFrame::imageFormatFromPixelFormat(cloneFrame.pixelFormat()));
        qDebug()<<"2 "<<QTime::currentTime().msecsSinceStartOfDay();
        imageEdited = image;

        qDebug()<<"3 "<<QTime::currentTime().msecsSinceStartOfDay();
        QPainter p(&imageEdited);

        qDebug()<<"4 "<<QTime::currentTime().msecsSinceStartOfDay();
        if(rectMetaData.count() > 0){
            while(rectMetaData.at(rectSequence).frameNumber == frameSequence && rectSequence < rectCount-1)
            {
                if(rectMetaData.at(rectSequence).fillStatus){                        p.setBrush(QBrush(QColor(rectMetaData.at(rectSequence).color)));
                } else {
                    p.setPen(rectMetaData.at(rectSequence).color);
                }    
                p.drawRect(rectMetaData.at(rectSequence).x1,
                           rectMetaData.at(rectSequence).y1,
                           rectMetaData.at(rectSequence).x2,
                           rectMetaData.at(rectSequence).y2);
                rectSequence++;
            }
        }
        qDebug()<<"5 "<<QTime::currentTime().msecsSinceStartOfDay();
        if(circleMetaData.count() > 0){
            while(circleMetaData.at(circleSequence).frameNumber == frameSequence && circleSequence < circleCount-1)
            {
                if(circleMetaData.at(circleSequence).fillStatus)
                {
p.setBrush(QBrush(QColor(circleMetaData.at(circleSequence).color)));
                } else {
                    p.setPen(circleMetaData.at(circleSequence).color);
                }

                p.drawEllipse(circleMetaData.at(circleSequence).centerX,
                              circleMetaData.at(circleSequence).centerY,
                              circleMetaData.at(circleSequence).radiusX,
                              circleMetaData.at(circleSequence).radiusY);
                circleSequence++;
            }
        }
        qDebug()<<"6 "<<QTime::currentTime().msecsSinceStartOfDay();
        p.end();
        //imageEdited.save(QString("frame"+QString::number(frameSequence)+".jpg"));
        frameSequence++;
        cloneFrame.unmap();
    }

    if (surfaceFormat().pixelFormat() != frame.pixelFormat()
            || surfaceFormat().frameSize() != frame.size()) {
        setError(IncorrectFormatError);
        stop();

        return false;
    } else {
        currentFrame = QVideoFrame(imageEdited);
        // check by reconverting above to image if it is lost in this conversion
        // else it is a rendering problem
        widget->repaint(targetRect);
        return true;
    }
}

The Output of debugs in two different video case is:

30 FPS, 1920*1080 pixels
1  68862821
2  68862822
3  68862822
4  68862892
5  68862893
6  68862893
25 FPS, 1280*720 pixels
1  69319622
2  69319622
3  69319623
4  69319653
5  69319654
6  69319654

Please point me to some solution which is suitable for these two problems.

Above performance is captured with Windows 10 64 bit / Intel I5 2nd Gen 2.3 Gig 2 Cores / 8 gig RAM is someone wants to compare.

0

There are 0 answers