qtimer and opencv running slow

Question

I'm writing a code that uses QTimer to trigger calls to opencv videoCapture to read video frames. I usually read large chunks of videos so, wondering is there any other way to accelerate the process.

here a snapshot of my code that uses QTimer :

timer = new  QTimer();
timer->setTimerType(Qt::PreciseTimer);
connect(timer, SIGNAL(timeout()), this, SLOT(read_shape_params()));

//in a loop stop timer and setup the next video stream then start

void next(){

  timer->stop();

   stream = new video_stream_reader();
   stream->setColorGray(grayImage);
   stream->set_begin_end(begin_at,end_at);
   stream->open(video_base_path+video_path);

   timer->start(0);
}

void shape_param_finder::read_shape_params(){
   Mat frame;
   frame = stream->read_frame();
}

Mat video_stream_reader::read_frame(){
   Mat frame;
   bool bSuccess = capture->read(frame);
   return frame;

}

Answer 1

It has little to do with QTimer. But

timer->start(0);

is an issue. Your video, you input camera has a frame per second, which means a period to produce frame. For instance, 25fps means you will get a new frame every time period, 40ms in this case.

Short answer: Without a proper hardware synchronization, set the timer timeout to 1000 / expected fps.

Long answer:

A timer with timeout = 0 will schedule a read_shape_params as fast as the framework can. which means that the performance bottleneck end up being capture->read(frame);, assuming the other part of the code (display, etc..) work perfectly.

There are 3 cases about capture->read(frame):

1. It takes more time that the resolution that time period : You can't do anything. It will be slow.
2. It is exact same time. This is the sweet spot. This is also very unlikely.

3. It takes less time that the resolution that time period : Should be good right? Wrong. You read the same image multiple times. Which means at best you waste cpu resource. At worst, things start to behave like in case 1 from your point of view. How is that? Let say it take 30 ms to present a frame (read and show, I assume you do it linearly).

   Read 1 : 30 ms, frame 1
   Read 2 : 30 ms, frame 1 // wasted read, the other party has not updated the frame yet
   Read 3 : 30 ms, frame 2 // your second frame has 60 ms latency, not 40 ms
   Read 4 : 30 ms, frame 3 // frame 3 has 120 ms latency, sweet spot.
   Read 5 : 30 ms, frame 3 // wasted read
   Read 6 : 30 ms, frame 4 // frame 3 has 120 ms latency, sweet spot.
   

If on top of that you keep queuing item for display, and display is slow, your perceived fps will be lower.

• You need to benchmark the execution time of video_stream_reader::read_frame to be sure.
• You also need to benchmark whichever code take care of displaying the image. I suspect there is a bottleneck there.