QtConcurrent mapped and progress report

1.9k views Asked by At

I'm using QtConcurrent to do some heavy background image processing and I want to display the image while parts of it are being updated progressively. Each line of the image is computed separately and is passed a functor.

To compute the full image I then have a sequence of item that I pass to QtConcurrent mapped and each line emits a signal when it is done computing

Here is the instantiation of the class Worker:

    //living in the main(gui) thread !
    Worker::Worker(VideoEngine* engine):_engine(engine){
        _watcher = new QFutureWatcher<bool>;
        _watcher->setPendingResultsLimit(200);
        connect(_watcher, SIGNAL(resultReadyAt(int)), this, SLOT(onProgressUpdate(int)));
        connect(_watcher, SIGNAL(finished()), engine, SLOT(engineLoop()));
    }

Here is the slot to report progress:

void Worker::onProgressUpdate(int i){
    if(i < (int)_rows.size() && i%10==0){
         cout << " index = " << i << " y = "<< _rows[i] << endl;
        _engine->checkAndDisplayProgress(_rows[i],i);
    }
}

Now the usage:

void Worker::_computeTreeForFrame(.../*unrelevant args*/){
....
....
    _watcher->setFuture(
                   QtConcurrent::mapped(_sequence,
                   boost::bind(&VideoEngine::metaEnginePerRow,_1,output)));
    }
}

All the signals are emitted but the slot onProgressUpdate gets called only when Qtconcurrent::mapped is done with all the items in the sequence.

When executing it has a huge delay while the sequence is processing and then all slots are executed sequentially afterwards.

I have tried all types of signal/slots connection and none of them changed this behaviour.

Any clue ?

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ EDIT after Shf suggestion +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

The call was made until now in the main(gui) thread. I changed the call to :

_computeFrameWatcher->setFuture(QtConcurrent::run(_worker,&Worker::computeTreeForFrame));

Since _computeTreeForFrame is now executed in another thread, I changed the call to QtConcurrent::mapped to:

_watcher->setFuture(QtConcurrent::mapped(_sequence,
                     boost::bind(&VideoEngine::metaEnginePerRow,_1,output)));
_watcher->waitForFinished();

This results in exactly the same behaviour as before.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ EDIT after Marek R suggestion +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Ok so I made so tests and here is what I observed:

QtConcurrent::map :

  • Doesn't emit the signal resultReadyAt(int)

QtConcurrent::mapped

  • Emits resultReadyAt(int) only when finished

It doesn't matter if the call to the map function is done in a separate thread the same behaviour is encountered.

I also gave a try to the signal progressValueChanged(int) as the Qt progressDialog example suggests. The signal progressValueChanged(int) gets emitted only for 2 lines in the image (the first and last). This is really weird as in the Qt progress dialog example it is emitted smoothly.

I changed a bit the Qt example to launch the map function in another thread than the main thread and it still works well in that case.

The issue must arise from somewhere else.

Maybe the GUI event loop is doing something I don't expect ? I have no clue what.

I will now try QtConcurrent::mappedReduced and report with the results :-)

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ EDIT after giving a try to QtConcurrent::mappedReduced +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

It doesn't work and calls the "reduce" function ONLY when the "map" function is done. In other words it does the same than the previous signal/slots mechanism.

I'm running low in possibilities now

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ EDIT I'm back to a solution as close as the Qt progress dialog example +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Something must be wrong if I can't get the same behaviour than the Qt example.

Here's the code now:

//created in the main thread! (gui)
Worker::Worker(VideoEngine* engine):_engine(engine),_watcher(0){
    _watcher = new QFutureWatcher<void>;
    _watcher->setPendingResultsLimit(200);
    connect(_watcher,SIGNAL(progressValueChanged(int)), _engine, 
                    SLOT(onProgressUpdate(int)));
    connect(_watcher, SIGNAL(finished()), engine, SLOT(engineLoop()));

}

//executed on the main thread
void Worker::computeTreeForFrame(...){
...
_watcher->setFuture(QtConcurrent::map(_sequence,boost::bind(metaEnginePerRow,_1,output)));
...
}

The call to computeTreeForFrame...

...
    _worker->computeTreeForFrame();
...

This call is done in a slot .

It emits the signals for the line 0 and for the last line as told before but doesn't emits anything else.

Shouldn't this do EXACTLY what the Qt example does?

2

There are 2 answers

4
Shf On

It seems, that QtConcurrent::mapped does not put VideoEngine::metaEnginePerRow in another thread, judging by the documentation. If image is processed in the same thread as GUI, then your slots indeed will be executed after processing, no matter what type of connection you select, just as you've described.

The solution is to either run Worker::_computeTreeForFrame (as i understood, your main processing function) in another thread via QtConcurrent::run or to put your Worker object in another thread probably via QObject::moveToThread(). Then, the connection type you should use is Qt::QueuedConnection (or if you will put Worker in another thread before connection, you can connect even with Qt::AutoConnectionor Qt::UniqueConnection, caller and receiver will be in a different threads, so qt will automaticly chose QueuedConnection`)

EDIT:

I'm not sure, but your _watcher = new QFutureWatcher<bool>; is still created in the main thread and if you call

_watcher->setFuture(QtConcurrent::mapped(_sequence,
                 boost::bind(&VideoEngine::metaEnginePerRow,_1,output)));
_watcher->waitForFinished();

would _watcher set GUI thread to wait, in what it was created or thread, where this command is executed. If _watcher->setFuture(QtConcurrent::mapped(_sequence, boost::bind(&VideoEngine::metaEnginePerRow,_1,output))); if the end of a function, is _watcher->waitForFinished(); needed at all? Qt will destroy thread right after it's execution and you set your processing function to run, why wait?

And _computeFrameWatcher should be of QFuture<void*> type.

EDIT2:

Ok, before i give up, i suggest you to test QObject::moveToThread:

before you call _worker->computeTreeForFrame(); , put it in another thread:

QThread *workerThread=new QThread();
_worker->moveToThread();
_worker->computeTreeForFrame();
/* connect _worker's finished signal with workerThread::quit and deleteLater slots */

and all connections within _worker should be DirectConnection and all connections between _worker and main (GUI) thread should be connected with QueuedConnection. Also it's probably good to create new thread in _worker constructor and move it to another thread immediately, this way you can destroy thread in _worker's destructor and don't worry about thread problem's in GUI thread

0
Marek R On

From task description it looks like you should use mappedReduced. Problem is that I don't see a good way to get partial results. One way to overcome this problem is to emit signal form reduce function.

It is possible that this thread may help.