Changing a C/++ OpenCV program to CUDA from video stabilization program

Question

I am doing a C++ video stabilization/ anti-shaking program which: - Gets points of interest on the frame of reference (using FAST, SURF, Shi-Matoshi or SIFT, might try a few more) - Calculate the Lucas-Kanade Optical flow with calcOpticalFlowPyrLK - Gets the homography matrix - Corrects the shaking image using warPerspective (see code below)

//Calculate the Lucas Kanade optical flow
calcOpticalFlowPyrLK(original, distorted, refFeatures, currFeatures, featuresFound, err);   

//Find the homography between the current frame's features and the reference ones's
if(homographyRansac){
    homography = findHomography(currFeatures, refFeatures, CV_RANSAC); /*CV_RANSAC: Random sample consensus (RANSAC) is an iterative method to
    estimate parameters of a mathematical model from a set of observed data which contains outliers */
}else{
    homography = findHomography(currFeatures, refFeatures, 0);
}


//We use warpPerspective once on the distorted image to get the resulting fixed image
if(multiChannel){
    //Spliting into channels        
    vector <Mat> rgbChannels(channels), fixedChannels;
    split(distortedCopy, rgbChannels);
    recovered = Mat(reSized, CV_8UC3);
    //We apply the transformation to each channel
    for(int i = 0; i < channels; i ++){
        Mat tmp;
        warpPerspective(rgbChannels[i], tmp, homography, reSized);
        fixedChannels.push_back(tmp);
    }
    //Merge the result to obtain a 3 channel corrected image
    merge(fixedChannels, recovered);
}else{
    warpPerspective(distorted, recovered, homography, reSized);
}

If you have any alternative to my stabilization solution, feel free to say so, but it's not this thread's topic.

Since all this takes a lot of time (around 300ms per frame on my i5 computer, so a VERY long time for a 30 min video) I am considering using CUDA to speed things up. I've installed it and go it working, however I'm not sure as to how to proceed next. I've done some test and I know the most time consuming operations are getting the optical flow and the frame correction using respectivly calcOpticalFlowPyrLK and warpPerspective. So ideally, at least at first, I would only use the CUDA versions of these two function, leaving the rest unchanged.

Is this possible? Or do I need to re-write everything?

Thanks

Answer 1

Since OpenCV 3.0, a CUDA implementation of video stabilization is available. It is recommended to use the already available implementation instead of writing your own unless you are sure that your version would be better or faster.

Here is a minimal code demonstrating how to use the OpenCV video stabilization module to stabilize a video.

#include <opencv2/highgui.hpp>
#include <opencv2/videostab.hpp>

using namespace cv::videostab;

int main()
{
    std::string videoFile = "shaky_video.mp4";

    MotionModel model = cv::videostab::MM_TRANSLATION; //Type of motion to compensate
    bool use_gpu = true; //Select CUDA version or "regular" version

    cv::Ptr<VideoFileSource> video = cv::makePtr<VideoFileSource>(videoFile,true);
    cv::Ptr<OnePassStabilizer> stabilizer = cv::makePtr<OnePassStabilizer>();

    cv::Ptr<MotionEstimatorBase> MotionEstimator = cv::makePtr<MotionEstimatorRansacL2>(model);

    cv::Ptr<ImageMotionEstimatorBase> ImageMotionEstimator;

    if (use_gpu)
        ImageMotionEstimator = cv::makePtr<KeypointBasedMotionEstimatorGpu>(MotionEstimator);
    else
        ImageMotionEstimator = cv::makePtr<KeypointBasedMotionEstimator>(MotionEstimator);

    stabilizer->setFrameSource(video);
    stabilizer->setMotionEstimator(ImageMotionEstimator);
    stabilizer->setLog(cv::makePtr<cv::videostab::NullLog>()); //Disable internal prints

    std::string windowTitle = "Stabilized Video";

    cv::namedWindow(windowTitle, cv::WINDOW_AUTOSIZE);

    while(true)
    {
        cv::Mat frame = stabilizer->nextFrame();

        if(frame.empty())   break;

        cv::imshow(windowTitle,frame);
        cv::waitKey(10);
    }

    return 0;
}