Tracking multipel opject using Optical Flow PyrLK from haar detection result

Question

I am trying to track multiple object (cars) in videos using Optical Flow PyrLK from haar detection result but the problem is some time the haar detection (object) does not appear in the frames ! How can I track an object that the haar detection (object) does not appear in few frames ?? how can I approximate it is place automatically ,,,

I am trying to find the object using Optical flow PyrLK ... but this technique lost when the haar detection (object) does not appear

Any suggestion ideas ??? please.....

Green Rect = Haar Detection result, Red Point = Corners from GoodFeaturestoTrack

this is my code : can you give me solution ??

video >> prev_frame;
Rect roi = Rect(50, 180, 540, 240);
prevROI=prev_frame(roi);
cvtColor(prevROI, gray, CV_BGR2GRAY);
gray.convertTo(prev_img, CV_8UC1);

while(true)
{
    video >> frameROI;
    Rect roi = Rect(50, 180, 540, 240);
    Mat ROI=frameROI(roi);
    cvtColor(ROI, gray, CV_BGR2GRAY); //=====> RGB to Grayscale
    gray.convertTo(imgROI, CV_8UC1);

    vector<Rect> cars;
    Casmobil.detectMultiScale(gray, cars, 1.1, 3,
                              CV_HAAR_DO_CANNY_PRUNING|CV_HAAR_SCALE_IMAGE,
                              Size(0,0));

    for (size_t i = 0; i < cars.size(); i++)
    {
        Rect square = cars[i];
        areax = (cars[i].x + cars[i].width*0.5);
        areay = (cars[i].y + cars[i].height*0.5);

        Point cen_point = Point(areax ,areay);

        rectangle(ROI, square,CV_RGB(0,255,0),2,8,0);
        circle(ROI, cen_point, 3,CV_RGB(255, 0, 0),-2);
    }

    for (int i = 0; i < cars.size(); i++)
    {
        Rect square2 = cars[i];
        cropcars = imgROI(square2);
        ROIOF = ROI(square);

        //prev_frame
        goodFeaturesToTrack(cropmobil, prevframe_corners,maxCorners,
                            qualityLevel,minDistance,Mat(),
                            blockSize,useHarrisDetector,k);
        calcOpticalFlowPyrLK(prev_img, imgROI, prevframe_corners,
                             frame_corners, found, error,
                             Size(win_size, win_size), maxlevel,termcrit);

        Rect box = boundingRect(frame_corners);
        rectangle(ROIOF, box, CV_RGB(0,255,255),2,8,0);
        Boxx = (box.x + box.width*0.5);
        boxy = (box.y + box.height*0.5);
        cen = Point(boxx, boxy);
        circle(ROIF, cen, 3, CV_RGB(0, 0, 255), -1);

        for( int j = 0; j < frame_corners.size(); j++ )
        {
            circle(ROIOF, frame_corners[j], 2, CV_RGB(255, 0, 0), -1);
            circle(ROIOF, prevframe_corners[j], 2, CV_RGB(0, 0, 255), -1);
            line(ROIOF,frame_corners[j], cen, CV_RGB(0, 255, 0),2, 8, 0);
        }

        prev_img = imgROI.clone();

    }

Answer 1

If you're using Haar detection for every frames, so why are you using LK tracking?

In object tracking, since Haar detection is very heavy weight and slow, people usually apply Haar detection in first frame and track the object using other methods.

In your case, using CamShift seems to do the job. So, you do a Haar detection in first frame and let CamShift track the cars.

Note that CamShift algorithm is completely based on image histogram, so in cases where color of object is considerably different from its background, it works well.