Re-establishing new feature points using Matlab's vision functions

Question

I am using Matlab's built in vision functions and pre-made example code to track feature points. In my sample video, the camera pans horizontaly, introducing new objects and scenery to the field of view while previous objects and scenery move out of view.

My problem occurs when trying to identify new feature points while the camera pans across the scene. I am using the "detectMinEigenFeatures" function within the video-step while loop in order to find new feature points after a specified number of frames have been stepped through. However, doing this somehow does nothing towards re-establishing new feature points.

Some quick info: Using GoPro video, sampled down to 720p and saved as an .avi

Code is posted below, and I will be happy to provide anymore information that may help understand or solve this problem.

Thanks!

clc;clear all;close all;

videoFileReader = vision.VideoFileReader('GoProFlyingMidFlightResized.avi');

videoFrame = step(videoFileReader);

%Create Video writer
TrackingVideo = VideoWriter('TrackingVideo.avi');

open(TrackingVideo);

% Detect feature points
points = detectMinEigenFeatures(rgb2gray(videoFrame),'MinQuality',0.04,'FilterSize',3);
% points = detectMinEigenFeatures(rgb2gray(videoFrame));

% Create a point tracker
pointTracker = vision.PointTracker('NumPyramidLevels',7,'MaxBidirectionalError', 8, 'MaxIterations',70,'BlockSize',[5 5]);

% Initialize the tracker with the initial point locations and the initial
% video frame.
points = points.Location;
initialize(pointTracker, points, videoFrame);

videoPlayer  = vision.VideoPlayer('Position',[100 100 [size(videoFrame, 2), size(videoFrame, 1)]+30]);

% Make a copy of the points for transformation between the consecutive feature points
oldPoints = points;
FrameCount=0; %For identifying that new feature points must be obtain

while ~isDone(videoFileReader)
    % get the next frame
    FrameCount=FrameCount+1;
    videoFrame = step(videoFileReader);

    if FrameCount==30  %If 30 frame have stepped though, find new feature points
        disp('help')
        points = detectMinEigenFeatures(rgb2gray(videoFrame),'MinQuality',0.04,'FilterSize',3);
        points = points.Location;
        FrameCount=0;
    end

    % Track the points.
    [points, isFound] = step(pointTracker, videoFrame);
    visiblePoints = points(isFound, :);
    oldInliers = oldPoints(isFound, :);

    if size(visiblePoints, 1) >= 2 % need at least 2 points

        % Estimate the geometric transformation between the old points
        % and the new points and eliminate outliers
        [xform, oldInliers, visiblePoints] = estimateGeometricTransform(oldInliers,   visiblePoints, 'similarity', 'MaxDistance', 10);

        % Display tracked points
        videoFrame = insertMarker(videoFrame, visiblePoints, '+','Color', 'red');

        % Reset the points
        oldPoints = visiblePoints;
        setPoints(pointTracker, oldPoints);
    end

    % Display video frame using the video player
    writeVideo(TrackingVideo,videoFrame);
    step(videoPlayer, videoFrame);
end

% Clean up
release(videoFileReader);
release(videoPlayer);
release(pointTracker);
close(TrackingVideo);

Answer 1

In the if statement you detect new points:

    if FrameCount==30  %If 30 frame have stepped though, find new feature points
        disp('help')
        points = detectMinEigenFeatures(rgb2gray(videoFrame),'MinQuality',0.04,'FilterSize',3);
        points = points.Location;
        FrameCount=0;
    end

Now, inside that same if you have to tell the point tracker about those new points:

setPoints(tracker, points);

Otherwise, your variable points gets overwritten by the next line:

[points, isFound] = step(pointTracker, videoFrame);

This is why you never see the newly detected points.