I'm currently working on a project which involves motion compensation.
At this point I have the input image in cv::Mat input;
cv::resize( input, input_resized, cvSize(64,48), 12, 12, CV_INTER_AREA);
input_resized.convertTo(input_float, CV_32F); //this might be redundant while cv::rezise dst should be 32F.
Now the motion vectors are stored in a 2 channel cv::Mat motion and it's stored like this:
// w -image.width , h-image.height
X0 Y0 X1 Y1 X2 Y2 ... Xw-1 Yw-1
Xw Yw Xw+1 Yw+1 Xw+2 Yw+2 ... X2w-1 Y2w-1
...
X(h-1)w Y(h-1)w ............. X(h-1)(w-1)Y(h-1)(w-1)
So if I use :
std::vector<cv::Mat> motionvect(2);
cv::split(motion, motionvect);
I will get X in motionvect(0) and Y in motionvect2(1).
Furthermore I've tried:
std::vector<cv::Mat> rgbChannels(3);
cv::split(input_float, rgbChannels);
cv::remap(rgbChannels[0], outChannels[0], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));
cv::remap(rgbChannels[1], outChannels[1], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));
cv::remap(rgbChannels[2], outChannels[2], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));
cv::merge(outChannels, input_with_motion);
The results I get after this are very strange and somebody told me it is because of the Y vector. Aparently if X is like this:
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
Then Y should be :
1 6 11
2 7 12
3 8 13
4 9 14
5 10 15
Is that correct? Sort of transpose matrix. If it is true, is there any function that will change the motiovect[2] to look like this or should I do it for each individual component?
Also, for the final step I would need to apply the remap at a different time in my process. Something like this:
Function1 ( (float*) input_float.data , (float*) motion.data, (unsigned char*) out.data );
Function2 ( float* input, float* R, float* G, float* B);
Function3 ( float* R, float* G, float* B, float* W, float* Z);
where R, G, B, W, Z are float* types and they are allocated with
R = (float*) malloc (frame_size * sizeof(float) );
I need to apply motion compensation to float* A and float* B, but these are not allowed inputs to remap. They have to be InputArray and I can't find any method to convert float* to any accepted InputArray type.
I know I'm not very organized in thoughts, but I hope you'll understand at least one of my questions. 1. How to apply remap if I have the motion vectors stored in a cv::Mat like x0 y0 x1 y1 .. ? 2. Can I convert float* to an accepted input for cv::remap ?
For your #2, you can do this to get float values from pointer into some Mat: