I am a robotics engineer and I am no expert in computer vision or visual servoing. This is one of my fitrst experiences with cameras.
Anyway, I would like to get an information of bearing associated to a certain pixel in an image.
The setup is the following. I have a camera with resolution 1040x776 which I calibrated. I have some object in my image which I am able to detect and associate to a point in the image. The pixels are expressed in u,v (I guess) so these points go between 0 and 1040 on the u axis (horizontal) and between 0 and 776 in the v axis.
Let's assume that u,v = 0,0 is at the center of the image (even if it is in the upper-left corner. But this is not a problem).
What I want is a unit vector that goes from the center of the camera to the aforementioned pixel. So for example if the pixel is in the center of the image the resulting vector should be (1,0,0).
I think I found already a reply on stackexchange but I am not 100 % sure is what I need : OpenCV: Calculate angle between camera and pixel
Recipe:
1 - Calibrate the camera, obtaining the camera matrix K and distortion parameters D. In OpenCV this is done as described in this tutorial.
2 - Remove the nonlinear distortion from the pixel positions of interest. In OpenCV is done using the undistortPoints routine, without passing arguments R and P.
3 - Back-project the pixels of interest into rays (unit vectors with the tail at the camera center) in camera 3D coordinates, by multiplying their pixel positions in homogeneous coordinates times the inverse of the camera matrix.
What do you think?
Thanks in advance for your attention.
Assuming the camera is positioned at the origin and is looking down the positive Z axis, the image plane is the XY plane with positive X pointing right and positive Y pointing down, the position of a pixel in 3D is:
(pixel.x - imageCentreX, pixel.y - imageCentreY, focalLength)You then need to multiply this by the camera's 3x3 rotation matrix to get the 3D direction vector:
rotation * (pixel.x - imageCentreX, pixel.y - imageCentreY, focalLength)You can then normalise this vector by dividing by the length to retrieve the unit direction vector to a pixel.