I am trying to calibrate my camera from a single image without a chessboard. To this end, I used the following application (http://w3.impa.br/~zang/qtcalib/index.html) which uses Tsai's method to calibrate the camera.
The application returns intristic, rotation and translation matrix.
After that, I would like to calculate world coordinates from the 2d image. Thus, feeding my program with 2d coordinates (x,y) and the matrices (intristic, rotation, translation) I would like to get the 3D coordinates (X,Y,Z).
I followed the instruction of some relative threads (Get 3D coordinates from 2D image pixel if extrinsic and intrinsic parameters are known) but the results were not as I was expecting. Plus I have no idea where is my origin (0,0,0).
What am I doing wrong?
The projective transformation is
x = K * T * X
where
K : intrinsic
T : extrinsic (your rotation and translation). Transforms X into the camera coordinate system.
x : pixel coordinate in image
X : 3D coordinate
Projecting a 2D pixel to 3D is comparable to a line in 3D space, thus there are an infinite number of 3D points possible. To get an ordinary 3D coordinate, you have to choose a depth value.
You can decide your origin of the resulting 3D coordinate:
Camera coordinate system, [0;0;0] is at the camera center:
X' = inv(K) * x
Your chessboard or Tsai-grid coordinate system, [0;0;0] is usually at one of the corners:
X' = inv(T) * inv(K) * x
However, as already mentioned, X' is more like a line. You need to scale it to get a proper 3D coordinate.
It's sometimes difficult to know what a affine transformation is doing. In case it doesn't do what you want, a quick engineering approach could be to try it with its inverse :)
If that does not help you, you can pass your matrices.