My goal is to calculate the angle between a plane and the direction of view of a camera. I know how to calculate the angle but I don't know how to get the direction of view in the form of a line.
For the camera a homogenous transformationmatrix is known describing its position and rotation within the global three dimensional coordinate system. I think I have to first eliminate the translation part by putting the camera onto the origin point, but how do I proceed with the roation to calculate the direction?
Your matrix brings a point in the referential of the world into the referential of the camera. You can inverse the matrix. The optic axis of the camera is z, so multplying the inverse matrix with the column vector [0 0 1 0] will give you the vector of the direction of the optic axis of the camera in the referential of the world. The right column of the inverse matrix will give you the position ( x y z) of the camera in the world.