So, I have a Camera class, witch has vectors forward, up and position. I can move camera by changing position, and I'm calculating its matrix with this:
glm::mat4 view = glm::lookAt(camera->getPos(),
camera->getTarget(), //Caclates forwards end point, starting from pos
camera->getUp());
Mu question is, how can I rotate the camera without getting gimbal lock. I haven't found any good info about glm quaternion, or even quaternion in 3d programming
glm makes quaternions relatively easy. You can initiate a quaternion with a glm::vec3 containing your Euler Angles, e.g glm::fquat(glm::vec3(x,y,z)). You can rotate a quaternion by another quaternion by multiplication, ( r = r1 * r2 ), and this does so without a gimbal lock. To use a quaternion to generate your matrix, use glm::mat_cast(yourQuat) which turns it into a rotational matrix.
So, assuming you are making a 3D app, store your orientation in a quaternion and your position in a vec4, then, to generate your View matrix, you could use a vec4(0,0,1,1) and multiply that against the matrix generated by your quaternion, then adding it to the position, which will give you the target. The up vector can be obtained by multiplying the quaternion's matrix to vec4(0,1,0,1). Tell me if you have anymore questions.
For your two other questions Assuming you are using opengl and your Z axis is the forward axis. (Positive X moves away from the user. )
1). To transform your forward vector, you can rotate about your Y and X axis on your quaternion. E.g glm::fquat(glm::vec3(rotationUpandDown, rotationLeftAndRight, 0)). and multiply that into your orientation quaternion.
2).If you want to roll, find which component your forward axis is on. Since you appear to be using openGL, this axis is most likely your positive Z axis. So if you want to roll, glm::quat(glm::vec3(0,0,rollAmt)). And multiply that into your orientation quaternion. oriention = rollquat * orientation.
Note::Here is a function that might help you, I used to use this for my Cameras. To make a quat that transform 1 vector to another, e.g one forward vector to another.