OpenGL - Cannot get cumulative rotation around correct axis and origin

Question

I am implementing an Arcball rotation in an existing project I wrote several years ago, using OpenTK & C#, and have got stuck at the final hurdle.

This is 'old-style' non-shader OpenGL. I am confident that the Arcball rotation is working correctly, the problem is just applying the resulting matrix. It should be fairly straightforward but it isn't working out that way.

I get the Arcball rotation as a quaternion (qCurrent) then convert that to a matrix. I have then tried a couple of approaches:

1. Simply apply that as an additional rotation to the existing scene:

   GL.PushMatrix();
   Matrix4 arcball_rot = Matrix4.CreateFromQuaternion(qCurrent);
   GL.MultMatrix(ref arcball_rot);
   ... render objects in scene
   GL.PopMatrix();
   

This applies the correct rotation around the correct origin (the camera target), but with respect to the world coordinate system, not the viewer. This makes sense because I am effectively applying the rotations in the wrong sequence.

2. Apply the rotations in sequence, with the arcball rotation first, which means starting from scratch. Here, rtn_complete is a matrix storing the modelview matrix as initialised, before the Arcball rotation:

   Matrix4 modelview = Matrix4.LookAt(camera.camerapos, camera.cameratarget, camera.cameraup);
   rtn_complete = modelview;
   

Then...

Matrix4 mm = Matrix4.CreateFromQuaternion(qCurrent) * rtn_complete;
GL.LoadIdentity();
GL.MultMatrix(ref mm);

This applies the correct rotation wrt the viewer, but around the wrong centre of rotation. It is a point a long way away from the camera target.

Obviously this approach needs a forward/backward translation either side of the rotation, but I have tried pretty much every possible combination of these and none of them work.

Camera and target positions are as follows :-

camera.camerapos = (-51.3, -67.9, 37.7), and

camera.cameratarget = (0.0, 0.6, 7.3)

When I add translations as below (and countless other permutations) I still get the scene rotating around the wrong origin.

GL.LoadIdentity();
GL.Translate(camera.cameratarget);
GL.MultMatrix(ref arcball_rot);
GL.Translate(-camera.cameratarget);
GL.MultMatrix(ref rtn_complete);

My feeling is that the root of the problem is likely to be the translation applied by using LookAt, which I am not taking into account when doing the above transformations.

However, when I check the rtn_complete matrix (which is the modelview matrix following the LookAt) the fourth column does not contain a translation. The matrix looks like this:

0.80, 0.20, -0.56, 0.00
-0.60, 0.27, -0.75, 0.00
0.00, 0.94, 0.34, 0.00
0.38, -7.02, -92.8, 1.00

I would have expected to a see a translation here.

EDIT 1:

Found it eventually. I was on the right track with my suspicion about the translation resulting from using Matrix4.LookAt().

The way I had listed the matrices for debugging resulted in a transpose, so the translation was there for me to see but I was missing it but it was in the fourth row not the fourth column. The translation is (0.38, -0.72, -92.8) Applying this translation on either side of the Arcball rotation results in the expected rotation behaviour, initially.

GL.LoadIdentity();
GL.Translate(0.38, -7.02, -92.8);
GL.MultMatrix(ref mm);
GL.Translate(-0.38, 7.02, 92.8);
GL.MultMatrix(ref rtn_complete);

EDIT 2:

Having worked out the above I am very close, but it's still not quite right when I move somewhere else in the scene. Again I have got a couple of problems, and can solve one or the other but not both simulataneously.

Because of the Matrix4.LookAt I separated out the translation and the rotation components of the rtn_complete matrix (respectively 'rtn_complete_trans' and 'rtn_complete_rot').

If I do this, the rotations are spot on:

GL.LoadIdentity();
GL.Translate(rtn_complete_trans); // modelview translation component
GL.Translate(camera.cameratarget); // translate to rotation centre
GL.MultMatrix(ref arcball_rot); // ongoing Arcball rotation
GL.MultMatrix(ref rtn_complete_rot); // modelview rotation component
GL.Translate(-camera.cameratarget); // translate back from rotation centre

But there is an unwanted translation each time the view is initialised. If I rotate, release and repeat a few times the object gradually moves off screen.

If I change the position of the second camera translation this unwanted shift no longer happens, but the centre of rotation is off:

GL.LoadIdentity();
GL.Translate(rtn_complete_trans); // modelview translation component
GL.Translate(camera.cameratarget); // translate to rotation centre
GL.MultMatrix(ref arcball_rot); // ongoing Arcball rotation
GL.Translate(-camera.cameratarget); // translate back from rotation centre
GL.MultMatrix(ref rtn_complete_rot); // modelview rotation component

Can anyone explain?