gluProject on NDS?

Question

I've been struggling with this for a good while now. I'm trying to determine the screen coordinates of the vertexes in a model on the screen of my NDS using devKitPro. The library seems to implement some functionality of OpenGL, but in particular, the gluProject function is missing, which would (I assume) allow me to do just exactly that, easily.

I've been trying for a good while now to calculate the screen coordinates manually using the projection matricies that are stored in the DS's registers, but I haven't been having much luck, even when trying to build the projection matrix from scratch based on OpenGL's documentation. Here is the code I'm trying to use:

void get2DPoint(v16 x, v16 y, v16 z, float &result_x, float &result_y)
{
 //Wait for the graphics engine to be ready
 /*while (*(int*)(0x04000600) & BIT(27))
  continue;*/

 //Read in the matrix that we're currently transforming with
 double currentMatrix[4][4]; int i; 
 for (i = 0; i < 16; i++)
  currentMatrix[0][i] = 
  (double(((int*)0x04000640)[i]))/(double(1<<12));

 //Now this hurts-- take that matrix, and multiply it by the projection matrix, so we obtain
 //proper screen coordinates.
 double f = 1.0 / tan(70.0/2.0);
 double aspect = 256.0/192.0;
 double zNear = 0.1;
 double zFar = 40.0;
 double projectionMatrix[4][4] = 
 {
  { (f/aspect), 0.0, 0.0, 0.0 },
  { 0.0, f, 0.0, 0.0 },
  { 0.0, 0.0, ((zFar + zNear) / (zNear - zFar)), ((2*zFar*zNear)/(zNear - zFar)) },
  { 0.0, 0.0, -1.0, 0.0 },
 };

 double finalMatrix[4][4];
 //Ugh...
 int mx = 0; int my = 0;
 for (my = 0; my < 4; my++)
  for (mx = 0; mx < 4; mx++)
   finalMatrix[mx][my] = 
   currentMatrix[my][0] * projectionMatrix[0][mx] + 
   currentMatrix[my][1] * projectionMatrix[1][mx] + 
   currentMatrix[my][2] * projectionMatrix[2][mx] + 
   currentMatrix[my][3] * projectionMatrix[3][mx] ;

 double dx = ((double)x) / (double(1<<12));
 double dy = ((double)y) / (double(1<<12));
 double dz = ((double)z) / (double(1<<12));

 result_x = dx*finalMatrix[0][0] +  dy*finalMatrix[0][1] + dz*finalMatrix[0][2] + finalMatrix[0][3];
 result_y = dx*finalMatrix[1][0] +  dy*finalMatrix[1][1] + dz*finalMatrix[1][2] + finalMatrix[1][3];

 result_x = ((result_x*1.0) + 4.0)*32.0;
 result_y = ((result_y*1.0) + 4.0)*32.0;


 printf("Result: %f, %f\n", result_x, result_y);

} 

There are lots of shifts involved, the DS works internally using fixed point notation and I need to convert that to doubles to work with. What I'm getting seems to be somewhat correct-- the pixels are translated perfectly if I'm using a flat quad that's facing the screen, but the rotation is wonky. Also, since I'm going by the projection matrix (which accounts for the screen width/height?) the last steps I'm needing to use don't seem right at all. Shouldn't the projection matrix be accomplishing the step up to screen resolution for me?

I'm rather new to all of this, I've got a fair grasp on matrix math, but I'm not as skilled as I would like to be in 3D graphics. Does anyone here know a way, given the 3D, non-transformed coordinates of a model's vertexes, and also given the matricies which will be applied to it, to actually come up with the screen coordinates, without using OpenGL's gluProject function? Can you see something blatantly obvious that I'm missing in my code? (I'll clarify when possible, I know it's rough, this is a prototype I'm working on, cleanliness isn't a high priority)

Thanks a bunch!

PS: As I understand it, currentMatrix, which I pull from the DS's registers, should be giving me the combined projection, translation, and rotation matrix, as it should be the exact matrix that's going to be used for the translation by the DS's own hardware, at least according to the specs at GBATEK. In practise, it doesn't seem to actually have the projection coordinates applied to it, which I suppose has something to do with my issues. But I'm not sure, as calculating the projection myself isn't generating different results.

Answer 1

That is almost correct.

The correct steps are:

• Multiply Modelview with Projection matrix (as you've already did).

• Extend your 3D vertex to a homogeneous coordinate by adding a W-component with value 1. E.g your (x,y,z)-vector becomes (x,y,z,w) with w = 1.

• Multiply this vector with the matrix product. Your matrix should be 4x4 and your vector of size 4. The result will be a vector of size4 as well (don't drop w yet!). The result of this multiplication is your vector in clip-space. FYI: You can already do a couple of very useful things here with this vector: Test if the point is on the screen. The six conditions are:

    x < -w : Point is outside the screen (left of the viewport)
    x >  W : Point is outside the screen (right of the viewport)
    y < -w : Point is outside the screen (above the viewport)
    y >  w : Point is outside the screen (below the viewport)
    z < -w : Point is outside the screen (beyond znear)
    z >  w : Point is outside the screen (beyond zfar)

• Project your point into 2D space. To do this divide x and y by w:

  x' = x / w;
  y' = y / w;

• If you're interested in the depth-value (e.g. what gets written to the zbuffer) you can project z as well:

 z' = z / w

• Note that the previous step won't work if w is zero. This case happends if your point is equal to the camera position. The best you could do in this case is to set x' and y' to zero. (will move the point into the center of the screen in the next step..).

• Final Step: Get the OpenGL viewport coordinates and apply it:

  x_screen = viewport_left + (x' + 1) * viewport_width  * 0.5;
  y_screen = viewport_top  + (y' + 1) * viewport_height * 0.5;

• Important: The y coordinate of your screen may be upside down. Contrary to most other graphic APIs in OpenGL y=0 denotes the bottom of the screen.

That's all.

Answer 2

I'll add some more thoughts to Nils' thorough answer.

1. don't use doubles. I'm not familiar with NDS, but I doubt it's got any hardware for double math.
2. I also doubt model view and projection are not already multiplied if you are reading the hardware registers. I have yet to see a hardware platform that does not use the full MVP in the registers directly.
3. the matrix storage into registers may or may not be in the same order as OpenGL. if they are not, the multiplication matrix-vector needs to be done in the other order.