Help me understand texture mapping

[barfoo34]

Hi all!
I thought I understood the theory of how texture mapping works, but it turns out I don't.

I'm trying to setup the TEV so that the texture drawn over a geometry is fixed to the screen pixels, that is the texels should map 1:1 to the screen pixels. Here's a screenshot of the desired outcome (the texture is the blackboard with the white lines, the red is the screen background):

Screenshot from 2024-06-26 18-19-45.png

(40.33 KiB) Not downloaded yet

The way I thought that it should work:

1. Enable texture coordinate generation from vertices' positions:

   Code: Select all

   GX_SetTexCoordGen2(GX_TEXCOORD0, GX_TG_MTX3x4, GX_TG_POS, GX_TEXMTX0, GX_FALSE, GX_DTTMTX0);

2. For the GX_TEXMTX0 matrix, use the modelview matrix combined with the projection matrix, so that each vertex will get projected onto a -1,1 x -1,1 square (that is, S and T coordinates will be ranging from -1 to 1)
3. To map the S and T coordinates to the range 0, 1 (the usual range for a texture, avoiding clamping or tiling) set a post matrix in GX_DTTMTX0 which translates the S and T coordinates right and down by 1 unit, and then a scale transformation which reduces them by half (since the range from -1 to 1 is 2, but from 0 to 1 is just 1).

In practice, this does not work: multiplying by the modelview matrix indeed result in a fixed texture (although with a wrong aspect ratio), but as soon as I multiply it with the projection matrix my texture gets distorted and moves along with the geometry, instead of staying fixed to the screen. The same applies if I setup the GX_DTTMTX0 as I described in point 3 above. Why is it so?

After a few trial and errors, I finally (almost accidentally) got a working transformation, but I don't understand why it works, and why it needs to be like this. Here it is:

1. Enable texture coordinate generation from vertices' positions (like before)
2. For the GX_TEXMTX0 matrix, use the modelview matrix multiplied by a scale matrix which scales the X by proj[0][0] and the Y by proj[1][1] (in other words, we only use the scaling parts of the projection matrix, and not all of it)
3. To map the S and T coordinates to the range 0, 1 set the following post matrix in GX_DTTMTX0:

   Code: Select all

       // ignore the - on the 00 element, that's to mirror the texture horizontally
       Mtx pm = {
           {-0.5,   0, 0.5, 0},
           {0,    0.5, 0.5, 0},
           {0,      0,   1, 0},
       };
   

   The 0.5 values on the 00 and 11 elements are expected, since we are reducing the coordinates range by half, but why are the translation components set on the third column and not on the fourth one? And why are they 0.5 instead of being 1?

I published the code in this repo, in case one is interested in trying it out and experimenting with it. I've also added comments with the "QUESTION" word in those points where the texture projection matrices are being built:

https://github.com/mardy/projection-test

Any help will be appreciated. Also I did not understand why the post matrix needs to be a 3x4 one: since we only get S and T coordinates after the transformation, wouldn't a 2x4 matrix be enough?

[barfoo34]

Small update, just for completeness: there's no need to use a post-transformation matrix: the matrix I was storing in GX_DTTMTX0 can instead be multiplied with the first transformation matrix (the scaled modelview one) and the effect is exactly the same. Then I can use GX_SetTexCoordGen() instead of GX_SetTexCoordGen2() -- although in practice it does not matter much, since the former is implemented by calling the latter with an identity matrix.