HLSL: What happens when I "squish" a 3x3 matrix into 3x2 matrix?

416 views Asked by At

Initially I thought it just means that the last column is simply discarded, but apparently I am getting differing results when I do such conversion as contrasted with just declaring a 3x2 matrix.

float3x3 TBN = float3x3(IN.tangent, IN.binormal, IN.normal);
float3x2 TB  = float3x2(IN.tangent, IN.binormal);

half2 vNormalTBN = mul(sampledNormal, (float3x2)TBN);
half2 vNormalTB  = mul(sampledNormal, TB);
1

There are 1 answers

2
derHugo On

I'm absolutely no shader expert so this is just a very wild guess and might be completely wrong! ^^

Afaik the components in the matrix are layed out column wise (vertically) and look somewhat like

IN.tangent.x IN.binormal.x IN.normal.x
IN.tangent.y IN.binormal.y IN.normal.y
IN.tangent.z IN.binormal.z IN.normal.z

and is memory wise simply stored as 9 floats in this order.

So similar when you explicitely use the float3x2 constructor you get something like

IN.tangent.x IN.binormal.x
IN.tangent.y IN.binormal.y
IN.tangent.z IN.binormal.z

When however you simply typecast I suspect that you simply cut off the last 3 float values and force the shader to interpret this now 6 floats array as a new float3x2 matrix which now might look like

IN.tangent.x  IN.binormal.x
IN.normal.x   IN.tangent.y
IN.binormal.y IN.normal.y