Transforming a plane with DirectXTK / DirectXMath

Question

I have a 3D plane defined by three points and I want to transform it with a 4x4 matrix using DirectXTK.

I tried two ways to do this:

1. Transform the plane with Plane::Transform() method - this gives a very strange result.

2. Transform the three points and create a plane from the transformed points - this works fine.

I also tried to transpose the matrix before calling Plane::Transform() and it got the plane normal right, but the constant is wrong (plus that transposing the matrix really makes no sense to me).

void TestPlaneTransform()
{
    Vector3 p1(-2.4f, -2.0f, -0.2f);
    Vector3 p2(p1.x, p1.y + 1, p1.z);
    Vector3 p3(p1.x, p1.y, p1.z + 1);

    Plane plane(p1, p2, p3);
    Matrix m = Matrix::CreateTranslation(-4, -3, -2);

    // transform plane with matrix
    Plane result1 = Plane::Transform(plane, m);

    // transform plane with transposed matrix
    Plane result2 = Plane::Transform(plane, m.Transpose());

    // transform points with matrix
    Vector3 t1 = Vector3::Transform(p1, m);
    Vector3 t2 = Vector3::Transform(p2, m);
    Vector3 t3 = Vector3::Transform(p3, m);

    // plane from transformed points
    Plane result3(t1, t2, t3);

    result1.Normalize();
    result2.Normalize();
    result3.Normalize();
}

Here are the results after normalization:

result1 x:-0.704918027 y:-0.590163946 z:-0.393442601 w:0.196721300

result2 x:1.00000000 y:0.000000000 z:0.000000000 w:-1.59999990

result3 x:1.00000000 y:0.000000000 z:0.000000000 w:6.40000010

Plane::Transform() calls XMPlaneTransform which is part of the DirectXMath library and its documentation says simply "Transforms a plane by a given matrix.". I guess the method is just fine, but then what is wrong with my code?

Answer 1

Since you are calling Transform on the DirectXTK Plane it seems that it's not required to have it normalized before doing the call. But the DirectXTK wiki for Plane::Transform states this:

Planes should be transformed by the inverse transpose of the matrix, which for pure rotations results in the same matrix as the original.

github.com/Microsoft/DirectXTK/wiki/Plane

Answer 2

From the specs:

The normalized Plane to transform. This Plane must already be normalized, so that its Normal vector is of unit length, before this method is called.

Calling this method with a plane that is not normalized will produce sort of rubbish I guess. So I'd normalize the plane first and check again.

Plane.Transform Method (Plane, Matrix)