How can I implement white out/black out functionality using ColorMatrix in ImageAttributes

Question

I have white out/black out (lighting designer features) implemented on a Color using the code below. Instead of iterating through colors in a bitmap I would like to implement this using a ColorMatrix on ImageAttributes. I have the blackout part done, basically adjusting Matrix40-42 between -1 (full blackout) and 0 (no blackout). I can also do whiteout by setting Matrix40-42 to 1 (full whiteout) and 0 (no whiteout). But I want to do them both in one shot. Here's the previous code operating on colors:

public static System.Drawing.Color AdjustColor(System.Drawing.Color input)
{
    var hsv = new HSV(input);

    // Test values for white out and black out
    double whiteOut = 0.5;
    double blackOut = 0.5;

    hsv.Saturation = hsv.Saturation + (HSV.White.Saturation - hsv.Saturation) * whiteOut;
    hsv.Value = Math.Min(1.0, hsv.Value + whiteOut) * (1 - blackOut);

    return hsv.Color;
}

The HSV helper class is just code I found online that maps the color to the HSV space, I don't think it's important for the answer to this question, but if it is I'll be happy to add references to it.