Correct alpha blending parameters in a transparent MTKView?

Question

I've recently tried to create a transparent MTKView where I can draw some geometry, including partially-transparent geometry, and have the results correctly composited with any views behind that MTKView. I had two issues trying to achieve this:

1. Drawing a partially transparent triangle over an opaque area makes that area no longer opaque.
2. The colours of partially-transparent pixels were not blending properly with the background, including the antialiased pixels from the multisampling.

I've create a test case which demonstrates at least the first issue and possibly the second issue. In it, the blending is set to:

colorAttachmentDescriptor.isBlendingEnabled = true
colorAttachmentDescriptor.pixelFormat = .bgra8Unorm_srgb
colorAttachmentDescriptor.rgbBlendOperation = .add
colorAttachmentDescriptor.alphaBlendOperation = .add
colorAttachmentDescriptor.sourceRGBBlendFactor = .sourceAlpha
colorAttachmentDescriptor.sourceAlphaBlendFactor = .sourceAlpha
colorAttachmentDescriptor.destinationRGBBlendFactor = .oneMinusSourceAlpha
colorAttachmentDescriptor.destinationAlphaBlendFactor = .oneMinusSourceAlpha

This is (I think) generally what I see recommended online.

My test case renders three triangles on top of a grid of different colours. In draw order, the white (float4(1, 1, 1, 1)) triangle is rendered first, then the blue (float4(0, 0, 1, 0.5)) and finally the red (float4(1, 0, 0, 0.5)). The clear colour of the MTKView is set to 0 for all channels.

On the left is the output I get from the test case, and on the right is the output I expect (drawn in an illustration program):

As you can see, the partially transparent triangles knock out the opacity of the opaque, white triangle. In addition, the triangles I draw on the left look a lot more transparent than they ought to be, and sampling the pixels confirms it. For example, I'd expect a pixel where the black tile and blue triangle overlap to be float3(0, 0, 0.5), but it looks closer to float3(0, 0, ~0.737).

There're also strange things going on with the multisampling: where the white triangle intersects the white tiles and only one of the transparent triangles, there doesn't seem to be any multisampling going on at all.

I saw in this answer that Core Animation expects views to use premultiplied alpha, and indeed setting the clear colour of the MTKView to float4(1, 0, 0, 1) does give the scene a reddish tint, whereas with straight alpha I'd expect there to be no difference. However my concern is that I have partially-transparent geometry overlapping both opaque geometry and nothing at all at the same time—can the solution of outputting premultiplied alpha from the fragment shader deal with this scenario? Perhaps by tweaking the blend mode parameters? What about the multisampling, which doesn't seem to resolve to use premultiplied alpha?

My test case is available here: https://github.com/Aquilosion/metal-composite

Answer 1

So, I figured out a set of changes which works at least for the sample project and my original project. I'm not certain why it works, but I have theories!

Don't use srgb frame buffers or textures

This actually gets me a lot of the way there: I changed my MTKView's pixel format, and colour attachment descriptor, to bgra8Unorm (from bgra8Unorm_srgb). I also made the same change to the textures in my original project. This specific change is I think what lets the multisampling blend properly.

Why does this help? My guess is that if the framebuffer's output format is set to linear RGB (rather than sRGB), blending calculations are more correct as the scene is drawing. Since the results seem to still be correct sRGB on the screen, I guess that Core Animation converts the non-sRGB scene to sRGB (or whatever else the screen is using) afterwards to compensate.

Don't use premultiplied alpha

All my shaders are returning straight alpha and everything appears to work. The answer I linked to contradicts this, so I'm not sure why this is the case. Possibly in older versions there are differences—I haven't done any research on this yet.

(Note that setting the clear colour to non-0 values for RGB but 0 alpha seems to imply that it does require premultiplied alpha, so I'm not 100% sure about this point, except that the blending that's going on does seem to be accurate anyway. I think as long as your clear colour is all 0s, it should be accurate.)

Adjust blending parameters

I used the following blend parameters, which I think are correct for straight alpha blending:

.isBlendingEnabled = true
.rgbBlendOperation = .add
.alphaBlendOperation = .add
.sourceRGBBlendFactor = .sourceAlpha
.sourceAlphaBlendFactor = .one
.destinationRGBBlendFactor = .oneMinusSourceAlpha
.destinationAlphaBlendFactor = .oneMinusSourceAlpha

I have no idea why the blending parameters I included in my question are so much more popular when these ones produce much more correct results as to how most people would except alpha blending to work (I'd have thought!)