Not able to access a struct object properly inside fragment shader in Metal?

Question

I have two sections in my question:

First Section

I have a requirement where I have to pass a structure containing 3 values of a color (RGB) varying from 0 - 1 but it turns out whatever values Im receiving is different when I tested the code hardcoding the values.

Here is my fragment shader method

struct RGBColors {
    half red;
    half green;
    half blue;
};
fragment float4
samplingShader(RasterizerData in [[stage_in]],
               texture2d<half> colorTexture [[ texture(0) ]],
               const device struct RGBColors *color [[ buffer(0) ]]
)
{
    constexpr sampler textureSampler (mag_filter::linear,
                                      min_filter::linear,
                                      s_address::repeat,
                                      t_address::repeat,
                                      r_address::repeat);
        const half4 colorSample = colorTexture.sample (textureSampler, in.textureCoordinate);

    float4 outputColor = float4(0,0,0,0);

    half red = color->red;
    half blue = color->blue;
    half green = color->green;

    outputColor = float4(colorSample.r * red, colorSample.g * green, colorSample.b * blue, 0);

    return outputColor;
}

My swift structure looks like this,

struct RGBColors {
    var r: Float
    var g: Float
    var b: Float

    func floatBuffers() -> [Float] {
        return [r,g,b]
    }
}

I pass the buffers to the fragment like this,

let colors = color.floatBuffers()

    let colorBuffer = device.makeBuffer(bytes: colors, length: 16, options: [])
    renderEncoder.setFragmentBuffer(colorBuffer, offset: 0, at: 0)

But if I change the parameter color in const device struct RGBColors *color [[ buffer(0) ]] to float3 like this constant float3 *color [[ buffer(0) ]] and access through rgb values it works properly.

Second Section

As you can see in my code that in let colorBuffer = device.makeBuffer(bytes: colors, length: 16, options: []) the length is 16 but if I change it to

 `MemoryLayout.size(ofValue: colors[0]) * colors.count`

it is crashing and saying

`failed assertion `(length - offset)(12) must be >= 16 at buffer binding at index 0 for color[0].'`

Im not able to figure out what is happening. Can someone suggest me.

Thanks.

Answer 1

The Swift Float type does not correspond to the Metal half type. To my knowledge, there's no good representation of half in Swift (or, for that matter, C or Objective-C). You're providing 3 32-bit values to something which is expecting 3 16-bit values. The values you're providing do not line up with with how the receiving code accesses them, so it's accessing sub-parts of the values.

So, I recommend switching from using half in your shader to using float, which is more easily represented in Swift.

Next, your RGBColors struct is basically just redundant with the built-in type half3 or, if you took my above advice, float3. So, I recommend you just use float3. That type is even available in Swift if you import simd. In Metal (and C), you'd be able to access its members using either .r, .g, .b or .x, .y, .z, but Swift seems to only support the latter. Both languages support accessing the members using array subscript syntax.

As documented in the MemoryLayout overview, you should not use the size property or size(ofValue:) method when calculating buffer sizes or offsets. You should use stride/stride(ofValue:). Furthermore, you shouldn't use the stride of one element of a compound type multiplied by the number of elements. You need to use the stride of the whole compound type. That's because the compiler can add padding to a compound type to maintain alignment requirements and the former technique does not take that into account.

One final note: in your shader, the color variable is only used to access a single color. That is, it's not an array of colors. So, you should probably declare it as a reference type rather than a pointer type. That lets the compiler know so it can generate better code.

const device float3 &color [[ buffer(0) ]]

Of course, then you'd need to change color-> to color..