Converting 10 bit Y'CbCr to 8 bit RGB using vImageConvert_YpCbCrToARGB_GenerateConversion on iOS

162 views Asked by At

Given a CVPixelBuffer with format kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange, I'm trying to generate a corresponding 8-bit RGB frame for further processing. I understand this will be a lossy conversion – the RGB frame will just be used to calculate some descriptive statistics.

I'm basing my conversion on this sample code provided in the Accelerate documentation, which shows how to convert 8-bit BT.601 color samples to RGB.

The core of what I'm trying to do revolves around the vImageConvert_YpCbCrToARGB_GenerateConversion function, which allows you to specify a color conversion matrix, pixel range information, input YpCbCr format, and output ARGB format (see here). So far, I've adapted the sample code in the following ways:

  1. I have changed the color conversion matrix. Unfortunately, there doesn't seem to be a built-in constant like kvImage_YpCbCrToARGBMatrix_ITU_R_601_4 for BT.2020. Instead, I've created my own based on the specification:
var colorMatrix = vImage_YpCbCrToARGBMatrix(
            Yp: 1,
            Cr_R: 1.4746,
            Cr_G: -0.5714,
            Cb_G: -0.16455,
            Cb_B: 1.8814
        )
  1. I have updated the vImage_YpCbCrPixelRange to (hopefully) account for the 10 bit format:
var pixelRange = vImage_YpCbCrPixelRange(Yp_bias: 64,
                                                 CbCr_bias: 512,
                                                 YpRangeMax: 940,
                                                 CbCrRangeMax: 960,
                                                 YpMax: 255,
                                                 YpMin: 0,
                                                 CbCrMax: 255,
                                                 CbCrMin: 0)

I am not confident about YpMax, YpMin, CbCrMax, or CbCrMin, but the other values should be correct based on the documentation's description of the 10-bit video range format (kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange).

  1. The core adjustment/problem (or so I believe): I am trying to change the inYpCbCrType to reflect the actual format of my CVPixelBuffer. The sample code uses kvImage422CbYpCrYp8, but I assume I'm looking for something like kvImage420CbYpCrYp10 (which doesn't exist). As far as I can tell, the only options are listed here. I have tried some of the 10 bit formats (4:2:2 and 4:4:4), but the results are pretty wonky (grainy with gaps and mostly yellow). Of course, that's probably expected since I'm specifying a random chroma subsampling ratio, not the actual 4:2:0 used in my pixel buffer.

I believe what I'm trying to do is possible; at least, the attached table in the documentation for vImageConvert_YpCbCrToARGB_GenerateConversion indicates that YUV10 to RGB8 is supported (though it doesn't say for which chroma subsampling ratios).

Table showing supported conversions

With that background out of the way, my core questions are as follows:

  • Is what I'm trying to do supported?
  • If so, which vImageYpCbCrType am I supposed to use?
  • If that's the wrong question, what workflow should I be using instead?

I am relatively new to image processing (I mostly deal with audio signal processing), so please let me know if I have any fundamental misunderstandings in addition to the described issues. Thanks!

0

There are 0 answers