iOS Broadcast extension crash randomly when I try to convert buffer samples to UIImage

Question

I am trying to develop broadcast extension to make screen recording. My path is get CMSampleBufferRef from extension, check the type if it is video convert to UIImage and then share with container app via MMWormhole. So far I can able to do it. But extension is randomly crashing during sample buffer to UIImage conversion. I have no idea why.

This is my CMSampleBufferRef to UIImage convertion function

-(void) convertAndSend:(CMSampleBufferRef)sampleBuffer {


    dispatch_async(dispatch_get_main_queue(), ^{

            // Get a CMSampleBuffer's Core Video image buffer for the media data
            CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

            // Lock the base address of the pixel buffer
            CVPixelBufferLockBaseAddress(imageBuffer, 0);

            // Get the number of bytes per row for the pixel buffer
            void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);


            // Get the pixel buffer width and height
            size_t width = CVPixelBufferGetWidth(imageBuffer);
            size_t height = CVPixelBufferGetHeight(imageBuffer);


            // Create a device-dependent RGB color space
            CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
            NSUInteger bytesPerRow = 4 * width;
            NSUInteger bitsPerComponent = 8;

            // Create a bitmap graphics context with the sample buffer data
            CGContextRef context = CGBitmapContextCreate(NULL,width,height,bitsPerComponent,bytesPerRow, colorSpace,kCGBitmapByteOrderDefault | kCGImageAlphaPremultipliedFirst);
            // Create a Quartz image from the pixel data in the bitmap graphics context
            CGImageRef quartzImage = CGBitmapContextCreateImage(context);
            // Unlock the pixel buffer
            CVPixelBufferUnlockBaseAddress(imageBuffer,0);

            // Free up the context and color space
            CGContextRelease(context);
            CGColorSpaceRelease(colorSpace);

            // Create an image object from the Quartz image
            //UIImage *image = [UIImage imageWithCGImage:quartzImage];
            UIImage *image = [UIImage imageWithCGImage:quartzImage
                                     scale:1.0f
                                   orientation:UIImageOrientationRight];

            // Release the Quartz image
            CGImageRelease(quartzImage);

            if (image) {
                NSString *imgData = [UIImagePNGRepresentation(image) base64EncodedStringWithOptions:NSDataBase64Encoding64CharacterLineLength];

                [self.wormhole passMessageObject:@{@"data" : imgData} identifier:@"streamData"];
            }
    });  
}

Usually extension is crashing on CVPixelBufferUnlockBaseAddress. Because of EXC_BAD_ACCESS (code=1, address=0x0)

Any idea or help?

Thanks.

Answer 1

I know this is a very late answer but, here is my solution:

Instead of converting it to an image and transferring it to the app layer, what I did is I am getting buffer information and writing those data to the shared container layer, and reading from there on the app layer.

    var imageBuffer: CVImageBuffer? = nil
    if let source = sampleBuffer {
        imageBuffer = CMSampleBufferGetImageBuffer(source)
    }
    CVPixelBufferLockBaseAddress(imageBuffer!, [])
    
    let bap0 = CVPixelBufferGetBaseAddressOfPlane(imageBuffer!, 0) //->convert to NSData
    let bap1 = CVPixelBufferGetBaseAddressOfPlane(imageBuffer!, 1)
    let byteperrow1 = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer!, 0)
    let width = CVPixelBufferGetWidth(imageBuffer!)
    let height = CVPixelBufferGetHeight(imageBuffer!)
    
    let data = Data(bytes: bap0!, count: byteperrow1 * height)
    let databap1 = Data(bytes: bap1!, count: byteperrow1 * height / 2)


    CVPixelBufferUnlockBaseAddress(imageBuffer!, [])

hope it will help you.