CVPixelBuffer with audio

1.4k views Asked by At

I am using AVFoundation to capture CMSampleBufferRef from the camera and then convert it into CVPixelBufferRef to write to the video. What I want to do is to modify some pixel inside the video frames. That's why I need to get the CVPixelBufferRef out of my CMSampleBuffer. My problem is that I couldn't include the audio data of the original CMSampleBuffer to my new CVPixelBufferRef

I also tried to recreate CMSampleBuffer with CVPixelBufferRef but it returns me error:

- (CMSampleBufferRef)newModifyImage:(CMSampleBufferRef)sampleBuffer {
    CVImageBufferRef cvimgRef = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(cvimgRef,0);

    uint8_t *buf=(uint8_t *)CVPixelBufferGetBaseAddress(cvimgRef);

    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(cvimgRef); 
    size_t width = CVPixelBufferGetWidth(cvimgRef); 
    size_t height = CVPixelBufferGetHeight(cvimgRef); 


    CVPixelBufferRef pixelBufRef = NULL;
    CMSampleBufferRef newSampleBuffer = NULL;
    CMSampleTimingInfo timimgInfo = kCMTimingInfoInvalid;
    CMSampleBufferGetSampleTimingInfo(sampleBuffer, 0, &timimgInfo);

    OSStatus result = 0;

    OSType pixFmt = CVPixelBufferGetPixelFormatType(cvimgRef);

    CVPixelBufferCreateWithBytes(kCFAllocatorDefault, width, height, pixFmt, buf, bytesPerRow, NULL, NULL, NULL, &pixelBufRef);

    CMVideoFormatDescriptionRef videoInfo = NULL;

    result = CMVideoFormatDescriptionCreateForImageBuffer(NULL, pixelBufRef, &videoInfo);

    CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault, pixelBufRef, true, NULL, NULL, videoInfo, &timimgInfo, &newSampleBuffer);
    return newSampleBuffer;
}
1

There are 1 answers

0
Or Arbel On BEST ANSWER