failed to create HWDeviceContext for h264 and Videotoolbox using FFmpeg for iOS

Question

I am attempting to decode an h264 video using hardware accelaration in FFmpeg, specially for iOS devices and utilizing VideoToolbox hardware device. While I have successfully generated decoder context, but encountering difficulties in creating hardware device context and setting buffer reference. I am new with ffmpeg and seeking assistance in resolving the issue. The relevant code snippet is here:

class Decoder_Setup {
private:
    AVCodecContext *decoder_ctx = NULL;
    const AVCodec *decoder = NULL;
    enum AVHWDeviceType type;
    AVBufferRef *buffer_ref = NULL;
    AVPixelFormat pix_fmt;
    AVCodecHWConfig *hwConfig = NULL;
    
public:
    int set();
};

int Decoder_Setup::set() {
    type = AV_HWDEVICE_TYPE_VIDEOTOOLBOX;
    pix_fmt = AV_PIX_FMT_VIDEOTOOLBOX;
    decoder = avcodec_find_decoder_by_name("h264");
    decoder_ctx = avcodec_alloc_context3(decoder);
    
    int res = av_hwdevice_ctx_create(&buffer_ref, type, NULL, NULL, 0);
    decoder_ctx->hw_device_ctx = buffer_ref;
//    decoder_ctx->get_format = (AVPixelFormat (*)(AVCodecContext *, const AVPixelFormat *)) AV_PIX_FMT_VIDEOTOOLBOX;
    avcodec_open2(decoder_ctx, decoder, NULL);
    return res;
}

Thank you in advance for your assistance!

Answer 1

Your code seems to be nice. Try using avcodec_find_decoder_by_name("h264_videotoolbox"); to find hw decoder. Also make sure that your ffmpeg libs are built with correspondent flags to support hw acceleration, like --enable-decoder=h264_videotoolbox and --enable-hwaccel=h264_videotoolbox.

Here is an example of script to build ffmpeg for android. I think you can customize it by setting the flags at scripts/ffmpeg/build.sh file so that it builds for iOS.