FFMPEG C Library: Encoding h264 stream into Matroska .mkv container creates corrupt files

Question

I want to use the FFMPEG C Library to create a Matroska Video .mkv file with only an h264 stream, but the resulting .mkv file comes out corrupt.

The file cannot be played back with Windows Media Player, ffplay, or VLC, and when I try to ffprobe the resulting file, these are the error messages:

[h264 @ 0000015060d8f5c0] No start code is found.
    Last message repeated 1 times
[h264 @ 0000015060d8f5c0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000015060d8f5c0] decode_slice_header error
[h264 @ 0000015060d8f5c0] no frame!
[h264 @ 0000015060d8f5c0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0000015060d8f5c0] decode_slice_header error
[h264 @ 0000015060d8f5c0] no frame!
[h264 @ 0000015060d8f5c0] non-existing PPS 0 referenced
    Last message repeated 1 times
# [...]
# this continues for a long time

I have followed the other troubleshooting steps for encoding h264 into Matroska, but none of them seemed to have done the trick for me:

• Setting AV_CODEC_FLAG_GLOBAL_HEADER: Invalid data when creating mkv container with h264 stream because extradata is null
• Allocating some space for extradata: Initializing an output file for muxing mkv with FFmpeg

This is my C code:

#include <stdio.h>
#include <libavutil/opt.h>
#include <libavutil/imgutils.h>
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

int main(void) {
    char *out_file_path = "./video.mkv";
    AVFormatContext *format_context;
    AVStream *video_stream;
    AVCodecContext *codec_context;

    avformat_alloc_output_context2(&format_context, NULL, NULL, out_file_path);

    const AVCodec *codec = avcodec_find_encoder(AV_CODEC_ID_H264);
    video_stream = avformat_new_stream(format_context, NULL);

    codec_context = avcodec_alloc_context3(codec);
    av_opt_set(codec_context->priv_data, "preset", "superfast", 0);
    av_opt_set(codec_context->priv_data, "crf", "22", 0);

    codec_context->width = 1920;
    codec_context->height = 1080;
    codec_context->time_base = av_make_q(1, 30);
    codec_context->pix_fmt = AV_PIX_FMT_YUV420P;

    if (strncmp("./video.mkv", out_file_path, 11) == 0) {
      printf("Writing .mkv...\n");
      codec_context->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
      codec_context->extradata = (uint8_t*)av_mallocz(1024 * 1024);
      codec_context->extradata_size = 1024 * 1024;
    } else {
      printf("Writing .mp4...\n");
    }

    // XXX avcodec_parameters_from_context is potentially superfluous (?)
    int ret = avcodec_parameters_from_context(video_stream->codecpar, codec_context);

    ret = avcodec_open2(codec_context, codec, NULL);

    avio_open(&format_context->pb, out_file_path, AVIO_FLAG_WRITE);

    ret = avformat_write_header(format_context, NULL);

    // create a black input frame
    AVFrame *input_frame = av_frame_alloc();
    input_frame->width = 1920;
    input_frame->height = 1080;
    input_frame->format = AV_PIX_FMT_YUV420P;
    ret = av_image_alloc(input_frame->data, input_frame->linesize, input_frame->width, input_frame->height, input_frame->format, 32);
    ptrdiff_t linesize[4] = { input_frame->linesize[0], input_frame->linesize[1], input_frame->linesize[2], input_frame->linesize[3] };
    ret = av_image_fill_black(input_frame->data, linesize, input_frame->format, 0, 1920, 1080);

    // write 2 seconds of video, all black
    for (size_t current_pts = 0; current_pts < 60; current_pts++) {
      input_frame->pts = av_rescale_q(current_pts, av_make_q(1, 30), codec_context->time_base);;

      ret = avcodec_send_frame(codec_context, input_frame);

      AVPacket* packet = av_packet_alloc();

      while (1) {
          ret = avcodec_receive_packet(codec_context, packet);
          if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
              break;
          } else if (ret < 0) {
              printf("avcodec_receive_packet failed");
          } else {
              av_packet_rescale_ts(packet, codec_context->time_base, video_stream->time_base);
              ret = av_interleaved_write_frame(format_context, packet);
          }
      }

      av_packet_free(&packet);
    }

    av_frame_free(&input_frame);

    // flush encoder
    avcodec_send_frame(codec_context, NULL);
    AVPacket *flush_packet = av_packet_alloc();
    while (avcodec_receive_packet(codec_context, flush_packet) != AVERROR_EOF) {
        //int ret = av_interleaved_write_frame(format_context_, packet);
        av_packet_rescale_ts(flush_packet, codec_context->time_base, video_stream->time_base);
        ret = av_write_frame(format_context, flush_packet);
    }
    av_packet_free(&flush_packet);

    ret = av_write_trailer(format_context);
    ret = avio_close(format_context->pb);

    return 0;
}

The error handling is stripped out, but it does not raise any errors.

This code works when I write into an mp4 file, but creates a corrupt file when writing into .mkv. You can change the output container format to mp4 by setting char *out_file_path = "./video.mp4";

You can compile this by running

clang -I$(FFMPEG_DIR)/include -L$(FFMPEG_DIR)/lib -lavformat -lavcodec -lavutil main.c -o makemkv

The output I’m getting while the above code is running:

Writing .mkv...
[libx264 @ 0x139804c40] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x139804c40] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x139804c40] 264 - core 164 r3108 31e19f9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=15 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

When I use the ffmpeg CLI, I can create a working .mkv from my working .mp4 like this:

ffmpeg -i video.mp4 -c:v copy created-with-ffmpeg-cli.mkv

I have uploaded the resulting video files here: https://drive.google.com/drive/folders/1FS-0fBAwKBbO-tyxC0VrFqcCyyqd0BR_?usp=sharing

Answer 1

The problem turned out to be this line:

codec_context->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

If you remove it, everything works fine.

It is important to allocate some space yourself for extradata like this:

codec_context->extradata = (uint8_t*)av_mallocz(32);
codec_context->extradata_size = 32;

You must zero the extradata (e.g. by allocating it with av_mallocz) and make sure that its size is 6 or greater. The actual size does not seem to matter as long as it is 6 or greater.

This is different from writing into an mp4 container, where the extradata does not seem to matter.

Answer 2

[h264 @ 0000015060d8f5c0] No start code is found.
Last message repeated 1 times
[h264 @ 0000015060d8f5c0] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0000015060d8f5c0] decode_slice_header error
[h264 @ 0000015060d8f5c0] no frame!
[h264 @ 0000015060d8f5c0] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 @ 0000015060d8f5c0] decode_slice_header error
[h264 @ 0000015060d8f5c0] no frame!
[h264 @ 0000015060d8f5c0] non-existing PPS 0 referenced
Last message repeated 1 times

It seems that your H.264 encoder does not set up keyframes with these settings. Please study the example code here: https://ffmpeg.org//doxygen/trunk/encode_video_8c-example.html

Fix Data Types:

I am worried about this line:

codec_context_->time_base = av_make_q(1, 90000);

Why don't you use AVRational type as it is suggested by the documentation?

time_base = (AVRational){1, 25};
framerate = (AVRational){25, 1};

There are no obstacles to see whether your framerate seems constant.

For lossy H.264 the GOP size would be bigger than 1:

    c->gop_size = 10;
    c->max_b_frames = 1;
    c->pix_fmt = AV_PIX_FMT_YUV420P;

Fix Timestamps:

My biggest concern here is that you never increase the pts field, so every frame in the sequence should have an identical timestamp as av_packet_rescale_ts function only scales up the valid timestamp but it does not increment or recalculate it. So, all the frames have a timestamp 0 because it was never set up according to the existing framerate.