Demux video and KLV data from MPEG-TS stream, in sync

Question

I need to demux the video frames and KLV data from an MPEG-TS stream in sync, frame-by-frame. The following command to demux the KLV data and outputs a text file with the KLV data.

gst-launch-1.0 filesrc location="some_file.ts" ! tsdemux name=demux \
demux. ! queue ! meta/x-klv ! filesink location="some_file-KLV.txt"

The following command to demux the video and outputs a video file.

gst-launch-1.0 filesrc location="some_file.ts" ! tsdemux name=demux \
demux. ! queue ! decodebin ! videorate ! videoscale ! x264enc ! mp4mux ! filesink location="some_file-video.mp4" 

On combining the above two:

gst-launch-1.0 filesrc location="some_file.ts" ! tsdemux name=demux \
demux. ! queue ! decodebin ! videorate ! videoscale ! x264enc ! mp4mux ! filesink location="some_file-video.mp4" 
demux. ! queue ! meta/x-klv ! filesink location="some_file.txt"

The command doesn't work. It just gets stuck after the following message on the terminal;

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...

and, the size text and video files is 0 bytes.

An example .ts file can be found at(this file hasn't been uploaded and created by me, it is part of data for some code on github(https://gist.github.com/All4Gis/509fbe06ce53a0885744d16595811e6f)): https://drive.google.com/drive/folders/1AIbCGTqjk8NgA4R818pGSvU1UCcm-lib?usp=sharing

Thank you for helping! Cheers. :)

Edit:

I realised that there can be some confusion. The files in the link above were just used to create the .ts file.

The .ts file I am using, is available directly in either of the links below:

https://drive.google.com/drive/folders/1t-u8rnEE2MftWQkS1q3UB-J3ogXBr3p9?usp=sharing

https://easyupload.io/xufeny

Answer 1

The file you are using (MISB.ts) presumably carries Asynchronous KLV stream_id=0xBD, stream_type=0x06. But it seems odd. There are 200+ or so continuous KLV packets in a burst. Often, the KLV data flows over from the previous PES Pkt (highlighted red in the attached pic - original) into the new PES Pkt. Then a new KLV block is started (orange header, white payload). And repeat. I think this segmentation is not correct.

I edited a few packets the way I think they should be. Note, there is not more overflow of klv data (red highlight) between PES header and KLV header.

I have also found I do not need multiqueue. This pipeline works well for me on valid streams with Asynchronous KLV.

gst-launch-1.0 filesrc location="MISB.ts" ! tsdemux name=demux   demux. ! queue ! video/x-h264 ! filesink location="/tmp/video.ts"   demux. ! queue ! meta/x-klv ! filesink location="/tmp/klv.ts"

Note: current gstreamer tsdemux does not support Synchronous KLV.

Regarding your demux pipeline and the KLV data not in-sync with the video: Asynchronous KLV is aligned by in-stream proximity, not PTS. It looks like the KLV embedding is clumped in large bursts. It is not spread out evenly. Could that be the problem?

Answer 2

It seems if we use Gstreamer's multiqueue element, instead of queue, the files are being created.

I tried the following based on a suggestion from a commenter on another website I had posted the question on:

gst-launch-1.0 filesrc location="MISB.ts" ! tsdemux name=demux \
demux. ! multiqueue name=mq ! decodebin ! videorate ! videoscale ! x264enc tune=zerolatency ! mp4mux ! filesink location="some_file-video.mp4" \
demux. ! mq. mq. ! meta/x-klv ! filesink location="some_file-KLV.txt"

But, the KLV data and frames are still not in sync. I need the KLV data corresponding to each frame.

Thanks & Regards.