I'm writing an service which takes a MPEG-TS stream and serves up the stream to browsers as WebRTC.
I know the MPEG-TS input will always have h264 video and OPUS audio and I want to avoid any unnecessary decoding/encoding so I'm testing a GStreamer pipeline which looks like this:
udpsrc port=8888 ! tsdemux name=demux
demux. ! tee name=video-tee ! queue ! fakesink sync=true
demux. ! tee name=audio-tee ! queue ! fakesink sync=true
When a client connects to the service from a web browser the following bin is added to the pipeline:
queue name=video-queue ! rtph264pay aggregate-mode=zero-latency ! webrtcbin.
queue name=audio-queue ! rtpopuspay ! webrtcbin.
webrtcbin name=webrtcbin
The queues video-queue and audio-queue are then connected to tees video-tee and audio-tee in code using ghost pads.
While all this appears to work, the video stream which is displayed in the browser is black. webrtc-internals reports that the incoming video stream has the correct dimensions and framerate.
Checking GStreamers logs the problem seems to be here:
GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<rtph264pay0:sink> caps: video/x-h264, stream-format=(string)byte-stream were not compatible with: video/x-h264, stream-format=(string)byte-stream, profile=(string)constrained-baseline, alignment=(string){ nal, au }; video/x-h264, stream-format=(string)byte-stream, alignment=(string){ nal, au }
My question is then how to allow the rtph254pay node to accept the h264 stream (ideally without re-encoding)? I thought the videoconvert node would do the trick, however it seems that no matter where I insert it in the pipeline it causes issues.
In order to interface different elements both using the h264 encoding but with slightly different caps, for example, your caps seem to not have profile,alignment, etc. You can use the h264parse element.
So the second bin (or at the output of the first one) something like: