I am working on a VMS that stores 10 second long video segments in MPEGTS format. Those segments can then be streamed using HLS, with playlists that look like this:
#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:11
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-START:TIME-OFFSET=1.0,PRECISE=YES
#EXTINF:10,
1602816779831000000.ts
#EXTINF:10,
1602816789831000000.ts
#EXT-X-ENDLIST
This is working great as long as those files are encoded in h.264. However, if I try creating a similar playlist using h.265 segments, it works only with our Android client, Apple and hls.js having decided to support h.265 HLS using fragmented MP4 only.
"Natively" supporting h.265 by storing fmp4 files directly isn't an option for me, so I would like to transpackage those MPEGTS files to fmp4 on demand.
So what I have attempted to do is, return this playlist instead (changing only the file extension):
#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:11
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-START:TIME-OFFSET=1.0,PRECISE=YES
#EXTINF:10,
1602816779831000000.mp4
#EXTINF:10,
1602816789831000000.mp4
#EXT-X-ENDLIST
and then lazily transpackage those MPEGTS files to fmp4 one by one using FFMPEG as they are getting requested:
ffmpeg -i 1602816779831000000.ts -c copy -movflags frag_keyframe+empty_moov+default_base_moof 1602816779831000000.mp4
ffmpeg -i 1602816789831000000.ts -c copy -movflags frag_keyframe+empty_moov+default_base_moof 1602816789831000000.mp4
Unfortunately, this seems to work only for playlists with a single segment (which means, up to 10 seconds in my case). As soon as I have 2+ files, it doesn't work, with a behavior that changes depending on which client I'm using: some will play the first file then stop, some will fast forward to the last file then play this one instead, some won't play at all...
I understand that the "normal" approach for fmp4 streaming over HLS is to use a "media initialization" segment and put it in a #EXT-X-MAP header for each segment, which are then usually encoded as *.m4s files instead of *.mp4. However, is it possible to make fmp4 work over HLS with self-contained segments, similarly to what we can do with MPEGTS? Since playlists with a single entry seem to support that, I would assume there is probably a way to do so.
Also, I know Apple got inspired by MPEG-DASH for this part of the HLS spec, and from what I understand, this is possible in MPEG-DASH.