We created a 10 sec 1920x1200 video file of a circle with a border to test HLS streaming pixel aspect ratio (PAR) correction. The video was scaled and encoded with x264 to 320x176 giving a PAR 22/25. Our source and output HLS fMP4 files can be found here https://gitlab.com/kferguson/aspect-ratio-streaming-files.git
The moov.trak.tkhd MP4 box in the fragment_1000kbps_init.mp4 file shows the correct visual presentation size of 281x176 and the ...avc1.pasp box shows the correct PAR of 22/25. (We used the "isoviewer-2.0.2-jfx.jar" application to view the MP4 boxes.)
If the master.m3u8 file is played with the VLC player or with gstreamer,
gst-launch-1.0 playbin uri=file:///master.m3u8
the aspect ratio is correctly displayed. However, with ffplay or streaming to hls.js embedded in a web site, the PAR is not corrected (the circle in the video is squashed). What is ffplay and hls.js 'looking' for in the MP4 boxes for them to playback correctly?
We did a further experiment by concatenating all the fMP4 files into one .mp4 file with the following Powershell command:
gc -Raw .\fragment_1000kbps_init.mp4, .\fragment_1000kbps_00000.m4s, ..., .\fragment_1000kbps_00000.m4s | sc -NoNewline .\fragment_1000kbps.mp4
Strangely, ffplay plays this concatenated mp4 file back with the correct PAR. (Also included in the git link above.) We assumed by this that there is some info in the fragment m4s file boxes that ffplay (and hls.js) require to playback correctly but, we cannot find it?
The issue is the requirement of the optional vui parameters (sar_width and sar_height) within the H.264 Sequence Parameter Set (SPS) of the "avcC" MP4 box. The vlc player and gstreamer only require the pasp box inclusion but, ffmpeg and hls.js players require the additional vui parameters to be set correctly.