Edit: Turns out this is actually a Firefox bug.
I have several videos on my Apache 2.2 server encoded with ffmpeg using -movflags faststart and they stream fine. However seeking past the buffer line takes an extraordinary amount of time with Firefox (about 30 seconds or more to buffer) whereas Chrome has no problem at all.
Chrome shows one network request for the mp4 with partial content, but Firefox always shows hundreds of 206 partial content requests in succession when playing the mp4 (open for detail):
Most interesting is how there is one large request after all the small ones. This is the point where the video actually begins playing, and it transferred 26MB out of 1.3MB? I am not sure what is going on here.
Can anyone make sense of this? Compare what I am getting in output to this mp4 file here. It doesn't happen on that file.