In HTTP Live Streaming the files are split into fixed sized chunks for streaming. Whats the rational behind this? How is this better than having a single file and using offsets to retrieve the various chunks.
My rough ideas at the moment.
Splitting the files into multiple chunks reduces file seek time during streaming.
From what I understand file are stored as a persistent linked list on the HDD. Is this even true for modern file systems (such as NTFS, ext3) or do they use a more sophisticated data structure such as a balanced tree or hash maps to index the blocks of a file? Whats the run time complexity of seeking (using seekp, tellp, etc) in a file?
HDD is not a consideration. Its done to simplify at the network/CDN layer, as well as client logic. HTTP is a request/response protocol. It doesn't deal well with long streams. Its also doesn't multiplex. To use multiple sockets, you must make separate requests. Requiring the client to become aware of flies structure, and be able to convert a seek bar to a byte offset is complicated. Especially for variable bitrate media. But if you know a video has 100 segments (files), and you seek to 50%, Its really easy of know what file you need. And finally, how should a caching tier deal with a range request? download the whole file from the origin, or just request data as needed and 'stitch' the file back together locally? Either way the caching tier would need that logic. Additional logic comes at the cost of fewer requests per second.