Is there an open-source alternative to Windows compound files?

Question

I'm trying to save a couple of files to a container file. The files can be modified later which means the container might have to be enlarged. The user should only see this container as a single file in the file system. The application is written in C++ and running on Windows, but the files should be portable to other platforms as well.

The requirements for the container format are:
1. Size: Allow files bigger than 4GB.
2. Portability: It should be feasible to open the container files on other platforms.
3. Performance: When changing a file inside the container, only the "chunks" that represent that specific file may be changed. Especially, there may not be a complete rewrite of that file.
4. Structure: It should provide a directory structure that allows to build a hierarchy of files and directories.

When leaving out requirement 1.) and 2.) compound files (like used by Word and Excel) are a good fit.

Of course, it's possible to design an API that fulfills all the requirements, but I wonder if there is already some open-source solution that tackles the very problem?

Any suggestions?

Answer 1

Is the HDF5 File Format what you are looking for if you are building an application in C++

Answer 2

Some archive format (zip, bzip2) might do the job.

Answer 3

To extend J.F. Sebastian's answer, tar might be a better format due to it not compressing. It's literally just a container.

I'm not certain how any of these archive formats are at being editable though. Have a play with SharpZipLib and find out, I guess =)

Answer 4

A google search for "portable structured storage" yielded: pole