Sending partial updates/additions for (potentially) very large text files

Question

Let's say an UA has requested my 800kB amalgamated source code file, and cached it. Let's say I then added a new function to that file.

How can I have the UA fetch only the added function, and update its cache, using standard HTTP headers? I explicitly don't want the UA to re-download the full 800kB.

Answer 1

If both the server and the UA are under your control, you can implement RFC 3229, which defines a protocol for this. See Section 10.7.3 for examples. However, be advised that RFC 3229 and RFC 7234 (on which it is based) are complex specifications that are easy to get wrong.

If you wish to interoperate with common UAs (which don’t implement RFC 3229), there’s no direct way to do what you want. You can emulate this by hand, though. For example, if the “source code file” in question is a JavaScript file for <script src>, then instead of changing that file, you could add another <script src> just for your new function. If you anticipate changes like this happening often, you could have two <script src> from the start: one for a large, infrequently updated base, the other for frequent small additions. See also another recent question about this: A solution idea for incremental updates using browser cache

Also consider other ways to reduce network traffic. Compress with the new Brotli algorithm, increasingly supported by Web browsers, which saves slightly more than gzip. If you’re concerned about spikes of bandwidth usage on the server side, try rolling out updates to users incrementally, so they don’t all try to fetch the entire file at the same time.