When writing C code involving files, should I define _FILE_OFFSET_BITS=64?

Question

I've noticed some C projects compiling code which accesses files with _FILE_OFFSET_BITS=64. Now, on my system (which is 64-bit), adding or removing this doesn't seem to do much - but perhaps on other systems it does.

When should I use _FILE_OFFSET_BITS=64 (or any other value for it)? Or, alternatively, what do I need to check to make sure that I don't actually need it?

Answer 1

Yes, do define it.

There is no drawback on using -D_FILE_OFFSET_BITS=64 on a 64-bit platform. On 32-bit platform if you're doing something that could potentially need to seek/tell anything on files where the offsets exceed 2³¹, you need this one or be prepared to use the separate 64-bit functions.

The reason why the behaviour isn't the default is that the C standard and POSIX specifies that long int be used for various functions - on 32-bit Unixen, long int can usually hold a value up to 2³¹ only. Now, the use of -D_FILE_OFFSET_BITS=64 would guarantee that the code that uses lseek, ftello etc, would continue to work in a 32-bit system as it would in a 64-bit system.

Quoting GLibc feature test macros:

Macro: _FILE_OFFSET_BITS

This macro determines which file system interface shall be used, one replacing the other. Whereas _LARGEFILE64_SOURCE makes the 64 bit interface available as an additional interface, _FILE_OFFSET_BITS allows the 64 bit interface to replace the old interface.

[...]

If the macro is defined to the value 64, the large file interface replaces the old interface. I.e., the functions are not made available under different names (as they are with _LARGEFILE64_SOURCE). Instead the old function names now reference the new functions, e.g., a call to fseeko now indeed calls fseeko64.

This macro should only be selected if the system provides mechanisms for handling large files. On 64 bit systems this macro has no effect since the *64 functions are identical to the normal functions.

And fseeko(3) man pages:

On some architectures, both off_t and long are 32-bit types, but defining _FILE_OFFSET_BITS with the value 64 (before including any header files) will turn off_t into a 64-bit type.

Answer 2

Autoconf provides a macro for this: AC_SYS_LARGEFILE. It adds the required option by detecting at configure time whether off_t is already a 64-bit type by default, and if not, whether setting _FILE_OFFSET_BITS to 64 turns it into a 64-bit type. It also detects another preprocessor macro, _LARGE_FILES, the same way, used on certain non-GNU systems ("AIX-style hosts" according to a comment).

If you're using autoconf, you can use this macro directly. If you're not using it, you can re-implement this same logic.

These macros are in the reserved name space for any use by the implementation, so it's better not to set them on systems that do not use them the way GNU does. They may have unexpected effects on other systems. While I suspect there are no malicious implementations out there that intentionally break when these macros are defined, some implementations, possibly future ones, could legitimately use macros with these same names, by coincidence, for subtly different purposes.