How to convert between a dev_t and major/minor device numbers?

Question

I'm trying to write a portable program that deals with ustar archives. For device files, these archives store the major and minor device numbers. However, the struct stat as laid out in POSIX only contains a single st_rdev member of type dev_t described with “Device ID (if file is character or block special).”

How can I convert between a pair of major and minor device numbers and a single st_rdev member as returned by stat() in a portable manner?

Answer 1

While all POSIX programming interfaces use the device number (of type dev_t) as is, FUZxxl pointed out in a comment to this answer that the common UStar file format -- most common tar archive format -- does split the device number into major and minor. (They are typically encoded as seven octal digits each, so for compatibility reasons one should limit to 21-bit unsigned major and 21-bit unsigned minor. This also means that mapping the device number to just the major or just the minor is not a reliable approach.)

The following include file expanding on Jonathon Reinhart's answer, after digging on the web for the various systems man pages and documentation (for makedev(), major(), and minor()), plus comments to this question.

#if defined(custom_makedev) && defined(custom_major) && defined(custom_minor)
/* Already defined */
#else

#undef custom_makedev
#undef custom_major
#undef custom_minor

#if defined(__linux__) || defined(__GLIBC__)
/* Linux, Android, and other systems using GNU C library */
#ifndef _BSD_SOURCE
#define _BSD_SOURCE 1
#endif
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum)           major(devnum)
#define custom_minor(devnum)           minor(devnum)

#elif defined(_WIN32)
/* 32- and 64-bit Windows. VERIFY: These are just a guess! */
#define custom_makedev(dmajor, dminor) ((((unsigned int)dmajor << 8) & 0xFF00U) | ((unsigned int)dminor & 0xFFFF00FFU))
#define custom_major(devnum)           (((unsigned int)devnum & 0xFF00U) >> 8)
#define custom_minor(devnum)           ((unsigned int)devnum & 0xFFFF00FFU)

#elif defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__NetBSD__) || defined(__DragonFly__)
/* FreeBSD, OpenBSD, NetBSD, and DragonFlyBSD */
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum)           major(devnum)
#define custom_minor(devnum)           minor(devnum)

#elif defined(__APPLE__) && defined(__MACH__)
/* Mac OS X */
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum)           major(devnum)
#define custom_minor(devnum)           minor(devnum)

#elif defined(_AIX) || defined (__osf__)
/* AIX, OSF/1, Tru64 Unix */
#include <sys/types.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum)           major(devnum)
#define custom_minor(devnum)           minor(devnum)

#elif defined(hpux)
/* HP-UX */
#include <sys/sysmacros.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum)           major(devnum)
#define custom_minor(devnum)           minor(devnum)

#elif defined(sun)
/* Solaris */
#include <sys/types.h>
#include <sys/mkdev.h>
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum)           major(devnum)
#define custom_minor(devnum)           minor(devnum)

#else
/* Unknown OS. Try a the BSD approach. */
#ifndef _BSD_SOURCE
#define _BSD_SOURCE 1
#endif
#include <sys/types.h>
#if defined(makedev) && defined(major) && defined(minor)
#define custom_makedev(dmajor, dminor) makedev(dmajor, dminor)
#define custom_major(devnum)           major(devnum)
#define custom_minor(devnum)           minor(devnum)
#endif
#endif

#if !defined(custom_makedev) || !defined(custom_major) || !defined(custom_minor)
#error Unknown OS: please add definitions for custom_makedev(), custom_major(), and custom_minor(), for device number major/minor handling.
#endif

#endif

One could glean additional definitions from existing UStar-format -capable archivers. Compatibility with existing implementations on each OS/architecture is, in my opinion, the most important thing here.

The above should cover all systems using GNU C library, Linux (including Android), FreeBSD, OpenBSD, NetBSD, DragonFlyBSD, Mac OS X, AIX, Tru64, HP-UX, and Solaris, plus any that define the macros when <sys/types.h> is included. Of the Windows part, I'm not sure.

As I understand it, Windows uses device 0 for all normal files, and a HANDLE (a void pointer type) for devices. I am not at all sure whether the above logic is sane on Windows, but many older systems put the 8 least significant bits of the device number into minor, and the next 8 bits into major, and the convention seems to be that any leftover bits would be put (without shifting) into minor, too. Examining existing UStar-format tar archives with references to devices would be useful, but I personally do not use Windows at all.

If a system is not detected, and the system does not use the BSD-style inclusion for defining the macros, the above will error out stopping the compilation. (I would personally add compile-time machinery that could help finding the correct header definitions, using e.g. find, xargs, and grep, in case that happens, with a suggestion to send the addition upstream, too. touch empty.h ; cpp -dM empty.h ; rm -f empty.h should show all predefined macros, to help with identifying the OS and/or C library.)

Originally, POSIX stated that dev_t must be an arithmetic type (thus, theoretically, it might have been some variant of float or double on some systems), but IEEE Std 1003.1, 2013 Edition says it must be an integer type. I would wager that means no known POSIX-y system ever used a floating-point dev_t type. It would seem that Windows uses a void pointer, or HANDLE type, but Windows is not POSIX-compliant anyway.

Answer 2

Use the major() and minor() macros after defining BSD_SOURCE.

The makedev(), major(), and minor() functions are not specified in POSIX.1, but are present on many other systems.

http://man7.org/linux/man-pages/man3/major.3.html

Answer 3

I have a program based on an antique version of ls for Minix, but much mangled modified by me since then. It has the following code to detect the major and minor macros — and some comments about (now) antique systems where it has worked in the past. It assumes a sufficiently recent version of GCC is available to support #pragma GCC diagnostic ignored etc. You have to be trying pretty hard (e.g. clang -Weverything) to get the -Wunused-macros option in effect unless you include it explicitly.

#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunused-macros"
/* Defines to ensure major and minor macros are available */
#define _DARWIN_C_SOURCE    /* In <sys/types.h> on MacOS X */
#define _BSD_SOURCE         /* In <sys/sysmacros.h> via <sys/types.h> on Linux (Ubuntu 12.0.4) */
#define __EXTENSIONS__      /* Maybe beneficial on Solaris */
#pragma GCC diagnostic pop

/* From Solaris 2.6 sys/sysmacros.h
**
** WARNING: The device number macros defined here should not be used by
** device drivers or user software. [...]  Application software should make
** use of the library routines available in makedev(3). [...]  Macro
** routines bmajor(), major(), minor(), emajor(), eminor(), and makedev()
** will be removed or their definitions changed at the next major release
** following SVR4.
**
** #define  O_BITSMAJOR 7       -- # of SVR3 major device bits
** #define  O_BITSMINOR 8       -- # of SVR3 minor device bits
** #define  O_MAXMAJ    0x7f    -- SVR3 max major value
** #define  O_MAXMIN    0xff    -- SVR3 max major value
**
** #define  L_BITSMAJOR 14      -- # of SVR4 major device bits
** #define  L_BITSMINOR 18      -- # of SVR4 minor device bits
** #define  L_MAXMAJ    0x3fff  -- SVR4 max major value
** #define  L_MAXMIN    0x3ffff -- MAX minor for 3b2 software drivers.
** -- For 3b2 hardware devices the minor is restricted to 256 (0-255)
*/

/* AC_HEADER_MAJOR:
** - defines MAJOR_IN_MKDEV if found in sys/mkdev.h
** - defines MAJOR_IN_SYSMACROS if found in sys/macros.h
** - otherwise, hope they are in sys/types.h
*/

#if defined MAJOR_IN_MKDEV
#include <sys/mkdev.h>
#elif defined MAJOR_IN_SYSMACROS
#include <sys/sysmacros.h>
#elif defined(MAJOR_MINOR_MACROS_IN_SYS_TYPES_H)
/* MacOS X 10.2 - for example */
/* MacOS X 10.5 requires -D_DARWIN_C_SOURCE or -U_POSIX_C_SOURCE - see above */
#elif defined(USE_CLASSIC_MAJOR_MINOR_MACROS)
#define major(x)    ((x>>8) & 0x7F)
#define minor(x)    (x & 0xFF)
#else
/* Hope the macros are in <sys/types.h> or otherwise magically visible */
#endif

#define MAJOR(x)    ((long)major(x))
#define MINOR(x)    ((long)minor(x))

You will justifiably not be all that keen on the 'hope the macros are … magically visible' part of the code.

The reference to AC_HEADER_MAJOR is to the macro in the autoconf that deduces this information. It would be relevant if you have a config.h file generated by autoconf.

POSIX

Note that the POSIX pax command defines the ustar format and specifies that it includes devmajor and devminor in the information, but adds:

… Represent character special files and block special files respectively. In this case the devmajor and devminor fields shall contain information defining the device, the format of which is unspecified by this volume of POSIX.1-2008. Implementations may map the device specifications to their own local specification or may ignore the entry.

This means that there isn't a fully portable way to represent the numbers. This is not wholly unreasonable (but it is a nuisance); the meanings of the major and minor device numbers varies across platforms and is unspecified too. Any attempt to create block or character devices via ustar format will only work reasonably reliably if the source and target machines are running the same (version of the same) operating system — though usually they're portable across versions of the same operating system.