Why does struct.unpack() not return a value that I expect?

Question

I have a binary file and I've read the first 50 bytes of it.

data = ftw.read(50)

print(f"     Read {len(data)}  bytes: {data}")

temp_tuple = struct.unpack("HHHL", data[0:10])

When I read the documentation (here: https://docs.python.org/3/library/struct.html#struct.unpack), it would seem that 'H' would specify 2 bytes and 'L' matches to 4 bytes. Therefore, 3 'H' translates into 6 bytes in total and 1 'L' adds another 4 bytes, for a total of 10. So, I don't understand why I'm getting the below error message:

    temp_tuple = struct.unpack("HHHL", data[0:10])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: unpack requires a buffer of 12 bytes

The last value 'L' is supposed to be an integer.

Why do I need 12 bytes? What did I not understand in the documentation?

I was expecting for this conversion to work and then I'd get a tuple with 4 variables.

Answer 1

See the note at the top of the struct documentation:

Note: When no prefix character is given, native mode is the default. It packs or unpacks data based on the platform and compiler on which the Python interpreter was built. The result of packing a given C struct includes pad bytes which maintain proper alignment for the C types involved; similarly, alignment is taken into account when unpacking. In contrast, when communicating data between external sources, the programmer is responsible for defining byte ordering and padding between elements. See Byte Order, Size, and Alignment for details.

So specify byte ordering and no padding is used:

>>> import struct
>>> struct.calcsize('HHHL')
12
>>> struct.calcsize('<HHHL')
10

Here you can see the two 00 padding bytes added so the 4-byte integer is aligned on a modulo 4-byte offset (offset 8) instead of offset 6 when unpadded:

>>> struct.pack('HHHL',0x1111,0x2222,0x3333,0x44444444).hex(' ')
'11 11 22 22 33 33 00 00 44 44 44 44'
>>> struct.pack('<HHHL',0x1111,0x2222,0x3333,0x44444444).hex(' ')
'11 11 22 22 33 33 44 44 44 44'