unpacking signed 24 bit little endian file to decimal

Question

I'm trying to loop over an extremely large signed 24 bit little endian binary file to convert to decimal

this is the code i'm currently using

def unpack_24bit(bytes):
    return bytes[0] | (bytes[1] << 8) | (bytes[2] << 16)

f=open('1.raw','rb')

#Read in the data as a byte array (8-bit array)
ba = bytearray(f.read())[enter image description here][1]
length = len(ba)
print('N-bytes=',length)
#Since the data is in 24bits, convert three bits at a time to a 24 signed bit
nvals = int(length/3);
print('N-recordings for 24bits (3-byte data)=',nvals)
dat = np.zeros(nvals)

for i in range(0,nvals):
    dat[i] = (unpack_24bit(struct.unpack('<bbb', ba[3*i:3+i*3])))

The result it gives is seen here [1]: https://i.stack.imgur.com/FfWPn.png its not quite correct

It should appear as a square waveform

any suggestions?

Answer 1

Figured it out and will post here incase someone has a similar issue.

first 3 bytes were unsigned and last byte as signed

def unpack_24bit(bytes):
    return bytes[0] | (bytes[1] << 8) | (bytes[2]) << 16

f=open('TX_Testing_201202_20201202_044904.0000000_Other_1_1 - Copy.raw','rb')

#Read in the data as a byte array (8-bit array)
ba = bytearray(f.read())
length = len(ba)
print('N-bytes=',length)
#Since the data is in 24bits, convert three bits at a time to a 24 signed bit
nvals = int(length/3);
print('N-recordings for 24bits (3-byte data)=',nvals)
dat = np.zeros(nvals)

for i in range(0,nvals):
    dat[i] = (unpack_24bit(struct.unpack('<BBb', ba[3*i:3+i*3]))*scale+offset+nulling)