Binary storage of numbers with overflow bits...how's the format called?

62 views Asked by At

For a serialization/protocol format I have to encode unsigned numbers up to unsigned 64bit integer in a space-saving way that should still be easy to implement (meaning, I'm not looking for a dedicated compression algorithm). I was thinking about the following:

if n<128  
    take bits 0..6 for representing n, set overflow bit 7 to 0
    store one byte
if n>=128 and n<16384
    take bits 0..6 of byte 1 as bits 0..6 of n, set overflow bit 7 of byte 1 to 1
    take bits 0..6 of byte 2 as bits 7..13 of n, set overflow bit 7 of byte 2 to 0
    store byte 1 followed by byte 2
 if n>=16384 and n<2^21
    ...set overflow bit 7 of byte 2 to 1... (and so on)

I have two questions about this:

  1. How is this format called? Where can I look up implementations?

  2. This is for a binary protocol that will be sent over sockets, where small numbers <128 will be sent very often. Do you think the extra processing is worth it?

2

There are 2 answers

0
user601395 On BEST ANSWER

Okay, after some more research I've finally found it. It's called 'variable-length quantity' and used in MIDI and ASN.1 (see Wikipedia Entry)

To answer my other question, I'm tending to believe it isn't worth the processing overhead but I'm still pondering about it.

1
cadrian On

Not the same as, but similar to UTF-8.

Edit

BTW: try and choose a known protocol. UTF-8, Huffman encoding...