Where is the Thrift format specification?

Question

Although Thrift format is relatively widely used (and project still alive), I haven't been able to find specification for the underlying data format. Or formats, given that there are apparently at least two formats; standard one, and compact format.

Given its wide usage I am hoping it is just my poor google-fu, and a specification or specifications exist (outside of source code). If so, please share!

EDIT: Java source code is not too bad for figuring out format; see TBinaryProtocol.java and TCompactBinaryProtocol.java; which contain primitives, called by generated classes (which extend TBase)

EDIT2: While it does not contain format spec, "the missing guide" is good reading.

Answer 1

The format for a Thrift message depends on the serialization protocol chosen. You can serialize a Thrift object using one of many protocols, including the TBinaryProtocol, TCompactProtocol, TJSONProtocol, and other obscure options. The protocol is described abstractly in the original Thrift Whitepaper PDF in terms of the API, but is not currently defined anywhere in the Thrift apache project.

The TCompactProtocol protocol is slightly better defined on the thrift wiki but not in simple terms.

Until better documentation is available, you can also examine the Python library code alongside the Python struct module documentation as another option.

Answer 2

I had the same problem and therefore decided to write a specification of the Thrift binary and compact protocol.

The spec is now available in the Thrift repo: https://github.com/apache/thrift/tree/master/doc/specs

Old answer:

You can find it here: https://erikvanoosten.github.io/thrift-missing-specification/

Answer 3

There are two binary encodings, compact and and non-compact.

Usually you will find the compact protocol in the wild:

https://github.com/apache/thrift/blob/master/doc/specs/thrift-compact-protocol.md

While the non-compact protocol is probably a legacy protocol:

https://github.com/apache/thrift/blob/master/doc/specs/thrift-binary-protocol.md

With this being a rough overview of the specification:

https://github.com/apache/thrift/blob/master/doc/specs/thrift-protocol-spec.md