nanopb - how to optimize encoding for runtime speed

Question

I am using nanopb to do logging on an embedded system: my logging messages will be .proto messages.
Speed of encoding is the most important factor; I have plenty of RAM and flash.

QUESTION
Specific to nanopb, how do I minimize encoding time?

I know of all the C optimizations I could make: inline functions, set pb_encode functions in RAM instead of flash, etc

Would it make a difference if I use all fixed-size types in my .proto file?

Answer 1

The easy methods I'm aware of are:

• Encode to memory buffer and enable PB_BUFFER_ONLY at compile time.
• If platform is little-endian and size of byte is 8 bits, you can define PB_LITTLE_ENDIAN_8BIT. It should be detected automatically on most compilers, though.
• In your message definition, minimize the amount of submessages. They require separate size calculations which take time.

These could result in speed improvements up to 2x.

---

It is further possible to speed up encoding by programming direct calls to the encoding functions, instead of going through the message structure and descriptor loops. I expect this to increase encoding speed up to 5x. Some knowledge of protobuf encoding spec will be needed.

For example, for this message:

message LogMessage {
    uint64 timestamp = 1;
    float batterylevel = 2;
    string logmessage = 3;
}

you could do:

void writelog(const char *logmsg)
{
    uint8_t buffer[256];
    pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));

    // Get system state
    uint64_t timestamp = get_system_timestamp();
    float batterylevel = get_batterylevel();

    // Encode timestamp
    pb_encode_tag(&stream, PB_WT_VARINT, LogMessage_timestamp_tag);
    pb_encode_varint(&stream, timestamp);

    // Encode battery level
    pb_encode_tag(&stream, PB_WT_32BIT, LogMessage_batterylevel_tag);
    pb_encode_fixed32(&stream, &batterylevel);

    // If we have explicit message, encode it also.
    if (logmsg)
    {
       pb_encode_tag(&stream, PB_WT_STRING, LogMessage_logmessage_tag);
       pb_encode_string(&stream, logmsg, strlen(logmsg));
    }

    // Save the encoded message data to storage
    save_log(buffer, stream.bytes_written);
}

While this results in hard-coding part of the message definition, the backward- and forward compatibility of protobuf messages works as usual. The tag numbers can be accessed by the Message_field_tag macros that are generated by nanopb generator. The field types are defined in the C code, but they are also embedded in the encoded messages so any mismatches will be detected on the decoding side.