"Protocol Buffers" has been the go-to for a long time but there are more options now.
For uses where serialization/deserialization CPU time is a concern, it seems to really a question of Cap'n Proto versus flatbuffers ( https://google.github.io/flatbuffers/ ).
Proto3 is not a large change. In fact it shares most of its code with proto2, as I understand it. The underlying encoding is the same.
Proto3 removes some features from proto2 which were deemed overcomplicated relative to their value (unknown field retention, non-zero default values, extensions, required fields) and adds some features that people have wanted for a long time (maps). But all of these features are things that are "on top" of the core, not really fundamental changes.
I think the only change which affects my comparison post (linked by GP) is removal of unknown field retention. This is actually noted in the comparison grid. I'm honestly very surprised that they chose to remove this feature since it is critical to many parts of Google's infrastructure.
I don't think any lookup table is provided (the wire order of entries is undefined). They are not lookup maps, they are syntactic sugar for repeated key/value pairs.
The "lookup table" is constructed at parse time. Yes, the items are just key/values on the wire, but when parsed (in C++, at least) they are placed in a google::protobuf::Map, which is hashtable-based. I guess it could differ across languages -- some might not have any specific support for maps yet.
For our use (serializing streaming market data from direct NASDAQ and BATS feeds) we found both Capn'Proto and Flatbuffers to perform similarly.
Ultimately we went with flatbuffers because we found the API much cleaner across languages (C++/Go), but it's ultimately going to depend on your use case which one you use. Performance is pretty much identical. We populate our own custom structs from the capn'proto/flatbuffers structs anyways so we're never really doing zero-copy. That said, since both of these formats transmit numerical data as little-endian memory-representations of ints/float/longs/doubles their performance is fantastic.
When we had to choose a few months back, we picked Cap'n Proto over flatbuffers mainly over the fact that we liked the API better. I couldn't retrieve my notes from back then so I sadly can't point out. In any case, we never looked back.
https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-...
"Protocol Buffers" has been the go-to for a long time but there are more options now.
For uses where serialization/deserialization CPU time is a concern, it seems to really a question of Cap'n Proto versus flatbuffers ( https://google.github.io/flatbuffers/ ).