Posted by PaulHoule 7 days ago
1. zero-copy means bytes are always inlined in the raw message buffer, which means the app should always access bytes by a reference/pointer
2. You cannot compress the RPC message, if you want to fully leverage the advantages from zero serdes/copy
3. RC itself
In some message schemas even though this isn't truly zero copy it may be close to it in terms of actual overhead and CPU time, in other schemas it doesn't help at all.
It's the same for any other high performance decoding of TLV formats (FIX in finance for instance).
To a very close approximation you can say that the official protobuf C++ library always copies and owns strings.
Even the decoder makes a copy even though it's returning a string_view? What's the point then.
I can understand encoders having to make copies, but not in a decoder.
(For full disclosure, I started the ConnectRPC project - so of course I’m excited about that part of the announcement too.)
I have been on a similar odyssey making a 'zero copy' Java library that supports protobuf, parquet, thrift (compact) and (schema'd) json. It does allocate a long[] and break out the structure for O(1) access but doesn't create a big clump of object wrappers and strings and things; internally it just references a big pool buffer or the original byte[].
The speed demons use tail calls on rust and c++ to eat protobuf https://blog.reverberate.org/2021/04/21/musttail-efficient-i... at 2+GB/sec. In java I'm super pleased to be getting 4 cycles per touched byte and 500MB/sec.
Currently looking at how to merge a fast footer parser like this into the Apache Parquet Java project.
If this fixes that I might consider switching.
However, Google is also working in a new grpc-rust implementation and I have faith in them getting it right so holding tight a little bit longer.
I ended up building a protocol for my own use around a very strict subprocess boundary for Python (initially at least, protocol is meant to be universal). It has explicit payload shape, timeout and error semantics. I already went a little too far beyond my usecase with deterministic canonicalization for some common pitfall data types (I think pickle users would understand, though). It still needs some documentation polish, but if anyone would actually use it, I can document it properly and publish it.
Plain C structs that fit in a UDP datagram that you can reinterpret_cast from is still best. You can still provide schemas and UUIDs for that, and dynamically transcode to JSON or whatever.
but yes schema changes is most likely to get you today
- you agree never to care about endianness (can probably get away with this now)
- you don't want to represent anything complicated or variable length, including stringsIn practice you probably want to have both, and choose what's most practical based on the message.
For topics which are sending the state of something, a gap naturally self-recovers so long as you keep sending the state even if it doesn't change.
For message buses that need to be incremental, you need to have a separate snapshot system to recover state. That's usually pretty rare outside of things like order books (I work in low-latency trading).
For requests/response, I find it's better to tell the requester their request was not received rather than transparently re-send it, since by the time you re-send it it might be stale already. So what I do at the protocol level is just have ack logic, but no retransmit. Also it's datagram-oriented rather than byte-oriented, so overall much nicer guarantees than TCP (so long as all your messages fit in one UDP payload).
TL;DR protobuf has version compatibility and compact number encoding.
Doing plain C structs doesn't prevent any of this.
I had the same issue when looking to adopt ConnectRPC for Go, which uses a custom wrapper type to model requests.