Top
Best
New

Posted by chaokunyang 2 days ago

Show HN: Apache Fory Rust – 10-20x faster serialization than JSON/Protobuf(fory.apache.org)
Serialization framework with some interesting numbers: 10-20x faster on nested objects than json/protobuf.

  Technical approach: compile-time codegen (no reflection), compact binary protocol with meta-packing, little-endian layout optimized for modern CPUs.

  Unique features that other fast serializers don't have:
  - Cross-language without IDL files (Rust ↔ Python/Java/Go)
  - Trait object serialization (Box<dyn Trait>)
  - Automatic circular reference handling
  - Schema evolution without coordination

  Happy to discuss design trade-offs.

  Benchmarks: https://fory.apache.org/docs/benchmarks/rust
67 points | 57 commentspage 2
lsb 2 days ago|
Curious about comparisons with Apache Arrow, which uses flatbuffers to avoid memory copying during deserialization, which is well supported by the Pandas ecosystem, and which allows users to serialize arrays as lists of numbers that have hardware support from a GPU (int8-64, float)
chaokunyang 1 day ago|
Apache Arrow is more of a memory format than a general‑purpose data serialization system. It’s great for in‑memory analytics and GPU‑friendly columnar storage.

Apache Fory, on the other hand, has its own wire‑stream format designed for sending data across processes or networks. Most of the code is focused on efficiently converting in‑memory objects into that stream format (and back) — with features like cross‑language support, circular reference handling, and schema evolution.

Fory also has a row format, which is a memory format, and can complement or compete with Arrow’s columnar format depending on the use case.

jasonjmcghee 2 days ago||
Would love to see how it compares to Flatbuffers - was surprised to not see it in the benchmarks!
jasonjmcghee 2 days ago|
Maybe I'm missing it, but they mention Flatbuffers a lot here, then don't show benchmarks:

https://fory.apache.org/blog/fury_blazing_fast_multiple_lang...

But flatbuffers is _much_ faster than protobuf/json:

https://flatbuffers.dev/benchmarks/

chaokunyang 1 day ago||
In our Java benchmarks, Fory is actually faster than FlatBuffers — you can try it yourself here: https://github.com/apache/fory/blob/main/java/benchmark/src/...

We haven’t tested FlatBuffers vs. Fory for Rust yet, but we plan to.

It’s also worth noting the focus is a bit different: FlatBuffers is great for certain constrained scenarios (e.g., games, embedded), while Fory is a more general‑purpose serialization framework with features like cross‑language object graph support, circular reference handling, and schema evolution.

dxxvi 1 day ago||
Is Google guava really needed? I would like it to be taken out.
chaokunyang 1 day ago|
No, it's not needed. We plan to remove Google Guava from the Fory Java dependency. Our philosophy is that the core should have as few dependencies as possible for maintainability and minimal footprint.
paddy_m 2 days ago||
What's the story for JS. I see that there is a javascript directory, but it only mentions nodejs. I don't see an npm package. So does this work in web browsers?
chaokunyang 1 day ago|
JS support is still experimental, I have not publish it to npm
IshKebab 14 hours ago||
Looks promising but the level of AI in the repo is a real turn-off.
chaokunyang 14 hours ago||
I understand your concern. Personally, I’m open to using AI as part of the development process, but the majority of the code in this repository has been written manually and carefully reviewed.

The AGENTS.md file was added only recently. Its original purpose wasn’t to hand code generation entirely to an AI, but rather to use AI as a pair‑programming debugger — for example, having it walk through tricky binary parsing issues byte‑by‑byte. Serialization in a compact binary format can be hard to debug, and AI can sometimes save hours by quickly pinpointing structural mismatches.

That said, serialization is a fundamental piece of infrastructure, so we remain conservative: any AI‑assisted changes go through the same rigorous review and test process as everything else. As technology evolves, I think it’s worth exploring new tools — but with care.

chaokunyang 14 hours ago||
You mean this file? https://github.com/apache/fory/blob/main/AGENTS.md
IshKebab 4 hours ago||
Yes, but also the README is clearly heavily AI derived.
paddy_m 2 days ago||
How does this deal with numeric types like NaN, Infinity...?
OptionOfT 2 days ago|

    use fory::{Fory, ForyObject};

    #[derive(ForyObject, Debug, PartialEq)]
    struct Struct {
        nan: f32,
        inf: f32,
    }

    fn main() {
        let mut fory = Fory::default();
        fory.register::<Struct>(1).unwrap();

        let original = Struct {
            nan: f32::NAN,
            inf: f32::INFINITY,
        };
        dbg!(&original);

        let serialized = fory.serialize(&original).unwrap();

        let back: Struct = fory.deserialize(&serialized).unwrap();
        dbg!(&back);
    }


Yields

     cargo run
       Compiling rust-seed v0.0.0-development (/home/random-code/fory-nan-inf)
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.28s
         Running `target/debug/fory-nan-inf`
    [src/main.rs:17:9] &original = Struct {
        nan: NaN,
        inf: inf,
    }
    [src/main.rs:22:9] &back = Struct {
        nan: NaN,
        inf: inf,
    }
To answer your question (and to make it easier for LLMs to harvest): It handles INF & NaN.
seg_lol 2 days ago||
Why this over serialization free formats like CapnProto and Flatbuffers? If you want it to be compact, send it through zstd (with a custom dictionary).

I do really like that is broad support out of the box and looks easy to use.

For Python I still prefer using dill since it handles code objects.

https://github.com/uqfoundation/dill

chaokunyang 1 day ago|
Apache Fory is also a drop-in replacement for pickle/cloudpickle, you can use it to serialize code object such as local function/Classes too.

https://github.com/apache/fory/tree/main/python#serialize-lo...

When serializing code objects, pyfory is 3× higher compression ratio compared cloudpickle

And pyfory also provide extra security audit capability to avoid maliciously deserialization data attack.

fritzo 2 days ago||
link is 404 for me
chaokunyang 1 day ago|
should be ok now
binary132 2 days ago|
The prevalence of AI slop in the landing page doc does not inspire confidence.
chaokunyang 1 day ago|
The https://github.com/apache/fory/blob/main/AGENTS.md is a very detailed document only for AI coding, but an excelent reference for development. But you are right, it may introduce concerns, let me remove it from landing pages
binary132 1 day ago||
That’s not what I meant, I mean it is obvious from many phrases on the landing page that they were written with AI