Top
Best
New

Posted by nonadhocproblem 1 day ago

Show HN: misa77 - a codec that decodes 2x faster than LZ4 (at better ratios)(github.com)
I've spent the last few months working on this codec.

It has the following characteristics:

  - SOTA decompression throughput in its ratio class
  - Decent ratios (comparable to LZ4 at high effort levels)
  - Slow compression
Most of the gains can be attributed to reducing branches and making decompression very friendly to out-of-order cores, by using a smart format.

Results on the tarred Silesia corpus on Intel x86-64 follow:

  codec       decode      ratio    encode
  misa77 -0   5219 MB/s   42.64%   54.5 MB/s
  misa77 -1   4274 MB/s   39.65%   51.2 MB/s
  lz4         2505 MB/s   47.59%   371 MB/s
  lz4hc -12   2531 MB/s   36.45%   7.31 MB/s
155 points | 47 commentspage 2
avph 1 day ago|
This looks awesome for boot firmware. How large is the decompressor code?
jmpman 17 hours ago||
Can you use it in parquet?
throwaway74747 1 day ago||
What's the Weissman score?
mervz 1 day ago||
how does it compare to macklin71?
zuzululu 1 day ago||
so whats the actual use case for this ? streaming ? games?
pmarreck 1 day ago|
cool, now do zstd!
paddor 12 hours ago|
Zstd already follows the pareto line very closely and also decompresses pretty fast. In case you're looking for a memory-safe version in Rust that avoids the FFI penalty in hot loops, optimized to hell and back, this here is the new kid on the block: https://github.com/paddor/zrip