JPEG's zig-zag is a primitive for quantization, throwing data away based on rough approximation of human perception and biology. That isn't compression itself. However, the rounded and zeroed-out data is then compressed using a combination of RLE and Huffman, set up to predict the data will have lots of zeroes and few other distinct values (which the earlier step forces to be true). Or if you think about the system as a whole, you could say that JPEG predicts images will be blocky low-frequency patterns of DCT.
That corresponds to PCFG models.
Say you have a computer file composed of two parts, the first represents the setup of an experiment, and the second is the data produced by the experiment.
If you have a good theory relating to this type of experiment, then you can predict much of the second part of the file. So you only need to store the first part and possibly some corrections to the least significant bits of some of the parts of the second part of the file.
Thus with good prediction, you can compress this type of file.
But if you had a predictor so smart that it could crack the encryption key, it could start predicting the rest of the encrypted stream, and therefore compress it.
[0]: https://faingezicht.com/articles/2026/05/28/shape-of-what-we...
However I would say that testing on alice29, enwiki8, text8 data is kinda cheating. Alice in Wonderland and Wikipedia are very likely part of the training data of the LLM models used there.
So I tried on HN comments from a few days ago, extracted from the text column of the public HN bigquery dataset.
Using RWKV v7 0.1B instead of RWKV v4, I get 0.962 bits per byte on alice29, and 1.156 bits per bytes on the HN comments. Still a lot better than 2.826 bits per bytes of xz level 9.
Have you seen this leaderboard of sorts[1], and this proposal to change hutter prize[2]?
I think it's a really clever idea that you could measure an LLM's prediction abilities and language understanding by some sort of held-out compression metric because file sizes are very concrete. They are already beating shannon's numbers using a human prediction for compression, from what i can see.
I’m starting to think that compression is not, in fact, prediction.