Posted by farlight 6 hours ago
It does seem like AVIFs win on small images, and JXL gives you progressive rendering, easy upgradability on JPEGs, and better lossless and high bitrate images.
2. Is it possible to encode AVIF with hardware acceleration?
When it comes to hardware encoders, the nicest thing you can say about them is they're fast. Software tends to do way better at quality. Hardware AV1 encoders will be tuned for video rather than stills.
I am still on webp, thinking that I am on the bleeding edge in a world where artworkers are obsessed with pixels with JPG/PNG as the only formats they know, which is an advance on GIF.
Every browser advertises what formats it will accept and if it is in 'data-saving, low bandwidth mode', so I just configure the CDN to keep the names the same '.jpg' but to send the bestest format for the situation/browser, whether that be webp or AVIF, fetching super-high resolution files from the origin server, outputting them in 'retina' resolution with none of those 'picture' elements referencing half a dozen 'responsive' sizes.
In this way nobody has to care what image format is being served, and if it is deemed that everything is way too compressed, the CDN can be flushed of the cached images, quality increased, and cache rebuilt.
It looks like I will need to update my server configuration to use JPEG XL in the usual five years' time, given the lack of any revolutionary benefit.
I was at Google for 10 years, then went to Shopify for 2 years, but I missed the web. For most of the time I was at Google, it mostly did what was good for the web, rather than pushing Google's interests, but that started to change, which is one of things that led to me leaving. E.g. see what happened with the Prompt API.
For me personally, AVIF is still better, for various reasons (not solely file size compression ratio as such, though this is my number one go-to objective; browser support is also useful but to me is not so important as it is).
I am keeping a close eye on JPEG-XL, but right now I am quite settled on .avif. All my local collection is in .avif, when before it was a mixture of .jpg, .png and a few animated .gifs. From my findings a few years ago, avif is superior to jpg, png and gifs on pretty much everything it matters. On very high compression, though, I also noticed artefacts, but we had that even more so with jpg, and for 1:1 comparable file size, avif beats jpg with both eyes closed.
By the way, great article. I hope he can do another review, based on that blog entry, in, say 3 years. Would be interesting to see how avif and jpeg-xl have improved. In all likelihood I will still keep on using avif, but if jpeg-xl is a real winner in some areas, I may use it. (Lossless is one area where jpeg-xl excels, but I don't need to store satellite data; I want to store old family pictures in an efficient manner, and here it seems avif is better.)
It's hard to rigorously judge a codec from a handful images, especially given the large impact they would have (with a widely adopted codec encoding bazillions of images in the coming years).
* There are many intricate details that go into codecs and visual perception. For instance, our sensitivity to visual errors are dependent on colors (all of hue, saturation, and brightness), spatial frequency of artifacts, local contrast and texture presence, et cetera.
And like OP mentioned, perceptual error metrics (including SSIMULACRA2) aren't perfectly representative of human-evaluated visual quality. For instance, libjxl deliberately makes choices that hurt SSIMU2 scores, but improve human-rated quality.
But yes, you can see this with jixel, which I'm really impressed by, but it's so tuned to SSIMU2 that it looks really bad to human eyes in some cases compared to libjxl.
But that is correct and why any proper study should first train the participants on the type of artifacts and trade-offs. Like the reference to JXL choices: think its creator mentioned that AVIF might look nicer for faces because it was "smoothing" them but it was a worse representation of the original. So the study participants would need to be conscious of that trade off and be clear in terms of what they're asked to evaluate - whether the result looks nicer or whether it represents the original better.
Perhaps we can use clever experiment design to address it, like doing a "full-reference" evaluation (e.g., pick most similar two images, out of encoder A, encoder B, and original) rather than strictly A/B.
Maybe the first step to designing better encoders is to build an extremely accurate visuoperceptual error metric. Seems like ML's rapid progress haven't caught onto it just yet.
Seems like others were curious too, given the Wikipedia page on Guetzli mentions:
> Butteraugli. ... How the hundreds of parameters that model the properties of the human visual system were derived remains unexplained.
My rough understanding is its multi-scale convolution architecture was inspired by biological visual perception (perhaps how the layers of visual cortex gradually represent higher level features, ish), and involved some sort of empirical parameter tuning based on human evaluations. It'd also be a very nice resource for beginners wanting to learn more about image/video codecs (like me :)