I hate these "The best X thing Y has ever released".

Unlike when Apple says "it's the best iphone we've ever made", LLMs are more or less interchangeable. So "OpenAI's best model" means nothing if "Anthropic wipes the floor with them" or "[open weights model] is 10x cheaper for 1% less quality".

As a reader, it feels like these titles are click bait.

the last image - it's barely visible to human eyes

> GPT 5.6 Sol is the best "vision" model OpenAI ever released

I mean I should hope so, as it is also the latest one

Where are the Qwen benchmarks in this? I would be more interesting to see how Qwen performs.

It's really quite good! I was amazed recently by its utter inability to read some faded handwritten cyrillic on the back of a wood carving - 3 or 4 words only, reasonably clear letter forms I found recently, and then stepped back a bit and thought about how insane that was as a benchmark - I just expect it to work so reliably on other OCR and translation tasks that it was surprising to encounter such a failure

Fuck ack. I'm working on a new benchmark that combines strong visual requirements with tool and coding requirements. I haven't even tested Sol yet, but between Sonnet, Terra & Luna I already see much better results from OpenAI's models. I'm not releasing anything yet as I still have issues in my harness that need to be fixed.

Am I the only one who cannot read the date on the blister pack even fully zoom in my phone?

If that is the full quality image given to the model, I think it's not surprising that the model confused with 03/2022.

I'm a little bit disappointed that vision seems to fall before language at scale.

It seems pretty counter intuitive that we can't do vision significantly better with specialized techniques.

Which is to say, still not ready for any production workloads yet. As in, it cannot reliably count the amount of objects in an image.

Still very impressive, but nowhere near the text chat revolution. OpenAI still trying to strike their second lightning