The labs use a pretty clear progression: Luna -> Terra -> Sol -> Astra; Haiku -> Sonnet -> Opus -> Fable/Mythos. Google doesn't make it easy.
Billions of people will now be exposed to a model that crushes sonnet 5.5 on benchmarks (in google ai search suggestions).
I kinda miss dumb ai in a way actually, was a certain charm to it
I do not understand ai marketing.
though I suspect they keep the price this high to increase the profit margin for the shortened release cycle (instead of 6 months I think they will release newer checkpoints in 3 months or less from now on).
still, my application now permanently use deepseek models instead. no point using gemini, neglectable real life improvements for extremely higher price.
They are trading improvements on pass@1 pointless benchmarks for worse disasters at p90/99 that you're going to hit every week in agentic workflows (if not daily):
overreach (doing more than asked), overwork (when it should have failed fast), harm (doing the opposite of what was asked), orthogonal work (doing something completely unrelated to what was asked), etc.
Google doesn't seem to be the only lab making this trade...
The set of benchmarks they released are disappointing. 10 mostly random benchmarks. Really?
argon
flash
opus
sonnet
now its
argon
hydrogen
flash
opus
sonnet
gptoss (i swear ths was removed earlier)
carbon
oh ahd hydrogen has 4th extra thinking.
holy shit just route this shit for me..
E: or not
This service has been disabled in this account for violation of Terms of Service. Please submit an appeal to continue using this product.
Haven't done anything, worked fine before update, now after. And lol where is even appeal form. Right after I said Google One was good value.
On the high-end, I feel like there is an implicit promise that if they keep releasing larger models and go past Oganesson (118), they are on the hook for AI-discovering a new stable element, so that is fair at least.