Next: Silver, then Iodine
Slowly spelling out ARC-AGI
How do you do fellow frontier models? </meme>
Excited to try it, opus level benchmarks with their omniscience score and multimodality seems worth testing!
https://artificialanalysis.ai/articles/artificial-analysis-i...
https://news.ycombinator.com/item?id=50268868
https://theainewsreport.com/2026-10-28-artificial-analysis-i...
IMO it emphasizes the wrong things if we are interested in "intelligence"
There are many more aspects which affect outcomes
Clicks link
Carbon isn't anywhere on the page and you can't toggle it on either. I'm sure it's just timing and it will be there soon but still just look amateur, which is about how I feel about Google and AI overall. I know the models have improved but I still have a bad taste in my mouth from testing Gemini in the past and I still feel that Google has an overly complicated "how to use our models" that changes way too often. That coupled with "you could lose access to your google account if we don't like what you do with Gemini" means I have little desire to test it out.
On top of all of that I'm completely over announcing a new model that we can't actually play with or try out. I don't think Google is lying about how well their model scored exactly but it still has big "my girlfriend goes to another school, you don't know her"-energy.
If you are going to hyperlink, maybe make sure that your model is actually on the page.
Also, the screen reader has a bug ([[duration]] minutes instead of the actual value) and you have once again announced a model we cannot use.
Edit: Both have been fixed after posting this. Maybe a Gemini 4 Carbon Agent has read and fixed the screen reader bug xD
I cannot fathom what motivates a lab, whose releases for the last year haven't been competitive, to shoot themselves in the foot like this. All this does is make it easier for competitors to snipe your actual launch, even if there was potential with this model (doubtful given 4 Argon, 3.1 Pro and 3 Pro), now everyone knows what exactly they need to release eval results wise to beat Carbon. There is this old idea of "Your competitor always is the first to have an idea what your next product will do", but in this case, Anthropic and co don't even need to guess, they got specific numbers to target.
Announce your models the moment they are available on API and in the subscription for all paying subscription users (cough Argon cough), stop with the cryptic tweets to drum up hype you know full well you won't live up to and make models that tool call, task adhere, task infer and handle long context tasks well beyond simple back-and-forth, chat style prompting. Reminder, during Ox Alpha, actual Google employees implied that was their model. How is this something anyone would believe is a good idea and how are they still allowed to tweet anything after? Please, stop that, it isn't helpful.
If frontier isn't your thing, stop wasting money on moonshot pre-train runs, fund research and open weight via Gemma and sell your inference hardware to other labs, that's far more lucrative anyway, as seen with Colossus 1.
[0] https://artificialanalysis.ai/evaluations/artificial-analysi...
Sort of? Yes, OpenAI has been bad about slow rolling but Anthropic's last few models have been available essentially at the same time as they announced it. In most cases I updated Claude Code and was using the new model right away.
Compare that to OpenAI who sometimes says "Available to all Plus/Pro/etc plans" without saying anything about "rolling out" and then it's a crap-shoot on when access will actually roll out. That said, they have gotten better, it used to be 1-3 weeks sometimes to get a new model/feature they announced was available for your plan.
And then there is Google... Who loves to talk about models like this and then give no timeline for release. Even when they do release the models it's miserable to find out how to even use it. Is it in the API? Which API? Google Cloud/Vertex/Gemini Cloud? Whatever it's called this week? Coding plan?
No, what Google does (or rather fails to do) in its rollouts does not compare to OpenAI/Anthropic.
As for the AA page, it's just embarrassing. Any smaller company doing that would get roasted, Google doesn't get a pass, learn how to coordinate or wait for something to actually be true/live before posting.
Gemini 4 Argon (at best Sonnet/Terra level) meanwhile took weeks to “come out” after their public announcement and when it finally did, paying Google AI plan users couldn't access Argon on Gemini or Antigravity even a week after. Three weeks from announcement to accessible for paying customers is not standard outside Google and this has been a problem with every Gemini release for the last year, so even using governmental requirements as an excuse isn’t possible here.
I will say that I dislike how OpenAI releases model only for Work and Codex but not via Chat, Anthropics "everywhere all at once" strategy is I feel the most user friendly.
Even in the optimistic case that this isn't some conspiracy, it highlights how arbitrary their scores are. I had trusted their previous ratings and their new index shows that even AA doesn't agree with them. What's to say this new index is any better and won't just change drastically again?
So if they are still vetting and changes can thus happen, that'd mean the score could change for better or worse. Not sure they'd want the embarrassment of a #1 score changing after announcement.
It's on my screen. Try asking your agent to refresh the page.
For historians, The blog post is 31 minutes ago so we can say that it was true for 20-30 minutes.
Slightly embarrasing but it seems to be the case that the model is actually worth what they were saying on AA. It's just that it is not available to people and I think that Anthropic and OAI will release a better model to public before google might.
AA has tons of its flaws too and the recent shift to their new v5 has been nothing but controversial for many.