The fact that frontier models actively attempt to subvert evaluation harnesses (0.4% attempt rate still remaining) remains fascinating and slightly unsettling.

So the sort by atomic number?

Barium > Argon > Carbon

Why is carbon then more capable?

They talk a lot without allowing people to use it. I'm just used to OpenAI doing rollouts the first day they announce their models. Google just either make you wait 2 weeks or go on a waitlist. They are still the joke of the 3 headed dragon.

What CLI harness should be used with this? Is antigravity usable these days via CLI?

Gemini 5 Most Wanted

Argon 4 is really good in my opinion, especially in chat. Usage is limited, but one big difference is that it seems to better understand what is and isn't worth mentioning.

It has better tuning towards not adding unhelpful caveats and irrelevant details. When I told it I was unhappy with some slop being dumped into some repo markdown files from Astra and Sol it actually went ahead and tastefully cleaned them up in line with how I expected them to be. Reductions are normally so bad from agents.

One big thing is that it really feels like talking to someone smarter than me. Other agents just seem extremely knowledgeable but Argon felt smart.

At this pace it will soon be Gemini 5 cornholio. We’ll soon need an llm to explain between llm’s…

I wonder what kind of problems are they internally testing these frontier models if not math problems?

Google is putting much higher priority on frontier models over Flash models, even though they have TPUs.

Can’t stand these paternalistic companies gatekeeping new models because “cyber”. We saw how this went with the PHP vuln last week with defenders using special / early-access models and then saying “hey there’s a vuln we won’t say what it is but it’s there and we won’t patch it until tomorrow”, and with the huggingface devs having to use Chinese models to defend from OpenAI because their American models wouldn’t help because cyber.

Any bad actor worth their salt is already a) going to have the weights downloaded, forever, to various models that are good at hacking and b) is out of reach of the U.S. government anyways. A ban on open weight models, plus paternalistic gatekeepers of cyber models like the big labs, just opens up Americans and American companies and municipalities to hacking from abroad. Or I guess to a really nice racket where only the big labs can offer a “we’ll continuously monitor and improve your cybersecurity for millions of dollars a year”.

“We're taking a deliberate approach to rollout and are actively engaged in the U.S. government’s voluntary process for pre-release model testing. Starting this week, we will make Carbon available to a subset of cyber defenders and will gradually phase access to additional trusted partners through our Fairwind Program. We’re continuing to gather feedback and iterate on guardrails before making Carbon available to developers, businesses, and consumers via our product surfaces as soon as possible.”