DHS now plays AI whack a mole, apparently.
[0] https://public-inspection.federalregister.gov/2026-21447.pdf
They mention that space bunny was an earlier checkpoints, so it may have improved (or may it could be benchmaxxed)
I find it interesting that so many models seem to be at almost exactly 74% specifically which makes me suspicious of the benchmark. I want to say 5.6-sol, Astra, deepseek flash 4.1, opus 5 (and 5.5?), Gemini 3.8-flash and a few others all ended up there?
There was talk of deepswe tests being flawed. I have yet to find a better gauge of useful work for my case though, so I still keep referencing it and sampling it. On the lookout for improved benchmarks.
"We found issues in at least 23 out of 113 tasks (over 20.3%), leading us to designate the benchmark as Flawed." - https://epoch.ai/benchmarks/deepswe/review
Models like Deepseek 4.1 flash, Muse 1.3, and MiniMax 3.1 flash are certainly capable and very cost effective, but I’m not sure anyone would say they’re better than Fable 5/5.1 in capability
H3, on the other hand, is phenomenal
2. Add "Flash" to the end
3. ?
4. Profit
Unless they come in at half the price (which I think will be impossible) on inference ... they have wasted their training compute imho.
Reasoning traces use truncated English:
> We need provide SVG code. Need creative. Ensure valid standalone. Pelican on bicycle, perhaps detailed vector. Need answer likely fenced svg. Could include title desc. Need be reasonably concise but image quality.
Ask HN: Any guesses on which model/provider is Space Bunny Alpha on OpenRouter? - https://news.ycombinator.com/item?id=49898971 - Sept 2026 (1 comment)