This is insane. Can we switch the linked page to this?
tomhow 18 hours ago||
We updated the main URL to this and added other related links to the thread's top text.
dom96 19 hours ago||
Ran it on my adversarial programming language (KillSwitch) benchmark[1]. Very significant improvement over K3. On par with Gemini 4 Argon and Fable 5.1. Much slower than both though.
But this is now the highest scoring open-weight model. Amazing stuff.
Why does your benchmark have Opus 5 scoring the highest?
wolttam 17 hours ago||
Because Opus 5 is the best at their benchmark.
dom96 17 hours ago||
yeah, it performed best for whatever reason. More specifically, it managed to deduce the meaning of the hack statement in the language better than Opus 5.5 did.
epolanski 16 hours ago||
How many runs you make for each model?
Do you report an average score or the highest.
dom96 15 hours ago||
2 runs for each task, for each task it's either a pass/fail (the small green/red dots show each task pass/fail)