Posted by systima 9 hours ago
This was the initial anecdotal evidence, but we undertook this small study to collect empirical data:
We added logging between the agentic coding tool (Claude Code and OpenCode) and Anthropic's endpoint, and captured all requests (and the returned usage blocks).
With one caveat (toward the end of the post) we found unambiguously that Claude Code was far more inefficient in terms of its cache strategy and its harness token usage than OpenCode.
In our case the alternative was nothing so I'm happy to have it, but currently claude is not as competitive as I'd have maybe expected given the hype
Supposed to be hacker news and half the posts are like "this harness steals this" like it cant be avoided.
These API costs are mad.
It pays to be marginally ahead of people stuck on open models.
If I self host a local model is there some way to make Android studio not time out after 10 minutes?
So not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model.
> Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests
Why do you need to do calibration requests to figure out how your own gateway is affecting requests?
> Its subagent lane did not complete cleanly through our gateway
> We attempted to toggle extended thinking in both harnesses and are declining to publish numbers. Our gateway applies its own thinking policy, neither harness's toggle demonstrably survived the path, and anything we quoted would be noise.
Why is your own gateway screwing with your testing?
Cost, mainly. The runs went through a Claude Max subscription rather than metered API billing, and pinning an older stable snapshot kept run-to-run comparisons clean and cheap. The fixed harness payload (system prompt plus tool schemas), so the headline numbers shouldn't change too much.
That said, happy to re-run the matrix on Fable and publish the diff; payload figures should barely move, tool-calling behaviour might.
Gateway:
Meridian (github.com/rynfar/meridian); proxy that bridges the Claude Code SDK to a standard Anthropic endpoint so a Claude Max subscription can drive OpenCode-et-al.
It's the auth route for all agent traffic on the machine, not something built for the benchmark.