Posted by adchurch 1 day ago
At Weave, we write most of our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided to build a model router to handle this for us.
The Weave Router acts as an Anthropic/OpenAI endpoint specifically for coding agents. It looks at every inference request and intelligently (more on that in a sec) decides what model to send it to, handling all the translations required along the way. So it can use faster/cheaper models (e.g. DeepSeek v4, GLM 5.2, Kimi K2.6) when possible, and frontier models (Opus 4.8 & GPT 5.5 (& Fable whenever it's back)) when necessary.
How do we know what model to route to? We trained an RL model on tens of thousands (so far!) of agent traces. We reward the routing model when it selects an LLM that successfully completes the given task.
Here's an example: if you ask the router to plan a complex change, it will (probably) route that request to Opus 4.8. Subagents exploring the codebase to gather context will be routed to more suitable models (e.g. DeepSeek V4 Flash). Then when you have the plan ready to implement, it will be (most likely) be handed to a quicker model (e.g. GLM 5.2) to carry it out.
We've been using this internally for the last month or so. We've saved 40% on tokens vs. what we otherwise would have paid, with no noticeable differences in quality or velocity.
The router is source-available under Elastic License 2.0, so you can self-host it. Or if you prefer, you can also use our hosted version: weaverouter.com.
I'll be here to answer any questions you may have!
Maybe you should move away from a subscription that started charging by the token instead of by the request?
And it is SO fucking cheap.
I imagine a solution like this will eventually be an enterprise-forced solution because there is no reason right now for individual developers to be selective about model pricing. Even more important is non-tech users who do stuff through MCPs like "give me a full overview of all analytics" and let it chug for half an hour.
As prices increase we will see more of these tools to optimise and make the best use of token budget
Have you done any A/B tests on this with evidence? (That's one thing I'd be very interested to see for claims like this - I'm not necessarily doubting you, it just seems like it could be useful to understand claims of quality/efficiency)
That is super curious - using more low quality cheaper models increased your velocity? My prior would have been slightly reduced velocity but massive reduction in token costs made it worthwhile.
Is that due to the faster inference time?