Posted by rvz 17 hours ago
I just want to point out some features of OpenRouter that make it more than just a model selection and routing endpoint and that I find incredibly useful:
0/ Default routing is to the cheapest provider, but they're usually not the most performant. I'd guess 99% of OpenRouter integrations never tweak the default routing. Here you can setup cheapest with performance minimums:
https://openrouter.ai/docs/guides/routing/provider-selection...
You can also stack model selection in priority
1/ Using broadcast you can push all your analytics to clickhouse / s3 / snowflake and a bunch of other compatible destinations. Setup a clickhouse server ($5 VPS[0]) and send all your traces to it:
https://openrouter.ai/docs/guides/features/broadcast
customise your own observability in your dashboards from there. Superwin
2/ Model router is also a natural home for llm security - OpenRouter has the beginnings of prompt injection detection:
https://openrouter.ai/docs/guides/features/guardrails/prompt...
there is also PII detection. This will show up in observability as rejections/blocks etc.
There are so many model routing solutions (same with observability, security etc.) but they're all 80% solutions - OpenRouter really rounds out with well implemented features that you need when deploying models at any scale and I gladly pay the toll.
[0] not sure if these exist any more but clickhouse is resource efficient
Maybe I'm biased from the perspective of a "harness provider", but I think Model routers often have too little context to act as well-informed prompt injection prevention. e.g. it lacks context of where which part of the message(s) originates from and sanitization/safeguards were already performed on the application layer.
Something like OpenRouter's "flag" mode is fine, but usage of auto-redact or auto-block should really only be used if there is significant risk exposure through your harness or otherwise they are a constant source of bugs.
But don't worry, many providers there charge for cache the same price like non cached ;)
And if they are, their compliance team is about to strike them down. The VCs forcing this acquisition do know this.
- Why would you add a penalty of 50 ms at a minimum? And that is not the p95... Just run LiteLLM in house and you dont even really need that.
- Their capacity pools are shared across the whole user base, a massive batch processing by another of their customers and think what that means for your response time...
- So instead of negotiating corporate rates with OpenAI or Anthropic, you would be using an intermediary and topping up the corporate credit card... for a 5% markdown ? Really?
- They can see all your critical corporate data on the in and out
- They present some pink SOC 2 promises but then wash their hands and defer to you and the providers. Its just the Bolt and Uber model the drivers are not our employees....
- They are a man in the middle proxy that is a massive security liability for your corporation
- They have no support for private cloud points
- No geofencing guarantees
- No intellectual property legal indemnification unlike what AWS or Microsoft or Google offers
- Its a provider roulette inconsistent with hosts providing different quantization levels causing random shifts in response quality
- Support via a Discord server...
The only reason they were not shutdown yet by Anthropic or OpenAI is because they have the same VCs, as those two. That would mean said VCs investment would go to zero. Oh and those are the same VCs that own Stripe...
Just setup a private proxy tier using something like LiteLLM, even if you really dont need it. Just code your enterprise apps to have have fallback loops on the core hyperscaler providers like AWS Bedrock or Azure Foundry...
All those features show it's just a model selector and router.
I want to know which vendor/model does best at my extracting-facts-from-text task? Which does best at my OCR-a-text-document task? Which can deal with a safe-for-work beach photo without a censorship system false alarm?
OpenRouter lets me run my tests against openai and anthropic and google and x and bytedance and qwen and llama, with a single sign-up and a single payment.
Turns out even a proxy can be worth $8bn with the right business model behind it.
Users get an array of providers competing behind a single API, meaning they have to compete on price and quality not vendor lock-in. This encourages users to join OpenRouter over specific model vendors.
Providers get easy access to revenue (and data) and new customers with little to no ad spending, encouraging them onto the platform too.
And that's all you need. Win win.
Well done and congratulations.
They make less than 140 million in revenue, what today is like 5 signing bonus for OpenAI :-) They are not profitable and see through less than 2 billion in revenue.
These are easy to check so...
An underpinning of their model is that API calls / inference are similar across providers allowing for commodity tokenization cost comparisons, but the providers are beginning to shift to non commodity features that don't easily shift.
For instance, calling Gemini with "search grounding" isn't something that OpenRouter can do (they sub their own web search in), last I checked they weren't doing real time voice models, etc.
It can: https://openrouter.ai/docs/guides/features/server-tools/web-...
It's apparently even the default these days (which makes sense, as it's usually better in my experience).
Another example might be all the managed agent features, where you get a VM + model, right now it's mostly Google and Anthropic that have this offering.
The service tier is another, where Google, Vertex, OAI, Anth offer it but only OAI and Google offer flex service tier and not all models from them get it.
OpenRouter for me is an abstraction over that complexity, they will have their work cut out for them.
Stripe and every other big company could build this without any issues.
It either is just a really stupid business decision or it is about the name. The only model proxy i know is OpenRouter despite plenty of other model proxies existing.
I also congratulate the founders for pulling this off.
People said a similar thing for Cursor simply being a VSCode wrapper for when it crossed unicorn status.
I really think they only know how to sell Azure.
No reason Microsoft couldn’t have shipped a better version of Cursor considering they wrote the IDE cursor is based on.
They are fighting to be the one and only
The smaller, less known could get a win, but the bigger lose.
On one walk I asked 'why are you doing another startup?'
For context, his last one, OpenSea, was valued well into the billions, so it wasn't for money.
His reply: "I just love solving all the puzzles."
It's incredibly hard to compete with someone who is playing the game for the love of the game.
Kudos on playing well, Alex.
DOn't get me wrong, great for him to make money and being able to move fast and succeed, but you could play this game to if you want.
I'm aware that OpenRouter checks response quality onboarding, and does further checks occasionally, but I'm concerned that it's basically a cat-and-a-mouse problem between the scammers and the detectors. For example, there could be a signal that a specific pattern of requests are from OpenRouter's quality testing bots. Or, they can just route 1% of requests to an inferior model and benefit a small gain, hoping it fits into the statistically allowed margin.
Stripe can use OpenRouter to build the financial and accounting infrastructure for every product that sells metered AI work.
I think the analogy is ADP. Payroll for all the work that's going to be done by AI agents.
As AI agents/harness/human are spenders of tokens, enabling them to derisk from being locked to a specific model provider & allowing to (re)route to any model at anytime for better leverage in a single API, similar to how they are doing for payments.
Example: access to real-time stock trading data, access to weather information, letting it make stock trades, etc.
At first I was confused why Stripe bought OpenRouter, but I think this makes sense.
The user could log into his personal OpenRouter account, authorize your application, and optionally set a budget, all in something like a Stripe payment screen.
Like standard oauth?
Perhaps stripe thinks metering is a lovely big market? A nice complement to their existing service. Plus a little AI buzz likely helps their valuation. OpenRouter could be a step rather than a goal.
As user agents become more common it's only natural that they will be used for taking the heavy lifting out of e-commerce purchases. There will be a big need for digital payments to verify and reconcile these purchases.
With LLMs that are plugged into digital payments we will essentially have buying agents in our pocket that can find us exactly what we want for the cheapest price and the quickest delivery.
Which is saying a lot.
And there is no necessity. We have accountants.
Also, how many accountants do you know that are truly happy with their work? I'd like to think many of them would love to do something other than crunch numbers all day.
We're far removed from "we've sold all the computers the world can buy" (back from when computers were the size of a building).
This is a disingenuous argument. The web and email were basically instant hits and people realized it. Similar story for home computing once the computing power caught up.
Lots of techies are tech-optimists ("tech always improves quickly").
Lots of people also have dollar signs in their eyes (or related, such as increased visibility and scope).
Hard to tell which is which.
And it's a lot more likely that LLMs can't be changed to become unreliable, it's just how they work. So we would need more basic research, that doesn't grow on trees and for which the timelines are basically open ended. Maybe tomorrow, maybe right after cold fusion hits mass adoption.
Truer words, never spoken. I'm not sure how exactly this will screw me over -but I do know that it will.
YouTube there’s no real alternative because hosting unlimited amounts of video for free is a money pit that’s impossible to turn into a viable business.
If some big corporation wasn’t willing to subsidise it for some ulterior motive, it simply wouldn’t exist
https://cortecs.ai/detailedServerlessView/deepseek-v4-flash-...
https://openrouter.ai/deepseek/deepseek-v4-flash-0731#provid...
They also support fallback by default so you don’t have to write wrappers and logic to choose models, it just works with their SDk using config.