Posted by entryleveljobsc 3 hours ago
If virtually any other company in the world said "we have internal AI servers and we let our employees use them" ... the company would measure that as they would any other internal expenditure: by how much it costs them. You blow $50k a year on servers running your LLMs? It cost you $50k.
Only a company like Anthropic, desperate to make news and help their IPO, would measure by the cost they charge customers. It's as ridiculous as Microsoft counting every employee's use of MS Word ... as if it had paid the full consumer price of each copy.
A clear example is Google. They can easily give all compute to Deepmind, yet majority is allocated to be sold at market rate.
So, every compute Deepmind and Googlers get is lost revenue opportunity and must be measured as such.
No other internal tools or servers has this much external demand to be comparable.
Also, Anthropic is showing that to operate at Anthropic level, you need $300,000 compute spend per employee
They have infinite money to burn so they aren't going to optimize.
I've seen that trivial changes to workflows can token usage by 30%. Repeat this a few times and my token usage has plummeted over the last few months.
Only outputting test failures instead of having agents read test run output in real time. Not having build logs get fed into context. Projects like UCL (unified context layer) so agents don't rediscover the same facts about a code base again and again.
Giving tools agent output modes so they only print out the minimal info needed for an agent to get work done.
When I hear a company has an absurd AI spend I assume they aren't even trying to reign it in.
you could not be more wrong, you cannot IPO if you say product is good enough. The whole point, the bet why its worth trillions is because they will reach AGI and "singularity" first, so the gap vs open source will only increase. If you say product is great as is, then you will be caught up and game over
They are doing the calculation on if the $150K/person is worth it, and it probably is.
Their revenues have exploded, they are unit-profitable, they need to also invest in the future.
AI infrastructure is supply constrained, so you have to make a decision: how much do I allocate to training vs inference?
Do I use it all for inference and do no further training? If so, what risks does that expose me to?
It’s not opportunity cost. It would be opportunity cost if their employees not using it would lead to additional revenue. However, if Inwas to go to Anthropic and double my token usage Anthropic wouldn’t decline my money. If Anthropic’s employees would stop using Claude, their revenues wouldn’t increase by an equivalent amount.
Their revenues are not currently supply constrained.
It’s just a cost cost. If their employees stopped using these services they would simply need less infrastructure, and therefore save on the cost to build and run that infrastructure. It wouldn’t translate into an increase in revenue.
I don’t agree. This is determined by elasticity of demand. If they have more compute to offer, that means they could offer higher subscription limits, more features like Duo to more people, better API pricing, and better enterprise contracts. This leads to some degree of increased consumption and more revenue. The degree is determined by how responsive customers are to pricing and features and performance. It’s not zero, so there is some amount that revenue would increase. We can debate the coefficient and the amount of compute we’re talking about here, but the premise is correct.
You could also look at the opportunity cost of not letting Deepmind do quite as much research. How much is additional research worth on the margin? There will be diminishing returns.
source? The most optimistic calculations don't come anywhere close to that.
They have never claimed that the API rates are the "real cost" of the compute spend.
I don't know where you got that from but the only time I maxed my $20 plan for a week I made the calculation that I had spend close to $200 in a single week. So maxing a $200 plan for a month should give you $8000 worth of token at API price. Obviously it's not $300k but it's still very far from the retail price.
$200 of usage on a $20/mo plan is a 10X multiplier.
The $200/month plan has 10X the usage of the $20/month plan.
So $200 * 10X = $2000/month. I don't know how you got to $8000.
Recent analysis by industry analysts like Semi Accurate have measured Sol 6.1 on the $200/month plan to be around $2000 of API equivalent use.
FYI there are 4 weeks in a month…
> Recent analysis by industry analysts like Semi Accurate have measured Sol 6.1 on the $200/month plan to be around $2000 of API equivalent use.
Sol isn't an Anthropic product though so maybe OpenAI is less generous than the former, I can't tell.
You have confused necessary and sufficient.
Maybe, but it also reduces supply that can drive up prices
There are a bunch of open source harnesses that are competitive with CC and they aren't using anywhere near that many tokens to do it.
Anthropic recently handed out free cloud resources (it's still free, guess they want you to get used to the fact that cloud is not free). I went through $250 cloud budget in less than 2 days.
Feels almost as expensive as LLM calls.
However, so is my phone!
Even within a single moment, if there is capacity for an internal request, just send it, if market traffic is currently at 100%, don't.
This is NOT the definition of opportunity cost. Please do not throw in random terms you don't understand. Opportunity cost is the value of the next best alternative you give up whenever you make a choice. WTF does it have to do with mis-classifying an expense?
Anthropic isn’t booking it as GAAP revenue, they’re just using the number for marketing.
If Anthropic had revealed their internal cost, there comments would be complaining that they were hiding something and giving useless, unrelatable data.
The expenditure is most interesting when compared to retail rates, because that's the rate the rest of us have to compare against.
As an example, as I understand it, if a hotel chain allows an employee to stay in one of their hotels for free when they go on vacation, the tax value of that benefit is what they would have charged someone for that hotel room, not how much it increased the hotel's costs to have one more hotel room full — which is generally almost nothing.
Of course, the IRS does agree with you when it comes to expenditures that are for the employer's benefit, rather than the employee's. If it didn't, almost all companies would have negative profits from the IRS's point of view.
Suppose a company could transfer an asset to an employee and have its value set as the employer's cost. A company could make a bunch of volatile investments and see which ones pay off. Then the ones that go south get sold back into the market as a tax loss and the ones that do well get transferred as compensation to employees at the purchase price instead of the current value, reducing the tax liability of both the employer (no taxable gain) and the employee (less taxable income). What would happen if any employer could do that with any employee and any asset?
Conversely, there is a separate carve out for "employee discounts", which is essentially the employer offering its employees its own products for something in the neighborhood of its own cost. Why? It's not in any way a principled accounting distinction, it's because it's harder to use for tax avoidance, because then a soap company can give its employees a discount on soap, but people only need a finite amount of soap and then the soap company can't use it for cars and real estate and stocks because they don't offer those products to the public.
If what you said was how taxation worked, a hand tool manufactuting company giving hammers they make for $1 and retailing for $2 to workers building a shed on their property would be increasing the workers' income tax bill by $2. Of course, in reality the workers get no income tax bill on the hammer given to them for work and the company can get a $1 tax writeoff from the opex of making the hammer.
The extremely discounted internal cost promote waste, so there is always a room for improving efficiency, but that too has a cost.
In a similar way, internal employees would have also a "standard daily rate" like an external employee.
Plus electricity
Yea, isn't the entire point of RSI to replace your employees altogether? So you'd spend an infinite amount more on AI than employees in that theoretical.
Like, if you were in an energy generation business spending way more on fuel and capital expenditures is way better than having zillions of employees earning checks , it would be a sign of efficiency.
So you are apparently now claiming my local train company or law firm will be using as many tokens per employee as an A.I. lab.
This is as if you learned that Anthropic scanned 1000 books per employee this year to train their models and your conclusion was that the best law firm will be scanning 1000 books per employee by 2029.
This is the world of spend more, get valued more; obviously the investment will be used to spend more and get more, so it must be good. This is investor-think. If you don't have spend, why do you need investment?
(Or it could be bad. Very very bad)
Is that the selling point?
>I wonder how much dog shit has crept into their codebase by this point.
You'd think they'd know not to shit where they eat… ?
Given the speed at which Anthropic regularly pushes updates (eg their ridiculous app versioning that's in the thousands for v4 now) I doubt they're doing anything that looks like quality processes, even AI-integrated ones other than basic code review skills.
I imagine it's as much of a mess internally as it is with any other company whose execs are concerned more with bottom-line valuation rather than maintainability, extensibility, and sustainability. Just lots of spaghetti-at-the-wall hoping that something sticks long enough to drive up the IPO.
Mythos at the highest thinking level is not the only product from Anthropic though.
I guess kinda put you in a bubble a little, I wonder how the experience of Anthropic eng who left the company would look like, when such eng would go and spend few $K a day in non AI lab, would that be encourage/allowed/taken as example?
Especially when I need to plan my next hike up Mount Shasta.
Now spend 0.5x their salary on AI, and they become 5x productive compared to no AI.
Now spend 1.1x their salary on AI, and they become 20x productive compared to no AI.
It's still obviously worth it to spend the 1.1x. But it is truly remarkable that the biggest cost of hiring a knowledge worker is no longer their salary, but their AI usage.
There is no implication here that employees are being replaced by AI spend, as long as all of the productivity is generating profit. The AI usage still needs a knowledge worker to direct it in ways that it can't direct itself, so you still need the employees.
This is what I am getting at. If this were true - and I've yet to see evidence it is, despite some anecdotal evidence (organizationally, IME, it's about 2-3x for power users, and probably a net loss of productivity in certain areas) - why would their headcount be growing so much still YoY, pretty much doubling every year. If the AI is 10-20xing why such a big jump in headcount? Yes, the numbers are shiny, the growth is apparent, I would just expect with this AI narrative they'd be "doing more with less" as this thing improves every year.
> The AI usage still needs a knowledge worker to direct it in ways that it can't direct itself, so you still need the employees.
This is the story AI sells though, and Anthropic has participated in it heavily. Yet, their interview process requires hard skills and doesn't let you use AI, so clearly, what you're saying is correct, it just doesn't really align in my head with the whole knowledge worker being replaced narrative.
But it's also true that when I'm working on 2 things at once, the bottleneck is me. I'm spending all my time giving Claude direction that it can't give itself.
So you basically have to separate out the tasks where AI can produce a 20x productivity boost, and the stuff where it can't help at all.
There's still so much stuff where it can't help at all, you need real human employees for that. That's why Anthropic is continuing to hire at breakneck pace.
And that's what I'm saying -- knowledge workers aren't being replaced. They're just accomplishing tons more work, and companies aren't running out of work to do.
And the product now iterate 2 times faster with 10 times more bugs and customers are pissed.
Looking at you Apple: https://aperocky.com/blog/post.html?slug=apple-watch-sucks
There's a lot to unpack about this statement.
Including the fact that it's probably not true.
If something is compensation, it can often be negotiated and traded off against other compensation components. How many people have successfully negotiated AI tokens away in favour of salary/bonus/stock? I'm going to guess "approximately zero".
I haven't seen anyone treating AI tokens as employee compensation. If anyone can show me an article where employees were paying taxes on their token use at work, please share it.
Companies have been treating AI tokens as part of the cost of having employees, like the cost of their office or travel expenses.
True, though that phrase understates the author's problem a bit, as it is surrounded with more wrong-words, indicating it's a wrong-idea rather than a slip of the tongue.
As you note, it's closer to office/travel expenses. Employee computers, licenses for software on those computers, licenses for SaaS stuff, paying for bandwidth, etc.
Many companies used to brag about about how nice the laptop or office was as a way of convincing people to work there. Often in the same breath as compensation.
But compensation is only one component of cost.
So I think this guy is confused about what it costs to have an employee and what the employee gets out of it. Possibly intentionally.
By giving hundreds of workers each a powerful desk-sized Alto minicomputer for their own needs, with a high-end graphical display, they gained insight into what desktop personal computing would look like 15 years later, enough insight to shape it profoundly. It's been 55 years, and although multitouch screens are proliferating, I'm still typing this on a keyboard hooked up to a framebuffer-driven display with a mouse hooked up to it, displaying hypertext, festooned with windows, icons, and a mouse that moves an onscreen pointer at the same 135° angle they settled on 55 years ago, positioning a vertical line to indicate the insertion point, with scrollbars. The display has twice as many pixels, the pixels are color, the mouse pointer is white rather than black, the screen is a flat LCD, the scrollbars are on the right, and the cursor blinks, but it's very recognizably descended from https://smalltalkzoo.computerhistory.org/HOPL-St78.html rather than, for example, ITS or GENESYS.
Only people with the luxury of unlimited zero-cost CPU cycles had a chance of seeing what the future looked like, and that allowed them to find out what worked and what didn't work in that environment. Nowadays, the regular workers who don't have the luxury of unlimited zero-cost tokens are manually reviewing voluminous AI code, because, even though that's a lot of slow work, they're still much faster programming that way than they were without it. Anthropic, instead, is doing insane things like eliciting a billion lines of Lean proofs to show that existing open-source libraries can't fail their OSS-FUZZ tests — and fixing all the bugs that effort turned up, in the process.
I doubt they are bringing that much value to the company for whatever work is done with that much token.