Posted by karakoram 4 days ago
I wouldn't say this was the norm, but I have burned through £1k of tokens in a couple of days before. And, on that same gig, I realised a test I was considering running would burn £2.5k worth of tokens over the course of a few hours so spent a couple of hours of my own time re-engineering it to get that down to £150.
The point is that I don't necessarily know that spending a shitload of money on tokens really tells us anything useful. If he's getting a return on that investment, fine, whatever.
Obviously most of us couldn't afford to spend that much ourselves, no matter how much value it might deliver, and many of our employers wouldn't be too happy if we spent it on their dime either. Equally, if you can, and it's delivering real value, then I don't see why you wouldn't?
And, in fact, I got it from £2500 to £150 by careful model selection (I ran a trial over a small dataset with different models until I found the least expensive/capable model that would reliably do what I needed), using a local model for one step, and paring down the inputs as well so the token count plummeted.
They feel like they're super smart but it's just as smart as hiring a $500k/yr PhD as a PA to schedule your meetings and answer the phone. Sure it does a good job but you're just wasting money and abilities.
They feel super edgy and bleeding edge but all they really do differently is switching the AI chat from auto to the most expensive option all the time. When I see what they're using it for they're only grasping the basics of its abilities.
They're also people sending 2 page AI generated emails that make no sense for literally everything where a "Yeah sure let's do that" would have sufficed.
Unfortunately most of these are business leaders who just want to look edgy and "all in on AI" so we can't really limit their tokens. Yet they're also the ones saying we spend too much and want to limit some of our dev teams actually putting it to good use.
Management succumbed to AI psychosis, and so we were directed to show that we were adopting and using AI, but what we were using it for was not considered important. So we went out of our way to burn as many tokens as possible.
If you tell Claude to be inefficient and burn as many tokens as it can, it’ll happily oblige. It will also offer useful advice as to how to use even more tokens.
Once you factor in cost of electricity and depreciation of hardware, it is more expensive to run locally.
There are many arguments for running local but cost isn't one of them.
Let's do quick back of the napkin math.
- Mac Studio M5 Ultra, 80-core GPU, 256GB memory, 2TB storage -- $11,299.00.
- Power (385w continually 100% of month) is $40/mo (my regional) or $480 annually. Let's say power is 3x more expensive in California, which it probably is, that's still only $1,440 annually in power.
details please, if it's interesting
You can do it if your employer has endless pockets.
Example: $30/1M input tokens (I don’t count output or cache, for simplification), you have several daily agents for everything => 3M tokens daily per agent * 10 agents ~> $900/day => $328k/year (API spending)
With AI spend - especially at Anthropic/OpenAI API prices - spending 0.5-1 macbook a month worth of tokens per employee seems normal (from what I've seen, heard, what happens in this forum and thread and what my employer does).
Tokenmaxxxing (followed by commit/PR-maxxxing) is celebrated. Don't think - just launch orchestrator that will create dozen child agent sessions to do useless work to impress your superiors. If buy the end of the day you will ask to increase your token allowance - you're a superhero.
That's hardware.
> “It’s too expensive right now,” he said, adding that he knows the amount of money he spends on AI agents is “a crazy privilege that not everyone can do.”
Once you make the jump to agents managing agents the only limit is your budget and your harness's ability to coordinate. We'll all be using far more tokens than this in a year or two.
Or are people paying $1K/mo (subsidized) just because AI is fun? Can you imagine a video game charging $1K/mo?
In Tan's case, I think the "fun" thing is the right way to look at it. He gets to play at being a big shot developer and show off on Twitter, so clearly he feels he's getting some value from his spending even if no-one else does.
There is a more serious question about to what degree the AI vendors have a duty of care to prevent addicts from getting themselves into financial trouble - perhaps they might in future face the sort of regulation applied to high street bookmakers. I doubt Tan would fail any putative affordability test, though, so it would still be up to him to seek help for his problem if he feels its getting out of control.
But you might have hit the nail on the head - yes, it's extremely fun and maybe that's the main reason after all. And FYI many, many people pay a lot more than $1k/m for video games.
It sounds like a lot when I type it like that, but it's not actually that much if you delegate brutally. Like I said, I'm constantly token starved and could do a lot more!
You can get access to DeepSeek 4.1 Flash for almost nothing, on a $1, $10, or $100 subscription. That can complement your 5 Claude accounts, or replace some. DS Flash 4.1 is so fast and cheap it’s ridiculous
I occasionally get my agents to do sensitive things; I'm not comfortable having my transcripts collected and sold, so I don't use them. They say they anonymize them but who can ever tell? Not for me, thanks.
This is my daily intrusive thought. "Why the hell am I doing any of this? Why am I not yelling louder at people that have so much and give so little?"
This website is full of people that could save millions of human lives in aggregate, but choose not to and actively argue against the idea because, and I do swear it's a damned curse at this point, "there is no incentive to do it."
I'm not even remotely rich. I have no "savings" or "investments". Yet I have a comfortable life in a house with access to food and care as I need. Sometimes I really hate it.
There are many people in this world who have more, view people as dirt, and dedicate their lives to keeping people down. Good, kind people who are disempowered by a system that betrayed us all are not at fault. Feel better man.
Sleep is one of the most important things in the world. Why not have the agents work while you sleep?
Even better, why not have the agents sleep for you?
EDIT: for those who say this is not realistic, I've had Claude sleep for 10 hours in just 30 seconds. Saved me an entire night.
[1] https://www.amazon.com/Million-Plus-Random-Digits-Symbols-Gu...
Resume it later.
People in psychosis don't exactly weigh all of the options with a level head. He may not recognize that he has "cyber psychosis" or that his lack of sleep is a bad thing.
Don’t make me defend Garry Tan…
It was productive, but as you say, it's dangerous to gamble with sleep.
Not cloud costs, though; consumer costs. These agents must run on-device for privacy protections, because anything less will be a global non-starter. That means we need models that excel on phones, tablets, laptops, and desktops; we need home compute modules to run larger, “frontier”-class models for bigger tasks but without guzzling electricity (the Mac Mini’s power envelope is an ideal target); we need smart harnesses that can foist off inference to bigger models elsewhere on your private network of devices as needed without the user prompting for specific thinking modes or model names. All of this has to happen automatically, privately, securely, without leaking data to advertisers or surveillance companies.
It’s hard work, but it’s definitely the future of AI. I would argue the business case is solved with being able to build bespoke stacks of kit and run models through neocloud providers with ZDR policies. It’s the consumer angle that’s hard to crack, because everyone wants a repeat of Web 2.0 for ad-dollars instead of a privacy-centric modern tech stack that forces companies into lower revenue streams.
The future of personal assistants is most likely akin to Muse Spark, and yes I expect Meta is training on all of those interactions and using them to target ads.
I beg to differ.
"People just submitted it. I don't know why. They 'trust me.' Dumb fucks."
- Mark ZuckerbergThis is a difficult question to answer ! I would ask a related question which is equally hard. Has AI-use increased your human cognitive capability or decreased it ?
I accept this loss because the productivity is there. & hope to retire in a few years if things don't go south. For people entering the field I worry
That's where I struggle - when I would spend 200k/year on AI, i expect me to deliver at least 1 human output more than before.
Software? I'm not aware of anything.
Garry Tan is fully aligned with Elon Musk in going after MacKenzie Scott's charitable giving, making spurious arguments that she's doing it in the wrong way when they are all just running cover for the fact that she's one of the few at that level of wealth being charitable at all these days.
Silly woman, she should give her money away the right way, which I guess is by attempting to buy political power.
https://garryslist.org/posts/mackenzie-scott-s-26-billion-su... - The rant itself.
> This isn’t generosity. It’s negligence at scale.
Is when I realized I was wasting my time reading the article
2020s, techbros: "Here's why it's a huge problem for non-billionaires to have things."
https://en.wikipedia.org/wiki/The_Dearborn_Independent
The robber barons weren't better or more moral. They were afraid of a powerful US government that brought them to heel under popular public sentiment.
That's all it takes to have good things.
"Garry, this is a text file".
It's hard for me to burn all my weekly limits of claude ultra, codex pro and grok bot 100$ plans on a weekly basis - they are generous for daily use.
git clone https://github.com/garrytan/gstack
cd gstack/
git ls-files | xargs wc -l