Posted by Jach 18 hours ago
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)
Not sure how either of those things are conducive to performance
Like we have OLAP and OLTP, there are still batch jobs that need to happen before you can see your dashboard, we have multiple users using the same resources, there are still DB Locks when writing needed, there are "first load" issues
— heck now we have to compete for resources with even more bots than ever.
Transferring 10GB or 1TB of data still not going to be instant laws of physics still apply.
I haven’t seen as big of an impact on the “good” side. People with attention to detail and craft can use it with great results but they’re not pumping things out at the speed of the slip slingers.
Woke up today with 2x improvement and fully tested.
The code optimized was mostly written by humans over the years. That code was too cumbersome to optimize by humans so we never touched it.
That's probably one of the biggest sources of software slowness, and we can just stop using it now, with almost no trade-off. Neat!
See https://deepseek.com/harness/en/
Is anyone really impressed by this gimmick anymore? Just give me a blank HTML with <table>. Its fine. I dont think anyone care.
Fun story, I was writing some 3D graphics animations for a website, and the only dependency I allowed was three.js. It ran well, but the site was loading slightly too slow so I told the LLM to optimize. It essentially hit a hard wall and told me something along the lines of "I can't optimize further, three.js is the bottleneck due to blablabla, but you don't need it. If you want I can rewrite it with pure WebGL". I said sure and load times were cut by like 90% and framerate went up too. Staggering.
Anything else kind of just feels like "make GTA 6, make no mistakes". The ability for you to write fast and correct software is really going to depend on well you can specify your problem, and having the ability to specify correct programs in such detail an LLM can reasonably performance tune it is a big enough reason for me to believe there will still be slow software