Posted by Jach 17 hours ago
Those at least already exist, without the need for any LLM: Sublime Text, BBEdit, Notepad++
The point is: Code needs to be maintained if it is to be relied on. I haven't heard good things about LLM's on that front.
You want code that’s fast, correct, stable, memory efficient, secure, usable, and more
The reason code might be slow is about product tradeoffs. It’s certainly easy to have an LLM optimize performance only, but what do you sacrifice? Is that an acceptable tradeoff?
Maybe slow code is a good idea if it’s the only way to get correct and secure code?
Take slack for example. It's not fast, it's not correct, it's not stable, it's not memory efficient, and it's not particularly secure. There's a reasonable argument that it's usable, but that doesn't seem to have much to do with tradeoffs in their technology choices.
I disagree with the premise that this is the desired outcome. If every piece of software is bespoke and everyone’s instance of it works slightly differently, then it’s impossible to get support or a shared knowledge of how it works. There’s no “just share the excel file”, there’s no “press the this button on the left”, there’s no “oh I use program X to solve Y” (instead you have to know what you need so the custom software can solve it, but my time in startups taught me that most users don’t know what they want or need).
Love the concept of agents adding epicycles to fix bugs. It maps closely to my experience.
Whether it can apply it directly from the codebase, I’m unsure and depends on many factors.
- Your boss/most of the market doesn't care if your software is slightly slow
- Your boss/the market wants new features
It's easier now to slip in fun optimization features when no one's looking, but the incentives are still off.
If your boss doesn't care about your software being slow he's incompetent.
https://services.google.com/fh/files/blogs/google_delayexp.p...
https://business.google.com/ca-en/think/marketing-strategies...
https://medium.com/ft-product-technology/a-faster-ft-com-10e...
https://www.pingdom.com/blog/page-load-time-really-affect-bo...
It’s often been hard for me to make coworkers care as well which is even more frustrating.
Unless your job really prioritizes performance and builds in benchmarking people often focus on finishing a feature as fast as possible and ignore performance (and accessibility, bugs, UI polish, etc.)
It obviously depends a lot on what you do, but often that doesn't matter.
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)