Posted by Jach 16 hours ago
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.
Try turning off wifi and see if it improves.
Things like this really pushed me to everything local systems. I’ll move actual files around if I want to do anything on the network. Or sometimes even use cables! Shock, horror!
The original design had the speakers setting up a private mesh network, and the app would send commands directly to the speakers via your LAN. Then, they got the brilliant idea to route commands via their cloud service. The app would send commands to an endpoint, which would send them back to your speakers. Imagine trying to smoothly fade volume with a WAN hop. This went over as well as you’d expect, and they’ve since promised to work on performance. Thus far they seem to have been doing so; it isn’t as snappy as the original, but it’s quite a bit better.
Device developers, stop doing this! You people realize that LANs exist, don't you?
I argued breathlessly against it for days. I’ll never forget the sales chad raising his voice to shut me down with a cop-out:“This is the way the industry is going!”.
It’s not the developers making these changes.
But if I were to pick one single thing that would speed up the most UIs across the board, it would be poor handling of the UI in networked systems. As you noted, that doesn't mean eliminating them, it means handling the inevitable in a way that doesn't tank the UI feel.
Somehow I don't think I'm the only one who presses a button and when nothing happens presses it repeatedly until something happens, or I kill the app, or even power off whatever piece of shit computer I'm using.
You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.
Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.
Best example in a monolith case is oracle...
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don’t anthropomorphize your lawnmower, the lawnmower just mows the lawn - you stick your hand in there and it’ll chop it off, the end. You don’t think "oh, the lawnmower hates me" – lawnmower doesn’t give a shit about you, lawnmower can’t hate you. Don’t anthropomorphize the lawnmower. Don’t fall into that trap about Oracle."
I feel this is so much real. Like if there were two kinds of companies: the ones who deliver, they care about their product but most of all they care about their customers, the managers get their hands dirty and everyone pushes towards the same direction; then there are companies in which you open a ticket and wait for two weeks for something that should take 5 minutes, customers and product don’t matter because you’re focused in cost attribution and no body does anything if it doesn’t come in your JIRA board, the managers are all coming from consultancy companies and all they do is finding someone to blame.
The problem usually comes from inappropriately arranging the systems of record such that information needs to be communicated beyond the scope of one computer in order to satisfy a single logical request.
Moving information between physical processors tends to be significantly more expensive than local computation over that same information. JSON serialization is a really good example of this. You need a network with bandwidth in excess of 10 GbE to begin overtaking simdjson.
SSR or SPA doesn't really matter if the server still takes a minimum of 300ms to compose any kind of response due to how its database or other infrastructure is set up. Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful. Stringing the user along with psychological tricks is a lot cheaper than hiring a skilled developer to do it the right way.
> Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful.
Yes we can - we can fix why the information isn't available. If someone said to you "sorry, we don't have the info because the other thread is doing Sleep(5000);" you'd call them an idiot right? You'd go and delete the sleep call to make it faster. Most real problems are harder than that, but there's no fundamental rule saying your database has to be slow. Ping time across your LAN is probably under a millisecond, so where are the other 299 milliseconds going? Is your database doing a full table scan? Is it using spinning rust for frequently accessed data?
As to your database comment, IME most of the time the bottleneck is the ORM and/or language. The amount of work an ORM does to generate a representation of a row is frankly shocking. Not understanding the cost of context-switching is the language half of it: Python, of course, is single-threaded, but you can use greenlets to cheat, because they’re I/O bound — except for all of them serializing behind a single process handling serdes for the queries.
With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU).
With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger.
Now the network to you is distance to the server (not your rail/nextjs backend taking 400ms). Things like 4G and 3G are fine. The real problem is when you have such bad signal you effectively have no down or up.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.
Even though benchmarks suggest this sort of performance should be trivial, most real-world servers I have interacted with do not reliably managed to process a request, make a roundtrip to the DB, and return a response in <5ms
It's one thing not to e.g. spend the extra time to ensure everything is cached and mutations are queued up. It's another thing not to do the bare minimum to ensure what is already available and working locally is gated on the network being up.
Case in point: The other day I was checking our train tickets in an app, and the network was awful, and the train tickets which the app has local copies of took 30+ seconds to appear when the network went down. Everything I needed worked once the timeouts had been hit, it was just ridiculously slow waiting for timeouts for functionality I wasn't trying to use to be hit first.
* Must feel very responsive * Network requests can take up to 500ms end to end
This gave me a chuckle because I personally hate things like watching the browser jump through 50+ redirects when logging into a website.
We've come full circle.
If there's a connection to your services outside of the user's machine you can control all of that.
Don’t know about that, pretty much everything is hosted on Cloudflare, Azure or S3 these days and all of them have at least one CDN on each continent.
I wish Atlassian listened to you.
I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.
We as an industry should use AI to enable a standard of software quality that was previously uneconomical.
After traveling around in places with very poor wifi/phone data speeds. I couldn't agree more with you.
It's infuriating when it's obvious that the developer of an app only ever tested it in a simulator on their dev machine on their super fast WiFi. It never seems to connect with those people developing a mobile app (or web app) that the "mobile" part has a meaning more than just on a handheld device.
https://nextjs.org/docs/app/guides/prefetching#hover-trigger...
In my view, websites should not take seconds to load with gigabit fiber. Whatever happened to "mobile first"?
Funny thing is it apparently predates JSON. It's a bunch of data[foo][bar] = baz; - go look.
(Website's in German obviously, and a surprising number of German electronic terms are very different from English. They use two different words for stranded and non-stranded wire.)
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Speculation...
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web. Why does it feel like we've landed on the worst possible one?
1. You need write access to the server if you want to add one
2. The server could change its behavior at any time and there's no way to know that caches now need to be invalidated
3. If something goes wrong with connectivity or name resolution, there's no fallback since the authoritative thing was not something durable like a trusted human via a public key but rather an ephemeral thing: a named server which has pinkey promised to stay online.
It asks the user to treat a server like a trustworthy source of perisisant data.
But there's no reason to couple these kinds of trust. The skills necessary to persist and traffick data are orthogonal to being trustworthy about content. Coupling them creates needless load on single sources of failure which are simultaneously single points for corruption to target.
Trust people, not servers. Use digital signatures to validate that what you're seeing came from those people.
<a> tags are the opposite of this. They encourage us to trust servers by name, which isn't really working out.
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
Welcome to Next.js
The .js part makes me think not.
<script type="speculationrules">
{
"prefetch": [
{
"source": "list",
"urls": ["/checkout.html", "/thank-you.html"]
}
]
}
</script>I should be able to get the lay of the land without trusting the site enough to blindly execute whatever code it points me at. It's needless attack surface.
Also it's not really pointing me at data, its pointing me a certain kinds of requests which I have to trust will be responded to consistently. I'd much rather have a hash so if I have that data lying around I can just forgo the request entirely and use what's present locally.
The optimisations the article is talking about would help even with this problem if backends responded faster - although not as much as actually avoiding unnecessary network requests in the first place, of course.
Obviously agent access goes away with internet failure but the state doesn’t need to… we use CRDTs and a virtual FS. There’s a toy-ish version of the harness at https://ourhearth.ai … if local first is interesting to you I’d love your feedback
second is on a smaller timescale. so many features and crap that is never asked for and never used is crammed into software so the systems that execute it are just juggling pretty much dead code in and stale data in their caches all days long.
We don't turn faster hardware into faster programs, we turn it into more program. AI isn't going to change that. We'll just get even more program because the optimisation has freed up space for that.
Unfortunately most of the time, the more program isn't for our benefit. I note that by far the heaviest program I use is my web browser. The one thing I don't get to choose what code gets thrust upon me.
If I want to watch streaming videos, I don't have a choice about how I do that.
Fine Firefox is basically a bloated YouTube app. That doesn't change the fact that it is inefficient (from the pov of my CPU) for doing that.
Recently I was frustrated by Windows 11's seeming inability to open a context menu with acceptable speed - right click an item in the taskbar and there is nearly a 1000ms delay before the menu appears. That is unacceptable.
When I need to run old software, I now try to the "minimum viable runner" OS - start with an XP VM and slowly move upwards if it doesn't work. Obviously I'll lock it down from internet access/etc., but it really shows that modern OSes really don't have a grip on performance.
https://github.com/eaftan/safere
https://eaftan.github.io/safere-intro/
Mine is for Java and is intended to be production grade. The first goal is to guarantee linear-time behavior to prevent ReDoS attacks. My collaborator and I have recently been optimizing it to try to surpass native RE2 in performance.
It turns out optimizations are incredibly well suited for an agentic loop. You've got concrete acceptance criteria (must show a meaninging improvement on a benchmark case, must pass tests). The agent is really, really good at using tools like a profiler and disassembler, better than I am (and I've been doing this for 20 years). It also papers over things that would take me a while to learn, like how the in-incubation Vector (SIMD) API works in Java. I understand the concept but it would take me a while to understand Java's implementation. The agent can just read the docs and go.
The key is creating a good benchmark suite and ensuring the agent doesn't ship optimizations that are too narrow or too focused on the benchmark cases. You also need a really strong test suite to make sure you're not regressing correctness. SafeRE has billions of tests; a subset of several million run on CI, and the others run on-demand.
One little thing I spotted is you use Boyer Moore Horspool for fast literal search. This is actually not linear in the worst case, although it is almost always sublinear. Worst case would be a literal composed of the same character searching a text of the same character, where it becomes quadratic.
You can actually search strings with character classes using Horspool if you want to, and I have some enhancements to basic Horspool which could maybe help. My library, byteseek [1], implements these.
I also have a much faster algorithm, HashChain [2] which also has a guaranteed linear time version. This was published in the Symposium for Experimental Algorithmics in 2024.
This will first massively slow down the Horspool scan, and then once you have done all that work, you rescan it all from the start with KMP if it is doing too much.
One little fix might be to only add to the work counter and compare it outside of the main character comparison loop.
But it would be better to use the linear version of Hashchain. It also uses KMP to make it linear, but it is fully integrated and you would not need to track the work or restart scanning at all. And its a lot faster than Horspool anyway!
My Java profiling knowledge is... let's call it "antique". I was really not looking forward to ramping back up for this work. Turns out, I didn't have to do any of it. The LLM chose the tools (flight recorder) and even built a JMH (also new to me) harness to experiment with different algorithms.
About half of the optimizations were things that I would have figured out on my own; the other half were definitely "wow" moments.
The whole thing was done in a couple hours, with just a few back-and-forths. Sans AI, it would have taken a week, with nowhere near the same gain. I'm impressed.
"Figure out how to make this process fast" is really a perfect activity for LLMs. And the prompt doesn't really have to be much more sophisticated than that.
do you have some samples? It would be interesting to learn what it might be.
* Using spherical points instead of trig to calculate distance between two geo locations.
* Packing data to minimize memory bandwidth consumption. Converting arrays of objects to multiple arrays of their component parts I sort of expected; bitshifting to pack and unpack multiple values into a `long` I did not.
Maybe other people would find these obvious, but I don't usually have to optimize at this level. My mental model of the relative speed of some CPU operations was a little out of date.
JMH as the framework to write microbenchmarks. It takes care of dealing with JIT warmup, etc. It's the standard way to write rigorous Java microbenchmarks.
async-profiler (https://github.com/async-profiler/async-profiler) for profiling. Java has a problem where many profilers are based on safepoints, which are biased toward particular program points. async-profiler is not biased in this way.
Java Flight Recorder for memory allocation data.
One thing I've observed in all of this is that it's really useful to have expertise in the programming language and ecosystem you're writing in, otherwise it's all Greek to you and you can't really guide the agent to do the right thing. I have opinions about e.g. profilers and I can point the agent to one that I think is more accurate than other options.
It works, it’s at least 5x faster, sometimes much more, and memory use is like 10x less and even less in cases where lots of map tiles are involved.
This shit rules.
What I’m wondering now is can we reliably evolve python and have codex act as an extremely unreliable transpiler to the rust.
Why you even start with Python at this point? Just write the Rust version straight up instead of porting things?
Personally I used to use dynamic languages for most things, because development and maintenance is so much faster and easier, particularly for larger projects (granted you know how to work with those sort of languages), but now when the LLM writes most of the code, I'm able to work as fast with Rust as with I used to be able to do with Clojure or other dynamic languages.
I mean what happens with network engineers when the biggest network of them all goes down?
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
Hasn’t been my experience at all. The latest LLMs can knock out assembly optimized subroutines and benchmark 100 different variations faster than I ever could dream of.
To correct those behaviors, you're going to write tools and skills, and that's going to help, but it is still clear that you are fighting the distribution (today).
If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says.
Granted, this does require you to not use obviously dumb models, like anything you can run locally today, and at least within reasonable range of SOTA models. But they been able to do this for 6 months or more at this point.
What is the most complicated thing you’ve built with LM agents? Have you done it with a single spec? How novel was it?
This comment is so laughably “you’re holding it wrong” I can’t respond to you seriously.
> If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things
The set of software that has followed this process is measure zero.
A story.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
Another possible confusing thing for an LLM is that getting close to max doesn't necessarily require any "tricks." A big part of getting in the ballpark is just not doing anything you don't have to. If program A is faster than program B, most of the time is not some magic algorithm. It's that program A just did less stuff.
> Maybe I should document what I did!
Please do!
Me and a bunch of friends worked on a project that used diamond-types as the backing CRDT engine. It does indeed go brrrr. But damn, it took way too long to reconstruct what it was doing (we needed some more fine grained knobs, so we were playing with the frontier directly). We eventually moved out to something a bit better documented, which was a real pity. I really liked the general architecture and simplicity (of the text-only based version at least).
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.
Reasons this doesn't follow: (1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising) (2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.
Contrary to what initiatives like the tigerbeetle team is doing with tigerstyle, or the 10 nasa coding rules, code created by llms tends to be verbose and slow.
Lol got my mac M5 128GB I'll import a JS framework for multiplying numbers
This person doesn't understand how to make efficient code. I can write code in almost any language (with a couple of exceptions) that outperforms "super-optimized assembly". Writing efficient code isn't about the language, and often isn't about the best algorithms either (but sometimes it is). Its about optimizing memory and cache use. And that's orthogonal to anything the author is writing about. Also, LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
The points I take away are:
- Good optimization is difficult and slow work, hence expensive, but LLMs can do it so we should be able to afford it more often now.
- There’s always a risk of over-fitting to your specific problem, but if everyone is now making bespoke optimizations maybe that isn’t actually a problem.
You said:
LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
There’s probably something in that, but it can be mitigated by testing against a local benchmark. LLMs are good at iterating tirelessly and finding incremental improvements. And as noted above, it doesn’t necessarily matter if your benchmark isn’t fully general.
The author is one of the most knowledgeable people about performance there is
If you actually know what you're doing in $language, and you know how $language wants to emit the assembly or whatever, there's no huge advantage to just programming directly in assembly.
The app and service are measurably and subjectively faster. Because I chose to have the LLM focus on solving those problems. It obviously can. It described the issues in big-O.
It is a priority problem, as it always has been, not a knowledge or skill problem, like it always has been.
> The other day, I saw a viral tweet saying [...]
I've already posted this elsewhere, but here it is again: https://news.ycombinator.com/item?id=49226923
A vibe-coded OS that runs on an 8088 with 256KB of RAM.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
"It's not the tool, it's how you use it..."
I’ve also worked on projects where the people in charge added 1mb client-side mapping libraries to render a static map and ignored my push back. Those website were slow
After crossing 40 years of age, and working for a while now, I believe it's also because of politics. You might think the goal is to deliver the best possible product, but territory grabs within companies are important, and done by people that don't have enough skill other than territory grabs. E.g. look at Trump and his behavior. No skills other than having his way, and then he gets to decide.
Of the technical reason, for sure we underestimate how much faster technology gets. Fred Brooks had this example in his Mythical Man Month book, how the os/360 got the option of a disk drive instead of mag tape, but the result was worse because everyone assumed it was much faster than it really was.
And that software is build by some of the highest paid software engineers on the planet with full access to all the LLM compute in the world.
Toxic corporate culture can go a long way to self-sabotage.
I think the "maturity and humility" phase of the LLM hype cycle is probably still 2-3 years into the future.
And that phase probably comes immediately after the "LLMs overinvestments have caused a massive global recession".
Big, feature rich, built in quick iterations - but also, in particular if you look under the hood, of extremely poor quality if measured by traditional software engineering standards (code structure as exemplified by the leaked Claude Code source code, resource usage, "buggyness" etc.).
Now look at what software they make. It speaks for itself.
I've been really, really impressed with how effective this is. I went from a 4s load on simulated slow 4g to ~750ms: https://image.non.io/speedup-graphs.webp
Side by side vid of the results: https://video.non.io/speedups.mp4
This was for https://non.io, which is something I had purposefully written to be as fast as possible (hand wrote all the comopnents, didnt even use react).
I've been considering creating a skill / utility to do this based on learnings from the speedups - would others find this kind of thing useful?
- in safari, every image is loaded twice, .heic and .webp
- default.png is re-downloaded 31 times, uncached
- images below the fold are request immediately, lazy loading would avoid that [1]
But most important, you have 229 requests for tiny files being served over HTTP 1.1. Without GZIP. From a pretty slow server - 700ms+ to download the main json data. Bundling your JS, or enabling HTTP2 or QUIC/HTTP3 alone would massively improve performance.
If this is the result of days of autoresearch, it's not really anything to celebrate.
[1] https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/...
And there are plenty of things you did not opimize for. Like making sure certain dimensions are already known to the dom renderer so that the layout doesn't jump around.
Or progressive images so that it doesn't just popup suddenly.
Maybe hugged but feels really sluggish to me for it is.
Should note the server is on USW and I don't have edge servers for it at the moment.
and then you threw it all away by adding transition animation
That seriously needs some optimising. For example: on click, could the bigger image be inserted behind the small one so it's hidden; on load of the bigger image, hide the smaller image; then do the slide-bigger transition with both images together?
