Posted by tnisonoff 5 days ago
So the comparison is not only "built with and without LLM" but "would you even build this if you didn't have the LLM?". The gap in productivity in this case is much more wide.
Without AI they might have first spent more time validating the idea was worth it.
Seems optimistic
* Learning stage: 0.5x - 1x. I change my system prompt to teacher mode, taking the productivity hit for actually learning the system/tool pays off dividends later. I change my system prompt to "teacher mode" and slowly loosen it as I get more confident.
* Working-knowledge: 2x - 3x. Once I am ramped up enough I feel like I can get a decent productivity boost. Most of the time is spent at the planning stage. This is my mode for areas I don't really own or care about, just need to get work done.
* Mastered: 10x+ I have been doing web front end for 12+ years, I can quickly review plan/implementations and for my initial prompt I already know most of what I want built.
1x == my speed before AI
This really needs to be calibrated to the type of work and complexity.
I can actually believe that LLMs would speed up basic web dev work in small, simple codebases 10X for simple requests.
These conversations usually turn into people talking past each other because they’re working on different things. For other less routine and more complex work, expecting a 10X productivity boost is not realistic at all. It doesn’t matter how good you get at writing prompts and reviewing plans. LLMs just don’t solve everything for you in a good way. Some times the true nature of the problem is revealed while implementing it and by deferring everything to an LLM you spend days throwing tokens at the wrong thing. There is a lot of work where the LLM speed up comes from helping you quickly search docs and codebases and double check your code, but handing the entire thing off to an LLM isn’t reasonable. These tasks aren’t going to reach this mythical 10X productivity boost that is genuinely achievable for much simpler work.
If you look at the code the LLM spat out (and you really, really should!) you will immediately notice that its shape is not what you thought it should be. You might not notice immediately if you're learning, but if you really "mastered" the domain, you will "just" see it. You will also recognize the problem with your assumptions, and immediately (or after some research) correct the prompt.
Looking at the diffs for everything the LLM does slows you down, of course. Not looking - or looking and not recognizing problems, for one reason or another - can be initially faster, but a single pathological case can eat both the time saving and tokens. For domains you truly "mastered", the current models can generate code as if they read your mind (because you can be that precise in the prompt, and quickly), so it's really glaring and very hard to miss when they somehow misread your mind.
It only works at the "mastered" / "unconscious competence" stages, and only in those narrow domains you truly mastered, but it does seem to work. Is it 10x? No idea, but there is a marked change in the speed boost when crossing from conscious to unconscious competence area, with everything else (harness, model) staying the same.
- 25% planning & aligning with other teams
- 25% coding
- 25% testing/verifying
- 25% code review/rework
One argument was that agentic coding speeds up that coding part a bunch. So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do.
You could argue that it speeds up development by 5X or more, but then it slows down testing/verifying, code review, and in many cases it makes it impossible to review/rework by hand.
I just do what I'm told at work but even though I'm sure I'm in the minority I'm extremely skeptical that LLMs can produce any good-quality code.
Looks to me like we've just lowered the bar--by a lot--and stopped looking at the code that goes out. Every time I look into how Claude implemented something it's completely insane, with no way to refactor it or maintain it in the future.
Amdahl's Law should be familiar to anyone with a 4y computer science/engineering degree. Why aren't they applying it to their own throughput?
- 25% planning & aligning with other teams
- 25% coding
- 25% testing/verifying
- 25% code review/rework
I'd say that thanks to LLM assistance I'm 10x faster at coding, code review/rework, and testing/verifying. (LLMs can partially automate testing/verifying too, and the code is higher quality now as well so less testing/verifying is necessary).
So that leaves us with:
- 92.5% planning & aligning with other teams
- 2.5% coding
- 2.5% testing/verifying
- 2.5% code review/rework
Obviously, that makes zero sense as a split. If you saw any organization doing that, you'd suggest having fewer teams, more silos, etc. Maybe you have designers produce code, instead of showing the designs to coders and having the coders implement it. Maybe you force all your engineers to dogfood the product that way they can identify issues themselves rather than needing QA teams to do it. And so on. so the last category, "planning & aligning with other teams", falls too.
1) because they’re not doing those others spheres of work
2) because they don’t think about the work
nor how tired they are afterwards
3) they’re proselytizing AI work as the future and
that conflicts with that vision
4) the other shoe hasn’t dropped
5) they really don’t see it
6) some people genuinely hate programming and this
helps them skip that.
Suddenly devs who were cranking out features with no interest in infrastructure are attacking giant refactors to make the code more understandable to the LLM.
Other devs are using LLMs to build themselves quality of life SDLC tools completely separate from the core code base.
Plenty of other examples of this.
Of course, the main issue is that they’re completely undebbugable now. My bash scripts used to be a sequential list of commands, now they’re 500 lines of variable laden functions.
Is my life any better? Dunno. But it’s satisfying (until there’s a bug)
Well, that means the quality actually dropped then :). Looking impressive isn't equal to quality, understandability and reliability is
A node glob() or a regexp string.replace call is probably easier to read than spaghetti shell.
And your llm might do a much better job of creating clean, readable and testable code.
From a firm that has fully embraced agentic engineering, they offered topline stats on their measurements of a 4x boost from November to February, and 8x on top from February to May. That's 32x since November 2025, coding with LLMs.
And like whatever, this is second hand and I'm not going to disclose the source. Actual hard, published, and peer-reviewed research is needed here to backup quantitative claims. 2x, 10x, 32x, whatever.
The 2nd thing is, how do I measure that.
---
In my case, the details of my work (Kinda DevOps, kinda Senior Dev) makes it that having an LLM to do the heavy lifting allows me to do things not only faster, but better, and across domains I do not hold expertise on.
An example of the effect of LLMs in my daily work is that I'm in the middle of a PHP upgrade for a rather large legacy application, and the "heavy lifting" is really out of the scale, letting me concentrate on what really matters, while at the same time if I do keep "the harness" tight I'm certain the results are the correct ones. Also correcting course is just as cheap.
Not having to worry on the tooling on exchange has the incredible desirable result of my velocity being incomparable to what it was before.
Then we have the side effect of how easy to do transfer knowledge: Rather than telling the QA guy how to do the work specific for this task, I defined a set of files (.md documents, skills, an off-the-shelf customised MCP server) that assist QA into doing the work in a way that helps me do my job better and faster.
There's also a clear possibility that what I'm doing will expand to the rest of the team I am in, completely altering the way in which we approach development.
If we take 'x' as 'mileage', yours might vary. Mine has, and I'm baffled at the positive net results I AM getting.
Also, this what I do (coding?) is extremely fun again.
Being able to work close to the speed of thought is the best high.
How would you know it's better when you have no expertise?
There are many domains where an intelligent human can act as a discriminator for output without knowing exactly in precise detail how the process itself works.
Really depends on the task, my the project and on, and honesty, my instrest and availability at the moment.
The precise and rigorous practice of “engineering” in 2026.