Where's the shovelware? Why AI coding claims don't add up

Posted by dbalatero 9/3/2025

Where's the shovelware? Why AI coding claims don't add up(mikelovesrobots.substack.com)

762 points | 482 commentspage 6

KETpXDDzR 7 days ago|

If LLMs make devs 19% less effective on average and the number of releases is growing the same, doesn't that mean that

1. Only a handful of devs use LLMs

2. For every developer getting less productive with LLMs, there must be developers that get more productive to keep the trend

djoldman 9/4/2025||

Faster? At making what? Pipe dev/null and you'll get a lot of stuff fast.

What if someone came out with a study saying they had a tool to make the fastest doctors and lawyers? You'd say that doesn't even make sense, what kinds of doctors doing what kinds of work?

AI coding isn't some across the board, helps everyone do anything kind of tool.

Maybe sometime soon we'll stop strawmanning this.

bmiselis 9/4/2025||

That's all based on the assumption that if you can build something in 10% of the time it'd take you to build the same thing without AI, then you'll spend this 90% of your new spare time to build something else next. What if you don't and you'll just use that time to spend with your family? The data won't show it.

sarchertech 9/4/2025|

Sure some people would do that. But plenty of other would use the whole 90%, more would use 80%, even more 50%… So you’d surely expect an explosion in new software.

giantg2 9/3/2025||

Until AI can understand business requirements and how they are implemented in code (including integrating with existing systems), it will continue to be overhyped. Devs will hate it, but in 10-15 years someone will figure out that the proper paradigm is to train the AI to build based off of something similar to Cucumber TDD with comprehensive example tables.

atleastoptimal 9/3/2025||

All these bearish claims about AI coding would hold weight if models were stuck permanently at the capabilities level they are now with no chance at improvement. This is very likely not the case given improvements over the past year, and even with diminishing returns models will be significantly more capable both independently and as a copilot in a year.

SchemaLoad 9/3/2025||

Sure, no one can say what the future will look like. The problem is these products are being marketed today based on what they might do tomorrow. And it's warping perceptions of management who get sold on hype that isn't real yet and possibly not for a very long time.

hinkley 9/3/2025|||

Hype cycles affect funding. When the Trough of Disillusionment hits anything that's being started will take years to finish due to a more difficult funding terrain.

The arrival of the Trough is predicated by the amount of lies and utter bullshit that have been shoveled out during the earlier parts of the cycle. So while it's unfortunate that the real goods don't get delivered for years and years after they might have been, it's typically and often entirely the fault of the people on the train that this has happened.

There's an awful lot of utter bullshit in the AI hype.

rsynnott 9/4/2025||

Ah, yes, jam tomorrow.

tobyhinloopen 9/4/2025||

> The last time I heard the phrase “continuous improvement” or “test-driven development” was before COVID.

I have prompt docs precisely on SOLID, TDD and all kinds of design patterns… but yes I see a lot of untested code these days.

AI has been incredibly helpful at analyzing existing, unknown to me, projects; basically for debugging and searching in these repo’s.

ramesh31 9/4/2025||

It's the cost. Full time serious agentic coding costs upwards of $100/day in Claude tokens (and Claude tokens are the only tokens worth even talking about). When this drops by 10x for a model at the level of quality and speed of Sonnet 4, it will change everything.

frays 9/4/2025||

Excellent article. It'll be really interesting to look back on this in 5 years and ask the author to regenerate these charts again to see if there is any impact.

Archived here: https://archive.is/WN3iu

ge96 9/3/2025|

I've already experienced being handed a vibe coded app, which so far it's been a communication problem/code cleanliness eg. don't leave two versions of an app and not say which one is active. And the docs man so many docs/redundant/conflicting.

More comments...