Posted by todsacerdoti 2 days ago
I’ve sure used various LLMs to solve difficult nuts to crack. Problems I have been able to verbalise, but unable to solve.
Chances are that if you are using an LLM to mass produce boiler, you are writing too much boiler.
1) Abstract data showing an increase in "productivity" ... CHECK
2) Completely lacking in any information on what was built with that "productivity" ... CHECK
Hilarious to read this on the backend of the most widely publicized AWS failure.
The first paragraph of said guidelines reads:
What to Submit
On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.
After the n-th iteration of the same self-congratulating, hype-pushing, AI-generated drivel, the point can be made that the original submission does not meet the HN guidelines.
To quote Jimmy in South-Park: it's an ad.
Comments like the one I replied too just make HN seem mean and miserable and that's definitely something we're trying to avoid.
Comments like the one I replied too just make HN seem mean and miserable and that's definitely something we're trying to avoid.
And guess what happens? Reality doesn't match expectations and everyone ends up miserable.
Good engineering orgs should have engineers deciding what tools are appropriate based on what they're trying to do.
But I think the objections can mostly be overcome with a minor adjustment: You only need to couple TDD with a functional programming style. Functional programming lets you tightly control the context of each coding task, which makes AI models ridiculously good at generating the right code.
Given that, if most of your code is tightly-scoped, well-tested components implementing orthogonal functionality, the actual code within those components will not matter. Only glue code becomes important and that too could become much more amenable to extensive integration testing.
At that point, even the test code may not matter much, just the test-cases. So as a developer you would only really need to review and tweak the test cases. I call this "Test-Case-Only Development" (TCOD?)
The actual code can be completely abstracted away, and your main task becomes design and architecture.
It's not obvious this could work, largely because it violates every professional instinct we have. But apparently somebody has even already tried it with some success: https://www.linkedin.com/feed/update/urn:li:activity:7196786...
All the downsides that have been mentioned will be true, but also may not matter anymore. E.g. in a large team and large codebase, this will lead to a lot of duplicate code with low cohesion. However, if that code does what it is supposed to and is well-tested, does the duplication matter? DRY was an important principle when the cost of code was high, and so you wanted to have as much leverage as possible via reuse. You also wanted to minimize code because it is a liability (bugs, tech debt, etc.) and testing, which required even more code that still didn't guarantee lack of bugs, was also very expensive.
But now that the cost of code is plummeting, that calculus is shifting too. You can churn out code and tests (including even performance tests, which are always an afterthought, if thought of at all) at unimaginable rates.
And all this while reducing the dependencies of developers on libraries and frameworks and each other. Fewer dependencies means higher velocity. The overall code "goodput" will likely vastly outweight inefficiences like duplication.
Unfortunately, as TFA indicates, there is a huge impedance mismatch with this and the architectures (e.g. most code is OO, not functional), frameworks, and processes we have today. Companies will have to make tough decisions about where they are and where they want to get.
I suspect AI-assisted coding taken to its logical conclusion is going to look very different from what we're used to.
> I suspect AI-assisted coding taken to its logical conclusion is going to look very different from what we're used to.
100%. I now design new libraries so that AI can easily write code for them.
This is really something striking to me about all these AI productivity claims. They never provide the methodology and data.
Fine for just you. Not fine for others, not fine for business, not fine the moment you star count starts moving.
Congratulations, you invented end-to-end testing.
"We have yellow flags when the build breaks!"
Congratulations! You invented backpressure.
Every team has different needs and path dependencies, so settles on a different interpretation of CI/CD and software eng process. Productizing anything in this space is going to be an uphill battle to yank away teams' hard-earned processes.
Productizing process is hard but it's been done before! When paired with a LOT of spruiking it can really progress the field. It's how we got the first CI/CD tools (eg. https://en.wikipedia.org/wiki/CruiseControl) and testing libraries (eg. pytest)
So I wish you luck!
Then they claim (and demonstrate with a picture of a commits/day chart) a team-wide 10x throughput increase. I claim there's got to be a lot of rubber-stamp reviewing going on here. It may help to challenge the "author" to explain things like "why does this lifetime have the scope it does?" or "why did you factor it this way instead of some other way?" e.g. questions which force them to defend the "decisions" they made. I suspect if you're actually doing thorough reviews that the velocity will actually decrease instead of increase using LLMs.