Posted by dimonomid 13 hours ago
We (collectively) were unprepared for a machine that presents itself in human forms. We were the frogs that boiled ourselves. We built a world of images and words on a screen. And then we built a machine that can (increasingly) mirror that world; it does so in a way which most of us are incapable of disambiguating.
It feels like there is indeed a ghost in the machine.
And there is, but that ghost is us. And that ghost is fading surprisingly quickly.
Maybe you are right, maybe not, let's see. Unfortunately I kinda agree, because humans are really good at being lazy and going in the path of least resistance (including me). It's genuinely difficult to not use AI even if it makes my work worse, as long as it's easier and faster.
I personally don’t trust coding agents to have enough context to write domain-specific table schemas, and I don’t have the patience to transcribe all of the context into a natural language prompt. If I ask it to, it’ll write something for sure, and maybe that can be a jumping point for me, but at some point I have to physically write what the columns will be.
No, the LLM doesn’t know our product strategy and why certain things matter and certain things don’t. That’s very much what I get paid to do. There’s not even agreement within our team about what path we should take through the domain-product space, there’s no chance in hell that the LLM will choose a profitable random walk through that domain-product space.
If it were able to do that, then AGI would have already been achieved and we’re only compute power away from OpenAI or Anthropic making the marginal utility of any piece of code $0.
LLMs are not a "random walk", they take in information and they explore the space according to the way they've been instructed.
But reading code? What does that accomplish, other than to slow your dev process down enormously? Serious question.
I'll let you know how it goes... My new VP of engineering is a 'no looking at code' type of guy and is ripping 10K LOC PRs / Docs / plans against our 25 year old codebase and I would not say that they're 'good' PRs.
Maybe I'm completely wrong, but I think reading the code is more valuable than ever when working in a full-stack / small company role. I can tell you exactly what the business logic or functionality is for a certain piece of our system, in truth, without having to step through and make sense of ambiguous docs (that were also AI generated).
(I have a sneaking suspicion that in two years or less, my small team is going to significantly compromise the integrity of this codebase. Maybe by then we can refactor with GPT 12.)
This is the classic "make no mistakes".
On a serious note, I might set as criteria "avoid code duplication". Does that mean that the model/agent will actually follow it?
> What does that accomplish, other than to slow your dev process down enormously?
I am an OSS developer and I often see PRs (i.e. from the general public) that look correct, pass all CI checks, are heavily documented and they are still wrong.
Most of the times either they duplicate code that already exists somewhere else, or they implement a "feature" by opening a can of worms for subsequent "features" in the same area.
Hmm, I have deep concerns about what and who is going to cease to exist.
At least I did not find a new thought in that (granted, relatable) rant.
"This is bad and you are bad" requires people to not defend their reality through rationalization, but the point we're at with AI right now is driven by exactly that. So this is at best highly ineffective at reaching the people it claims to want to reach.
That said, the underlying emotion of "you all suck and I hope you lose your jobs you frauds" is relatable and worth screaming from the rooftops of Linkedin dot com for the catharsis alone.
AI is nothing special or new in this regard. It just gives a single guy the velocity to ruin a codebase at the rate of a full enterprise team at double the speed.
A shitty code base still makes money, and that's all that will ever count for the majority of the employed developers, those who don't blog.
The short version of the result is that we're completely copying the leaders in our market that have 100x our marketing budget because they're the only ones with documented strategies for AI to cheatsheet from, with no sense or irony or concern that perhaps their scale is a core part of their strategy.
It's not just institutional knowledge. We're suffering the death of the specialist. Now every generalist is pulling triggers with no sense of limitations.
Personally for my job I struggle to justify writing code by hand. Agents are much better at typing the code, reading the code, finding patterns and discovering bugs. They don't have my knowledge and my background so the current models still miss things or overcomplicate the code. I don't know if that's going to be true of the next generation of models (or the one after that).
With current models, I cannot fully give in to the vibes. I _have_ to look at code (maybe not 100% of it but at least a majority of the lines). I _have_ to grill the agent to make sure it writes the best code that I deem possible for the problem at hand. This isn't the fastest way to do agentic development, but AI agents have already made us significantly faster than we were in the pre-AI era. I don't want to squeeze every ounce of 'development speed' if that comes at a cost to code quality, maintainability or debuggability.
For my side projects, things are different: either I only care about the outcome (and I give in to the 'vibes') or I care about the journey just as much as the destination in which case the agents are extremely advanced search engines, code reviewers and mentors but they don't get to write any of the code.
Yeah, I guess I personally fall on the whole spectrum: for side projects I'm at the extremes but for my professional work I fall somewhere in the middle.
We've had strong coding agents for less than a year. Anyone making such a definitive statement about how vibe-coded projects progress over time is basing it on guesswork, not evidence.
"A Project must have proper tests and specs, and only incidentally for a working program that executes"
https://12factor.net/ https://en.wikipedia.org/wiki/Twelve-Factor_App_methodology
Kevin Hoffman expanded on that with the 15-factor app: https://developer.ibm.com/articles/15-factor-applications/
Mind you I may be dating myself as I was first introduced to this paradigm in 2015 working as a Java SpringBoot engineer on an enterprise project that I then migrated (57 microservices) all to Scala, after onboarding two weeks to Scala fresh from no prior Java experience.
I feel like there is so much "wisdom" encoded in books and writings from some of the most prolific engineers and architects over the last several decades.
Look at Matt Pocock's skills with simple primitives like grilling the human, researching through wayfinder maps (a Godsend to my workflow prior to Cursor Projects and orchestrator patterns), and having a solid Domain Driven Design through defining a shared glossary and breaking up work around proper seams.
It is perfectly possible to vibe-code a badly designed app that still passes those 12, 15 or whatever points you define.
Some other ideas
1) Enforce architecture decisions (see archunit). But somebody needs to write them down first.
2) Check that tests actually break if the code that accompanies them is removed (several LLMs/agents today create tests that don't actually test the code they "guard against)
3) Automated performance testing. An LLM/agent might create a change that is "correct" but increases latency for 3x (best case) and 20x (worst case)
The hardest part that I see no solution for today is to understand when a change breaks backwards compatibility. LLMs/agents are trigger-happy and will happily refactor/remove stuff without any care about who is using that.I don't have a proposal for that, but the problem is there and is not covered by 12-factor config.
1) A lot of the time i spent deciding on interfaces (methods, classes, etc.) for humans. E.g. should this be two methods or one, should this method be in this class or moved to utility. Those problems went away. 2) What about performant code? This can be prompted away and when the measurements in your performance tests do not go down, then you can step in. 3) Sad to say but the AI has always been better than me at code-reviews. Maybe this is just me and if so I own that, but to the articles point, it might be harder to fix now. 4) "vibe-coded projects devolve over time into an unmaintainable mess". Preventing and managing this mess is the new skill sets we need to develop as software engineers. 5) Another skill-set we will need to master is how to maintain and grow our coding skills. Some ideas are: a) every once in a while implement a feature yourself. b) no AI Tuesdays! c) Have the AI quiz you on the code base. d) Have the AI develop HTML docs about how the code works.
One set goes into haxe files and I use reflaxe macros to write tiny compilers that generate docs, clients, servers, cli's, test cases, serializers and deserializers, etc in whatever language is appropriate. That leaves gaps, which the AI can then fill in.
So I iterate on the haxe stuff. If the AI is struggling to "draw the rest of the owl," I change the source of truth until it has enough guidance re: type related errors, failing tests, and documentation. As requirements change, these things change.
As for the rest of the owl, it's disposable. Every few months I'll delete it and have an AI rewrite it from scratch (now with a smarter model and better docs and API specs and tests to guide it). This keeps the cruft from accumulating while preserving human contact with the code.
How can we prevent and manage the mess if we're not actively working in the codebase though? (Sometimes, I'll get a 'feel' for when something needs changed or will become unmaintainable, but that required consistently interacting with the thing)
Currently, we look at code once during the PR and then we never touch it again until it comes up in the PR.
I'm still handling a few tickets a week without AI, because sometimes it's faster to make a 2 line fix than to write + review a prompt, but increasingly it's just to make sure I still 'got it'.
2) not even sure what you mean by this
3) probably shouldn't admit that. It implies the reviewer has a less than average understanding of the code they're reviewing.
4) preventing and managing vibe code devolving into a pile of slop requires programmers not use AI. 80% accuracy repeated in more and more layers === more and more failures. In other words, the skill required is exactly the skill of being a good programmer without AI.
5) e) no AI all the time or only use AI as search. You're almost there with a and b. With c, it just doesn't understand well enough to "quiz you". With d, how are you going to know if the docs are correct if you aren't reading the slop?