Posted by reasonableklout 5 hours ago
Even before LLMs generating entire programs, complex frameworks allowed developers to write the initial versions of programs very quickly, but at the cost of being hard to understand and thus hard to debug or modify.
Some of us are betting that the AIs will always be smart enough to debug, maintain and modify the programs written by AI, no matter how convoluted or complex. I’m not so sure.
I guess what I relate to the most is how dismissive people get about real software engineering work.
I may have skill issues, but I am yet to reach the level of autonomous engineering people tend to expect out of AI these days.
and we all live in a green utopia of flying cars and peace upon the world.
I like to think,
(it has to be!)
of a cybernetic ecology
where we are free of our labors
and joined back to nature,
returned to our mammal
brothers and sisters,
and all watched over
by machines of loving grace.
I know which outcome I'd put my money on.
I don’t agree, but that’s the thinking
...and it also needs more so-called AI companies present in the wreckage in this crash.
AI psychosis is undeniably real.
At the end of the day robots can do the vast vast majority of jobs better and faster. If not now, very soon.
I only worry our economic systems won’t keep up
But I only see mass layoffs and those who are working - are working longer and harder then before.
I really do worry - I especially worry about security. You thought supply chain security management was an impossible task with NPM? Let me introduce to AI - you can look forward to the days of AI poisoning where AIs will infiltrate, exfiltrate, or just destroy and there's no way of stopping it because you cannot examine the internals of the system.
AI has turbo charged people's lax attitude to security.
God help us.
Some time down the line, I discover CPU being maxed out, which is showing up in degraded performance in other parts of the system. I investigate, and I trace the issue to a boneheaded busy loop in this library that no human with the domain expertise to implement the library would have written. Turns out I'd missed one deeply-buried mention in the README that maintenance was being done via AI now, and basically the whole library had been rewritten from the ground up from the reliable tool it used to be to a vibecoded imitation.
Yeah, yeah, sure, bad libraries existed before all this. But there used to be signals you picked up on to filter the gold from the dreck. Those signals don't work anymore.
In any case, this is what blue-green deployments and gradual rollouts are for. With basic software engineering processes, you can make your end user experience pretty much bullet proof. Just pay EXTRA attention when touching DNS, network config (for core systems) and database migrations.
Distributed systems are a bit more tricky but k8s and the likes have pretty solid release mechanisms built-in. You are still doomed if your CDN provider goes down. You just have to draw a line somewhere and face the reality head on (for X cost per year this is the level of redundancy we get, but it won’t save us from Y).
The one thing I hadn’t mentioned - one I AM worried about - is security! I’ve been worried about it from before Mythos (basic prompt injection) and with more powerful models now team offence is stronger than ever.
Sure there are industry changing things going on. What if you're working on an app thats a decade old and has had different teams of people, styles, frameworks (thanks to the JS-framework-a-week Resume Driven Development)? Some markdown docs and a loop of agents isn't going to help when humans have trouble understanding what the app does.
It is definitely factual that there is a complete paradigm shift in the prioritization of quality in software. It's beyond just AI side effects, and now its own stand alone thing.
There have always been many industries, companies, and products who are low on quality scale but so cheap that it makes good business sense, both for the producer and the consumer.
Definitely many companies are explicitly chosing this business strategy. Definitely also many companies that don't actually realize they are implicitly doing this.
Wether the market will accept the new software quality paradigm or not remains an open question.