Posted by vantareed 13 hours ago
Perhaps the framing shouldn't be "haha slop" but rather why doesn't the AI write better quality software than we do? To which the answer is obvious IMO -- even emergent properties can't elevate AI intelligence too far above the training dataset. So how do we get to superintelligent (or at least "not-wreck-your-NVMe-endurance-telligent") AI, if we, as a whole, are not smart enough ourselves?
Judge not the slop-bot, lest ye be judged yourself, engineer.
My experience was always that 90% of code is ugly and clunky. I'm not at all surprised, while reviewing AI-generated code, to see many of the same ugliness we regularly commit. The quality of the output code is now consistently average, which means it's basically shit in 90% of cases, but it tends to mostly work (in the general case). The same kind of shit I've seen people push to production thousands of times in my career.
We don't fully know how to write good code. We don't really understand what good code should objectively look like. Spending more time on code doesn't automatically lead to better code (but costs a lot more). Above all, we don't need good code - the business side is perfectly fine with "good enough right now" rather than "maybe a lot better half a year from now". And that's what the models are trained on. They would, indeed, need quite a lot of "emergent properties" to go from that to consistently good code. ASI-level properties, I suspect.
Great! So next time the human will prompt the agent to watch out for and avoid this bug.
I actually created a system for something like this. The basic idea is, once you have identified what the issue was and fixed it, you can create lessons that lives inside the repository. Lessons are designed to be mapped to one or more files so if the LLM changes the files again, they can see what the issue was.
The main challenge is being able to summarize and create proper tags so the AI after any code change can easily find the lesson.
You can find it here: https://github.com/openai/codex/issues/28224#issuecomment-47...
I have been making noise about this bug for a week, so I'm glad to see this is blowing up on HN.
2. "One developer somewhere in the world made a bad mistake one time, so this represents the quality of all software devs everywhere". Maybe they were just a bad developer? Bad developers exist. I have never written a bug that has destroyed my users' hardware, and I think that writing such a bug is completely inexcusable in an enterprise environment with software that will be shipped to millions of users, as Codex is.
Probably whoever (human or agent) originally decided to put TRACE logs into SQLite also thought---or reasoned---so. Maybe the decision was right at that time but the amount of TRACE logs have increased enormously. You will never know.
Your comment, on the other hand, would be improved by including your own opinion on the matter.
/s?
They're clearly AI generated
Even with tests, the more complex the code base is, the more risky it is to vibe-code on it without introducing more bugs [0] and increasing the debt. Does not matter if the CI is green or if all the tests pass.
It gets even worse if you can't explain the change / pull request or what the implications are after applying that "suggested" fix.
[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...
Only now we're witnessing the consequences much more frequently thanks to accelerated slop.
The OS is a thin layer providing an abstract and consistent interface regardless of the hardware configuration. Policing applications is mostly related to security and resources utilization, not moronic errors.
This is called a hardware abstraction layer, not OS.
One can argue that these products are the flagship products of their respective AI companies aside from the AI models themselves of course.
I imagine that this story will be picked up by the news left and right, some stories just feel this way and this one is like that (given 12 upvotes on HN in 7 minutes)
The only logical conclusion (from this incident) that I can have is: An (vibe-coded?) product is hard to maintain even for some of the best engineers and is bound to have severe bugs.
2. Proper testing and taking issues seriously is the key if you still wish to do this and there isn't much. This is a week old issue which I can only classify as severe.
I wish to keep an nuanced opinion about it but oh this is bad for openAI (not as bad as them accepting autonomous AI within drones and mass surveillance though)
My point is: AI has both uphills and downward valleys and cliffs. It might as well just accelerate you, which could be, towards your downfall as well. Its recommended to keep an eye while driving and not drive too fast.
AI companies might be like car companies which don't offer a brake pedal.
because they trust the AI too much (and seem to be fin with acting knowingly negligent)
the problem is
- AI tends to produces very convincing looking code, even if fully wrong
- AI does mistakes of kinds no human would do, at least no human who is also able to write convincing looking code
- code reviews are hard, a lot of devs, including senior devs, put a lot of implicit trust into the co-worker behaving "sane and non malicious". But AIs behave sometimes not so sane and in a way (wrt. trying to be convincing). In the worst case in ways which if it where a human you might consider to be them trying malicious sabotage the product
Like a "dump" example from work:
- AI randomly removes a HTML element id while doing other changes in jsx/react
- the PR has a lot of changes, the id removal line looks innocent, like some on the fly cleanup
- human reviewers have the bad tendency to often not look too much at deleted lines, only if they need reference to how a new line was before (but it's only a deleted line and no new line)
- you don't expect humans to randomly without reason delete important properties of components when changing other things
- you maybe would still have found it, but it's a emergency fix for a production issue
- it happens to miss integration tests, but happens to still matter a lot for one specific important for complicated reasons not properly tested flow (similar people tend to not test logging too much, at best the presence of needed info but hardly ever the absence of noise)
I'd say this is also partly a problem of working under intense pressure and the demand to work faster and faster - even faster now with "AI". All these companies are competing with each other very aggressively and are driving their employees like horses in order to win the "AI" race.
Because it was deemed not Hard Enough task for real engineer to look at, so AI was sent to do it with no supervision, just checking the effects.
Also overly excessive logging is probably useful to them in chasing some of the edge cases, the cost to users doesn't matter in the slightest to them
People like to go on about how "good engineers review their AI code" but that's just not what's happening in reality. Not only is reviewing large amounts of AI generated code unpleasant and mentally taxing, it also negates most of the perceived productivity boost, so people are simply not doing it.
> Proper testing
There is no formal testing that would be expected to catch an issue like this. It can barely be classified as a bug, the logging is working as intended, just with negative side effects that weren't accounted for.
The only real way to proactively prevent an issue like this is for a human programmer to stop and think about this code as they're writing it and go "hmm, we're logging large amounts of data to disk at a fast pace here, this may be a bad idea". Without human involvement, this is just going to keep happening. All vibe coded software is bloated and unstable, I have yet to see a single counter-example.