Posted by vnbrs 12 hours ago
When implementing its often a lot of misses with a few golden hits. The other day it used flex for a table layout while our app uses tables everywhere sigh.
Another typical one is that it tends to prefere frontend aggregation and looping of data instead of letting the database and backend deal with it.
Using mix of claude, cursor composer and codex.
TLDR: Keeping your codebase human readable and reason-about-able is not just helping humans to stay relevant. It will save costs for LLMs to maintain it.
Now we are getting to the point where we are speed-running the deskilling of engineers into comprehension debt and they themselves rapidly losing confidence in reviewing code they did not write.
I think this blog post [0] is the best example of what could go entirely wrong and even worse when you do not know the technology.
If you cannot explain a change even when "the CI is green" or "all tests passing", I will immediately reject it.
Maybe great for vibe coding prototypes, but it all changes when that code is deployed onto mission critical systems. Just ask Amazon with Kiro. [1]
[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...
[1] https://www.reuters.com/business/retail-consumer/amazons-clo...
How do you verify that it works?
json='{ "left":2, "right":2 }';
result="$(
perl -e '($_)=<>; / "left":(\d+), "right":(\d+)/; print $1 + $2, "\n";' <<< "$json";
)";
printf '%s\n' "$result";
Yet, it is literally the same as: printf '%s\n' "$(( 2 + 2 ))";However, if AI provides a solution, as the person using AI, one should conduct research before making a decision. This is not in conflict with or hindered by the use of the ideas provided by AI.
The obvious counterargument is "well, just ask the AI for those answers," but the AI lacks the context and experience that you have. Sometimes, genuinely, the user really is just "holding it wrong," but none of the current AI models would ever admit that, and you'd spend hours trying to solve an unsolvable problem.
For example, I use a vibecoded internal tool written in Go. I don’t even know how to write Go. Haven’t read a single line of the code. I just wanted to move from bash scripts to using cloud SDKs for performance reasons.
But the internal tool is a convenience tool, and you can do everything it does using alternative methods. So if it break, there is no real negative impact besides personal convenience of anyone using it. There’s some documentation on how to do everything manually if needed.
Here’s another example: you’re making a static website. No JavaScript, no interactivity. Truly, what could go wrong? And while I do understand HTML a lot better than Go, it wouldn’t really matter if I didn’t.
What is this supposed to mean? How is a “cloud sdk” more performant than a shell script?
There’s a bit less waiting around.
Linking a huge file consuming clients’s bandwith for no reason. Embedding PII in the html source? And if setting up your own server, misconfiguring it?…
You also don’t need to know how to read HTML to recognize large files. You can catch issues like this with a simple website performance testing tool like pagespeed.web.dev
I’m also not sure how PII would enter the HTML source.
Human developer can work on a program incrementally, ensuring at each step that it is mostly correct.
But LLMs can't think, they fake reasoning and explore problem space in random walk until they stumble into something that looks like a solution. And these "solutions" will have hilarious and absolutely unexpected failure modes.