Posted by softwaredoug 2 days ago
Usually, when people say AI code is terrible, it's because they either don't understand the theory well but have grown through hands-on experience and can't explain things properly to the AI, or they don't know what they don't know. Or there are the very few who are just far better coders than AI. Some people will say they're among the rare few who can write better code than AI, and for some that may be true. But in my experience, the vast majority are not. Even from my perspective as a beginner, I could see flaws when I looked at their git code. It's a metacognition problem.
Realistically speaking, at the script level, it's quite common to see AI surpass human programmers as you increase the input level. You might disagree, but that's probably because you're a specialist in that field, deeply immersed in a very narrow area—it only holds true in that limited scope. In the general domain, most people would agree that AI writes code well.
Human programmers don't know much outside their own domain. But AI, while it loses in very narrow specialist areas, writes better code than humans across the broader range. It loses in the 1% zone (the expert's domain), but wins in the other 99%. Usually, when that's the case, you have two choices: become the 1%, or learn how to use AI.
Since I'm a non-native English speaker, I'm already at a disadvantage compared to native speakers in programming skills, so I chose the latter. But I still code. Not for any other reason—if I don't maintain at least some typing muscle, I won't be able to review AI code properly.
That's why I think coding is essential. Even if I can't understand the entirety of AI's output, I still need to understand the core business logic. At the very least, the core logic requires human understanding, so coding is necessary.
And I should mention that I have 30+ years of programming experience.
I’ve stopped using llms to generate architecture, which i design and write myself and let the machine pattern match the gaps. I also use it to review issues which I lot of the times push back against.
I’m working on a stateful application sitting on top of a data warehouse and have to implement a stream of messy half defined feature requests and navigate on top of an ever changing infrastructure layer. LLMs rarely get the infra layer even if it is written as code and have hard time grasping how to deal with tech debt, when and how to re-architecture parts of the stack or even implement stuff based on a detailed openspec design.
Usually, it produces code that would take three or four humans days to figure out—in just 20 minutes.
Even the professors and PhDs who hire me all use AI. Honestly, they hold PhDs and professorships, which puts them in a league I can't even touch—and even they use it. AI just does it really well.
Honestly, I learned from your book, 'rossant'—I never expected a programmer like you to say something like that. I thought my perspective was because I'm only an intermediate-level programmer. But you're in the 1% expert category I mentioned
.... and in the and you end up with a very deep prompt that exactly specifies the behavior. This is what a programming language is.
I'd rather describe a data structure in a language designed for this task, than a prompt the might be interpreted in many different ways.
I imagine at competitive coding the goal is quite clear, but in a real world project, the goal is not always so clear, and especially in hobby projects the ideas and goals are not that clear. I get inspiration on how to improve my project or its usability, not the LLM. I instruct it to do something a specific way, because it doesn't do anything on its own, and I need to tell it what to generalize, which it failed to see, because it didn't consider a simplification which is technically less precise, but due to user context and human nature doesn't matter (in this case it was interpreting "now" to mean the current second, which is a small time range, instead of a mere point in time).
So it still takes a ton of hand holding in a more open project. I imagine, one could also code it up in the same amount of time. But it is good for generating tons of test cases. Though one will have to review those, and impose a test style on them, give examples and so on.
Beyond hobby projects, most clients often don't really know what they want. And that's generally what we call domain modeling. This is definitely an area where AI is weak. As you know, it mainly pulls from generic patterns.
When there are specific constraints, AI struggles with core business logic. And as you said, it's also weak at choosing the right direction or the goal to pursue. But as you also know, 80% of programming is built on what others have already created. Originality is only about 20%. And in that 80%, AI is absolutely dominant. I agree with you and I've upvoted your comment.
I really like your perspective
Interesting article btw
If you think that everyone agrees on the "correct" way to use it, you're mistaken. If you think that your way is the best possible way to use it, you're arrogant. And if you think that the way you think is correct is obvious and that everyone should already know that's the right way, you're delusional.
I've seen so many of these already. It would be hilarious to see Brooks proving right once again.