Posted by rzk 10 hours ago
Ask HN: How to deal with shipping AI-written code you don't understand?
https://www.linkedin.com/posts/ali-hamie_im-an-engineer-at-m...
> I'm an engineer at Meta and I never felt this uncomfortable doing my job. I'm writing code these days and it all feels like a black box. This new standard of shipping with AI is moving so fast that if we're going to spend our time checking each line of code, it's going to take forever. That means you can end up with systems you've built where you don't truly understand how all the code works. You understand the high level, but not every detail. And maybe this is becoming the new standard. It goes against every instinct of being a software engineer. I'm used to understanding what I'm building. Now I'm trying to figure out how much I need to understand before I'm comfortable moving forward. Are other engineers feeling this too?
How are you handling this?
By that I mean that, rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.
The key thing here is that, by doing this, I DO know how the code works. I may not know how each unit is implemented, but I know each unit’s responsibilities and how they interact with one another. I’ve found that LLMs are HORRIBLE at writing decent abstractions, regardless of how much prompting and skills I’ve given it. It will always get the job done, but it will often be unmaintainable and require bigger and heavier refactors as I introduce more and more requirement changes.
But by flexing my basic design skills, I don’t have those issues. Future refactoring fit into easy abstractions that I’ve designed. And by documenting it up front, I also have what becomes a living document on how this work will turn out.
It takes a few hours up front to plan ahead, but it makes the LLMs job easier and use less tokens, and it makes the code better long term. I don’t care if one unit is suboptimal, as long as the abstraction is right — because at worst I can change an implementation for a better one later.
I'm old enough now to be a protected class, so I don't want people accusing me of ageism, because that's not where this is coming from - but there's a type of person that clearly is on their last job of their career and knows it and I try to avoid them like the plague, I've seen it too many times.
I just approve PRs, because I have to. I don't know what else to do. I can barely keep an eye on all my code.
Actively looking, but job market is rough in Northern Europe. I quit caring about quality in any capacity here, the company has clearly chosen its direction, against my very vocal complaints.
My last interview, with what I thought was a great company, failed when I didn't understand what NLP was when they asked me about it.
> NLP? > Natural Language Programming, it's the future. Claude, GPT, etc.
I laughed, he didn't, looked offended and said it was a serious question. I had thought it was a joke, said "oh you mean LLMs". I tried to answer seriously, of course, but that was my third round technical interview and I didn't get a call back.
Just getting a paycheck and producing mediocre work until I get something new. (Senior engineer, 10 YOE, patents, 3 years at a unicorn).
Also considering becoming an electrician.
Often, the lack of meaningfully challenging demands being put on you is punishing on its own. It can feel like a relief for a week or month or year, but if you catch a glimpse of that being your whole future, you can come to recognize yourself as being in a mind-eroding secular purgatory.
This anguish is especially common among on highly capable people, high achievers, and those used to some kind of ambition. And by accident of its history, there's a lot of those people in this industry and some of them are going nuts now.
Pretty much this. I know it sounds melodramatic but it literally feels like rotting.
I don't solve anything anymore. I don't get any spark of fixing bugs. I don't DO ANYTHING, and it's fucking chipping away at me.
Also I'm fairly sure people are going to realise soon that if they're happy with slop, they don't actually need highly trained engineers anymore.
This whole thread actually prompted me to look into electrician work, especially in renewables that we have here in Norway.
I might fix my burnout and go to school to get my 2 year qualification. AI can't rewire a hydro dam.
I went nuts for years, until I sold off my interest in my dream partnership.
10 years of heaven. All the benefits and none of the downsides of my internal colleagues. Then 14 years of bureaucracy aimed at predictability over progress, which was infinitely more difficult for me to dodge as an external player.
Compensation for my interest was very objectively valued, with the rate of stalled progress priced in. Which made complete sense, while also being its own madness.
I walked away from so much value, but I don't regret it for a minute.
No one cares at best, or I'm callously told "keep up or fall behind" at worst. When that's not even the problem, it just, sucks now. There's no joy in anything I do professionally.
Can I ask where you went? I'm seriously looking for an exit, I just don't know what yet. Maybe a trade, in Scandinavia the pay is actually comparable, I just don't want to go back to school.
Obviously there are mission critical systems where it matters, but if “your code can be A/B tested” or “slowly rolled out”, chances are, it doesn’t matter.
Tech debt is still tech debt, even if there exist models that can wade through, because it's going to require increasingly sophisticated models navigating increasingly large context windows and increasingly speculative workspace operations to keep making progress on it.
Vendors like Anthropic and OpenAI are laying a pit trap when they demonstrate how their models can independently code their whole platform as it becomes increasingly byzantine and inscrutable because not only do they benefit from internal/prerelease models but they have effectively unlimited spend because all that spending acts as a lure to draw you into their trap.
For everybody else, design still matters. People are just working really hard to make you forget.
Sometimes, not enough to believe you that you can do it better. And even if you could, to believe that it's overall worth it to work in that way.
It still applies to all things LLms can do from law to code to art to ideas.
They can do it - and very acceptably - but only to the untrained eye.
Way to come off as incredibly insecure. There's no need to randomly put people down.
That's it. Maybe you should go re-read?
Do you do this with every repository that's brand-new to you? You refuse to, for example, use an agent to learn where the feature should generally live? This seems absurdly slow for that purpose, because even though humans can still get a deeper understanding of a codebase after several months, an agent can get a much deeper understanding over a timeline of the initial few days. Unless everything is already meticulously documented for you, I suppose.
It can keep patching everything left and right whenever a new requirement comes up so it does not "need" to build abstractions, and it will only do it to please the human reviewer.
Are you working on a team? I've found that for solo projects this works but, even when you own the design of your piece, integrating with the work of several people doing this ultimately leads to even the part you initially designed no longer working exactly as you remember it.
A good chunk of my time now is spend working with an LLM to understand how other parts of the code base work and how they relate to and interact with the product features I manage. But I'm still on a small enough team that this is possible. I'm not surprised at all that an engineer at an org the size of Meta would not even be able to perform this level of work in understanding what's happening.
I have found that making the AI look the plantuml and the rendering and reviewing before presenting it to me results in much more legible designs. To me the peak of this would be a shared interactive whiteboard w the ai during design.
Of course I only arrived at the framework after trial and error with an LLM showed me the shape of the problem and the framework presented itself to me.
Planning is fundamentally so critical to ensuring alignment and outcomes, that over time how one plans can be better trained into the process to help do more and more of it.
Lots and lots of testing. I usually interactively test with the LLM. Now that I'm spending less time, coding, I get to spend more time, testing.
In my experience, I test more, with an LLM, than I did, with my own code. I don't find coding errors, but I do find implementation errors.
On another note: I ran an LLM on a bunch of really mature code that I wrote, and could swear was "perfect," and it found a number of issues.
Nobody's perfect. Trust; but verify.
What have we gained aside from shifting where time and effort is spent in the design loop?
This method also helps with the classic problem of being unable to identify essential complexity up front.
The verification is what we always wanted but never had time for. For 30 years I wrote side project games without extensive testing or a fuzzer or anything because I wanted to spend my time writing the game. Now I can do that and still have the testing I always knew I should be building.
Right now, we're in a "honeymoon" stage, where non-engineers think that they can somehow, magically produce stuff that is comparable to what experienced and disciplined engineers can do.
That won't last long.
Despite all the caterwaulin', LLMs are just another engineering tool, and good engineers will figure out how to implement them, in ways that absolutely blow away the garbage produced by non-engineers.
It's like that rich, no-talent guy that buys a PRS guitar, and thinks he'll beat the experienced musician that still uses his beat-up old strat, but consistently makes hits.
- code? up - build budget? Up - developer output? Up - developer oprrational costs? Waaay up!
Why is this Bad? Every one involved profits somehow.
The goal is to get to a point where I can say with a good level of confidence that a change didn't break anything. It might be the wrong change, and it might be crap code, but I can ship it without anything blowing up. I've been using it for a while across a few game projects and so far it's worked quite well.
If someone finds it burdensome to review my 50-line AI-generated bug fix, adding 200 lines of tests only gives them more lines to review.
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them. It’s better to spend more time on the core implementation.
I respectfully disagree. IME writing a good test suite is one of the areas where current LLMs perform worst. They certainly like to write plenty of tests but those tests aren't necessarily adding much value and they don't necessarily cover the important functionality.
Given how heavily LLMs and specifically agentic workflows still rely on producing something vaguely correct and then iteratively applying guard rails to reach something that actually works I humbly suggest that trusting a very substandard test suite to be one of those guard rails might be unwise.
I realise this is heresy, but my conclusion is that unit tests aren't important any more. E2E testing on the other hand...
(I'm agreeing with you about the quality of LLM tests, I'm just trying to make lemonade out of all these LLMons).
Testing is NOT verification.
EDIT: Sorry, oops, it's also called a kind of verification.
But there is the problem that tests only check or verify a small sample of possible cases or inputs. They can never show that the program is correct. (Famous Dijkstra statement.) To know something is really correct you have to understand and prove it.
Black-box e2e testing isn't suitable, or practical, for a lot of applications. Unit and integration tests generally require you to understand the code you want tests for, to know what to actually assert
Are you just trusting the AI to do it all?
Even if your test suite tested every single possible behavior and code path, tests usually can't intercept all possible unintended behavior that an LLM might have added.
In all likelihood, most test suites don't intercept any unintented behavior (such as unintended internet connections).
That's not a big problem when a honest human has written or reviewed the code, because unintended code is usually easy to spot.
But an LLM might have dropped any whatsoever code: maybe some of your project's code resembles the one of a trojan in its training, so it completes it with the trojan.
Or maybe it found that if your server is unreachable, it can upload the client's data to a public Github repo; or that if there's no space left on the device, it can delete some of the user's folders, easy!
Would your clients be understanding if that happened?
Eventually, the tools become trustworthy, and we no longer need to worry about their products.
We're just about there, with frontier LLMs, but I don't think we're all the way there, yet.
English isn't a good programming language if you want to trust that the output will be as you want it to be. High level code has no room for ambiguity - that's why we use it. It's a concise, very explicit language to tell the compiler exactly what to do.
Dealing with an LLM is a fundamentally different thing. The reason it's able to speed us up is because it makes assumptions for us and reads between the lines. The only world where we trust the output is one where it can literally read our minds. That's not coming any time soon
That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.
I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.
I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.
So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.
I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.
That doesn't mean I don't like progress mind :) This morning I got the preliminary results of a 'C' front-end to the compiler (it uses clang to compile to LLVM IR, then translates to our IR, then feeds that IR into the autovectoriser as if it had parsed the code itself). So for the exact same source code I now have:
┌─────────────┬────────────┬──────────────────────┬──────────┐
│ Benchmark │ clang -O2 │ xcc --par-auto (GPU) │ Speed-up │
├─────────────┼────────────┼──────────────────────┼──────────┤
│ saxpy │ 1,546 µs │ 726 µs │ 2.1× │
├─────────────┼────────────┼──────────────────────┼──────────┤
│ Mandelbrot │ 394,238 µs │ 2,110 µs │ 187× │
├─────────────┼────────────┼──────────────────────┼──────────┤
│ nbody │ 37,090 µs │ 1,035 µs │ 36× │
├─────────────┼────────────┼──────────────────────┼──────────┤
│ perlin │ 78,016 µs │ 1,850 µs │ 42× │
├─────────────┼────────────┼──────────────────────┼──────────┤
│ mandel_deep │ 73.2 s │ 55.8 ms │ 1,312× │
└─────────────┴────────────┴──────────────────────┴──────────┘
Hacker News isn't interested - any post I make linking to the webpages is immediately blocked/dead so I'll wait until the C++ front-end is ready (some time next week) and announce it somewhere else. A compiler that takes standard C/C++ code and speeds it up by automatically finding data-parallel areas and scheduling them on the GPU with no programming overhead is pretty rare.
Oh, and it manages the local network as well, so you can run xcd on machines on your lan, or via the TB4/5 bridge between Macs and it'll schedule on other GPUs, just like it manages multiple GPUs on the local machine. It characterises both latency and compute-time to figure out which devices are worth scheduling on.
For anyone wondering, the thing lives at https://compile-xc.org/ ;)
Still working on some of the data-parallel recognition in LLVM IR space. In order to support both C and C++ with the same data-parallel section extraction, I moved the detection from the ‘xc source code’ phase to the IR lowering phase, meaning that everything could ‘feed in’ to the same routines. That’s caused a little friction with the slightly different ways the LLVM IR from clang and xc’s IR encode loops. The above results are all single-loop extractions and the double-loop ones are still ongoing.
The system works in 3 stages: 1) by being greedy about which loops could possibly vectorise and marking them, 2) by analysing the IR to reject loops which do pointer-aliasing, or object manipulation, or… and then 3) by measuring the fastest way these known-safe kernels could execute.
Which brings me onto one nice thing I haven’t mentioned: that the GPU choice(s) are made at runtime, not compile time, and the system automatically measures (and then persists so it doesn’t have to measure again) the fastest way to run a kernel on the current machine. It handles MSL on Apple devices, PTX on Nvidia cards, Spir-v on Windows, Linux, and Android, and WGSL in the browser. It’ll quite happily divide up the parallel task across a 3090 and a 5080 on my Windows test box for example, with the 5080 doing ~2/3 of the work, because the scheduler is opportunistic.
All without writing anything other than plain C with nary a mention of the GPU. Give me a week or two, and I’ll probably announce it on phoronix (which might then get a mention on HN if it goes down well there, and it’ll be a different url so hopefully won’t be insta-dead).
I wouldn't be too sure about that :)
I think it’s my own fault, I’ve posted the link to the compiler pages several times in what I thought were relevant posts, and I think HN thinks I’m spamming. So I’m not posting the home page for a while and we’ll see if a future submission “takes”
Bottom line: GPL3 for the compiler, standard library exclusions for derived works, so use it for anything but feed me back improvements to the compiler itself, and you don’t get to redistribute it as your own.
If your definition of "works" includes "doesn't have vulnerabilities", you're going to need to do more than just poke at it and add tests.
A.k.a. "The concrete didn't crumble when I hit it with a hammer."
Process quality has been a thing for ages because it is essential.
Except it’s about 1000 “hammers” (aka tests), x 9 different architectures, x 2 different (a reference and a production) variants, and x 3 different optimisation levels. Every time I find something that slipped through, it gets at least one test written to cover it in the future. My main time-sink is the hours it takes to run the ‘gate’ before a release across the test-farm. And that’s after I’ve been playing with it myself for a while, pushing corner-cases through whatever I’ve been working on.
Jesus F Christ! No wonder most software lately (~decade) has been shit. Have y'all not heard of code review? Or did you think it just for show...you sit on a change for a week and then press "Accept" without reading it?
If you're an LLM prompter shipping products, you're outputs are more akin to a product managers, not an software engineer's.
It's hard to hold an LLM accountable for anything.
Software engineers are always responsible for their code except now they have to be responsible for Claude's code
Which is - of course - absurd. Accepting responsibility without demanding the corresponding authority is a fool's game.
I have never been much of an advocate for unionisation in the software development industry. I didn't think it was necessary to protect the interests of the workers because for most of the industry's history the workers have largely been able to stand up for themselves anyway. Likewise I've never been a big believer in accreditation and licensing practitioners in our field - because who is qualified to judge someone else worthy in such a young industry with so little hard data about what works or doesn't work? I find my views on these subjects are evolving now.
It's not about protecting jobs if they truly aren't needed any more or protecting people working in the industry who refuse to move with the times. The Luddite philosophy isn't going to get us anywhere here. And besides - if AI is as effective as its advocates have been claiming then there should soon be more demand for capable developers to take advantage of the new opportunities. (The Jevons Paradox is relevant here.) But it is about protecting standards. If the tools aren't doing a good enough job then expecting professional developers to ship the code produced by those tools even though they know it's junk is something that needs to be challenged. And if developers are being expected to ship anything important without having the slightest idea whether it even works then that's even worse.
We had a lot of silliness going on when the very low interest rates and huge amounts of VC funding were propping up ideas and practices that would never have survived otherwise. Today we have a different kind of funny money distorting the industry but it's still propping up ideas and practices that we wouldn't normally support or perhaps even tolerate. And just like before no individual is going to be able to hold back the tide and it's going to have to be a collective effort to push back - hence my evolving views on topics like unionisation and accreditation. We've all seen the drop in quality across much of the industry and we all know why it's really happening because we see how the sausage is now being made.
Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.
There seems to be disturbingly little objective evidence of that. There's certainly a lot of code being written but how much actual value that code is adding is another question. With the scale of resources that have been pumped into AI-supported software development over the past few years where are all the new applications and the quantum leaps in quality and performance in the applications we already had? If anything I seem to be seeing more anecdotes going the opposite way so far.
Disturbingly little evidence? I would hardly call that "disturbingly little"...
This does feel a lot like the "Agile era" that was the last silver bullet that was supposed to fix software development forever and the "Cloud era" that was supposed to fix infrastructure and deployments.
Agile had a few good ideas. Code reviews and automated tests are commonly used today.
It had a few plausible ideas that it took too far. We are better at recognising the value of incremental development over big up-front design today - despite the ironic clash this seems to be having with agentic workflows - but almost everyone hates stand-ups and lots of projects have gone off the rails by breaking things down far too much and never committing to anything and so never getting anywhere.
And it had plenty of ideas that almost all of us have thrown out as unpleasant and/or unworkable. Pair programming comes to mind immediately.
What didn't really change was the fundamental nature of software development. Did we adopt a few useful ideas and get a more more efficient and improve quality a bit? Certainly. Did we get orders of magnitude better at anything and give up all the tried and tested ways of working in favour of radical new ones that never quite felt right? No.
On our current trajectory I'm expecting a broadly similar outcome for LLMs. They're now very good at some things. But they are still pretty bad at other things that also matter - and some of the reasons they are bad at them are an inherent part of how they work and therefore probably can't be overcome (at least not with incremental improvements on the technology that still basically work the same way). So I am expecting to see a period of consolidation with a lot of organisations pulling back from their dramatic leaps into the unknown and downsizing of dev teams once the big AI companies start squeezing financially and the problems of developer burnout and stagnating skills and low morale that we're already seeing after just a few months become endemic. And then as I suggested in another comment I think it's also increasingly possible that we'll see developers unionise and/or push for professional regulation and licensing so they can resist the pressure to just ship whatever their agent wrote without even bothering to check it but still being responsible when inevitably things then go wrong.
Well, the fancy calculators do. Yes, they still need guidance to keep from going off the rails. But, at least from personal anecdotal experience, I need to go spelunking into the codebase less and less over time. The agents and models really did just start to become that good.
It's totally fair to load up some open source project into an agentic IDE and instead of asking it to write something ask questions like "how does X happen when I use feature Y?" or "can you explain how client code could accomplish Z"
All the time I see an AI use a language feature or library feature that I never used before or I see some code I don't understand and it does much better than a lot of humans do at explaining what it did.
I’ve been experimenting with writing a living spec — API endpoints, schemas, how things work at a low enough level that the LLM can fill in the details - and that’s been working well.
It also forces me to do the hammock-driven development necessary to build the right thing, and as a byproduct it keeps the design in my head longer.
Can you break down for me exactly how the unit and integration tests are made? Do you ask the AI what to assert?
There should be enough detail in the spec that it's pretty obvious. And unlike - many - human devs, agents don't shy away from writing tests. Add a coverage guardrail and tell them to use red/green TDD and you're already a good way there. The main thing is to ensure shared understanding, so they also know business expectations and write appropriate tests. And... nothing here is really specific to an agent, as a human dev also needs sufficient guidance (tailored to their experience level).
Here[0] we have benchmarks for AI writing tests for code that already has the correct behavior. Best case scenario is that they're at 67% with Fable, and that's with the highest quality specification we could possibly have (code). English is far more ambiguous
I have never in my career been afforded the time to make a complete spec.
Noone understood the code to begin with. People understand the concepts, the primitives, the "theory" as Naur said, the basis upon which a project is built. Noone ever understood a codebase worth understanding to begin with.
Just kidding, it's de same as with any other Open source plataform You install from their repo. It will be hard at the start but then You Will be very aware where it starts to fail and how to fix it.
I agree with the advice to design your system before implementing it. It is pretty easy to iterate on a design with your favorite coding assistant, and this is your chance to be ultra-picky and make sure you get exactly what you want. This was always where the "software engineering" was done, but in practice it's easy to have a vague idea and refine your (internal) list of requirements / tradeoffs as you type in the code. I find it harder to do that when the LLM is generating the code, just because one turn can change so much.
Keeping it under control requires discipline, just like before LLMs were a thing. You set a bunch of engineers loose on your codebase with no vision or supervision, you get spaghetti. You just get a little bit more control now. Agents don't feel shitty for the rest of the day when you tell them to do it some other way, so you are free to review as critically as you want without hurting anyone's feelings. I think it's a good thing.