Top
Best
New

Posted by rzk 19 hours ago

Ask HN: How to deal with shipping AI-written code you don't understand?

I saw this LinkedIn post from a Meta engineer and it resonated with me:

https://www.linkedin.com/posts/ali-hamie_im-an-engineer-at-m...

> I'm an engineer at Meta and I never felt this uncomfortable doing my job. I'm writing code these days and it all feels like a black box. This new standard of shipping with AI is moving so fast that if we're going to spend our time checking each line of code, it's going to take forever. That means you can end up with systems you've built where you don't truly understand how all the code works. You understand the high level, but not every detail. And maybe this is becoming the new standard. It goes against every instinct of being a software engineer. I'm used to understanding what I'm building. Now I'm trying to figure out how much I need to understand before I'm comfortable moving forward. Are other engineers feeling this too?

How are you handling this?

364 points | 500 commentspage 14
ulfw 16 hours ago|
It's Meta. Shipping crap is par for the course. You're fine.
nkohari 7 hours ago||
This was something I personally struggled with for a very long time, so much that we ended up abandoning the product we were working on to build a solution:

https://theseus.dev

Basically, Theseus reads your pull request and your codebase, categorizes the kinds of edits that were made, and writes human-readable documentation that describes what changed. Then, it shows the explanation side-by-side with the code, but presented in a way where you can decide what you want to skim and what you want to deeply review.

I don't think engineers should have to choose between wading through thousands of lines of slop and just smashing LGTM on every pull request. The problem is that the tools for code review are built for an age where code was expensive to write. Now that code is cheap, we need to start experimenting with tools that allow us to stay in sync with what's going on, while also not needing to review every single line individually.

remywang 10 hours ago||
Do not
dboreham 11 hours ago||
Replace "AI agents" with "a bunch of human coders" and you'll see the answer. Your product needs decent testing and it's your job as the responsible person in charge to make sure that's in place and working properly. So if anything you probably want to be reviewing the test code. But often that isn't necessary since you can do things like introduce bugs deliberately and see if tests fail; check test logs for information that confirms the tests were run as expected; try some manual testing once in a while; ask for new more advanced tests (e.g. test performance not just functionality); and so on.
chrisjj 11 hours ago||
> How to deal with shipping AI-written code you don't understand?

Recognise you aren't the one shipping it.

TZubiri 9 hours ago||
no one can force you, take responsibility for what you ship, what you choose to ship, how you choose to work, you signed that contract you go to work every day, your company is composed by people like you

don’t wash your hands, or at least don’t expect us to wash them for you

K-etoranger 4 hours ago||
[flagged]
edwin0 18 hours ago||
The part that makes me uneasy is when the AI writes both the code and the tests. They might agree with each other and still miss the same thing. How are people here checking for that without doing the whole job twice?
ch_sm 15 hours ago||
Not sure about that. This was structurally the same before AI: the author of the code wrote the tests. The tests still helped.
seanw444 11 hours ago||
The author of the code was physically touching the internals and potentially intuiting obvious shortcomings as they worked on it. That very crucial part is now gone.
Bootstrapper909 8 hours ago|||
We split them. The agent writing the code never writes the checks it's judged by, and those separate checks caught way more than its own tests did.
webarchitect 14 hours ago|||
I like fault injection (fuzzing, chaos), and mutation testing. I think introducing these helps the AI write better tests and helps having some confidence in the tests that are written.
fractallyte 13 hours ago|||
You should be using two (or even three) different LLMs. For example: Claude to do the coding, and Codex to do an adversarial review. This can go back and forth for several iterations before converging on an acceptably "correct" version of the code...

Occasionally, Claude may not be able to fix the issue – and it may turn out that Codex can, because it has a different approach.

Either way, at least two LLMs is the only way to get (mostly) reliable code.

eivindmeyer 12 hours ago|||
You can automate this entire orchestration with zeroshot.sh, mentioning it since it is relevant here.
NishanStepak 13 hours ago|||
Claude, ChatGPT, Google Gemini, then Replit...
unfazed213 9 hours ago|||
[flagged]
vijayharre10 15 hours ago||
[flagged]
hemat_energi 3 hours ago||
[flagged]
unfazed213 9 hours ago|
[flagged]
More comments...