The time that I would have spent typing in the code is now time I use to test the code. I really like to poke at things and ensure that my understanding and reality match.

I agree with the advice to design your system before implementing it. It is pretty easy to iterate on a design with your favorite coding assistant, and this is your chance to be ultra-picky and make sure you get exactly what you want. This was always where the "software engineering" was done, but in practice it's easy to have a vague idea and refine your (internal) list of requirements / tradeoffs as you type in the code. I find it harder to do that when the LLM is generating the code, just because one turn can change so much.

Keeping it under control requires discipline, just like before LLMs were a thing. You set a bunch of engineers loose on your codebase with no vision or supervision, you get spaghetti. You just get a little bit more control now. Agents don't feel shitty for the rest of the day when you tell them to do it some other way, so you are free to review as critically as you want without hurting anyone's feelings. I think it's a good thing.

The traditional answer is the same for engineers going into engineering management because managing a team and managing a set of agents is highly similar, only managing agents is far easier. One only ever has the illusion of control, and so you have to set your mind to containing the worst possible outcomes and looking at the net efficiency and efficacy of the team with the understanding that they will, on average, never be perfect.

I think there's a balance to be had. Leaving everything to agents when it comes to throwaway prototypes, one-off test env. scripts, extra tests I'm fine with. Doing the same with features - absolutely not - especially when it comes to financials. Seems like Meta is pressuring engineers into becoming expensive spec generators because they want to be in the cool kids club. Time will tell if that's a good play but to me it just sounds like a quick and dirty way to obliterate their engineering culture.

1) Need to triple down on infrastructure (safeguards, tests, sandboxes, etc.).

2) Ask agents to do more than just code.

They need to fully demonstrate and prove that their PRs are rock solid. Ask for screenshots, videos, code reviews, detailed architecture diagrams, etc.

It would be an insane amount of work for humans, but AI can do it easily for each PR.

Merging any PR from an agent should be a no-brainer if there's real proof that the feature is built correctly and working.

If there's still doubt, then the agent needs to do a better job of proving that the feature is built correctly, doesn't introduce regressions, etc.

Built a tool that helps me at least get a better gist of complex changes: https://github.com/sshah03/perspica

It's not perfect, but it's helped so far.

It comes down to the blast radius of a failure happening in your system. Some stateless frontend won't require much beyond visual inspection, but if you introduce some client-server architecture or require payment processing, you must consider the cost of a security intrusion when auditing code changes.

Maybe a bit unorthodox but I have it plant 1-3 bugs into the code it writes (depending on the size of the MR I'm working on); they range from simple one-off issues to something more structural.

It then gives me a hint on the area or a typical code flow that would hit the bug - whichever gives me less info. I can obviously ask for more hints as I read along.

Obviously all honour-code based since I could just ask it to fix the bugs itself. I think this is something that can't be fixed systemically - either you're going to bother reading and thinking through the code or not.

Other than that I guess reading the tests - to make sure they fail without the code / fix, to make sure the setup makes sense etc.

I only ship code I don't understand if it comes with a proof of correctness

You hit deploy and hope for the best, if it goes bad You just prompt the model again....

Just kidding, it's de same as with any other Open source plataform You install from their repo. It will be hard at the start but then You Will be very aware where it starts to fail and how to fix it.

I built and use https://smalldocs.org, which is a rich document format (mermaid diagrams, nice formatting, etc.) for agents. So I’ll be like “sdoc me the architectural ideas for this feature” and it helps.