This is an old automation lesson in a new costume. The tool that makes correct work faster also makes unsafe work faster unless the boundaries are real.

This reminds me of a James Micken's quote from "This World of Ours" in response to security people admonishing users for clicking links in email:

    "It’s not clear what else there is to do with computers besides click on things..."

If you have an API with exposed endpoints, it's not clear to the AI bot what else there is to do with the API besides call the endpoints.

This particular case was extremely unsympathetic, but a critical part of the failure was people being too credulous about the claims of AI providers. They are still refusing to take adequate responsibility for AI "making mistakes" - that is, going completely off the rails.

Now: the CEO gets paid the big bucks and has the least direct accountability, very much because it's their job to take responsibility for people more powerful than them, and likewise the CTO with major commercial software contracts like a Claude subscription. That's why this guy was so hard to take seriously: okay fine, you got burned by Anthropic, stop being a baby about it. Take responsibility for not listening to the critics.

But - to be a little more neutral about my personal distaste - I do think vibe coders are making a very similar mistake to C developers throughout the 90s, where problems with the tooling were not merely dismissed, but actively valorized.

Real Devs use buffers freely and don't make overflow errors.

Real Devs use hands-free agentic development and don't delete production databases.

Sounds like the author didn't even read the postmortem. At no point did the business owner try to implicate that they bore no responsibility. Rather, they pointed out that deleting a database volume *also deleted every single backup.*

That's a pretty nefarious edge to cut yourself on. AI has nothing to do with Railway's awful API surface here.

From 'the hacker did it' we have moved to 'the AI did it'. The problem set is roughly the same.

> Automation helps eliminate the silly mistakes that come with manual, repetitive work. We could have easily gone around asking "Why didn't SVN prevent us from deleting trunk?"

Do both? Question bad design and then do whatever you can to work around it.

As an example, that's why flags like this make sense, even if it's a pretty specific use case and there won't be many people using that option at all, preventing stupid default behavior is a good idea: https://superuser.com/a/742735

"move fast and break things" only sounds good when it's not breaking things in a serious and unfixable way. Maybe we shouldn't take hype mantras as instructive means to an end.

The problem with asking an LLM for "its reasoning" after the fact, is that any justification it might give is a post-hoc rationalisation rather than a pre-meditated reason.

Just skip straight to the Twitter post, it's way better than this secondary article.

  We had no idea — and Railway's token-creation flow gave us no warning — that the same token had blanket authority across the entire Railway GraphQL API, including destructive operations like volumeDelete" [...] Railway's volume backups are stored in the same volume.

Idk how this is anyone else's problem but Railway. Same could happen with a human user.