Posted by Brajeshwar 5 days ago
If you worked in cloud environments - every database had public facing api that can delete it.
For the rest or it - yeah, running autonomous pipelines in production which decide what to run and what not to run seems fine until it isn’t.
But every database deployed in a cloud environment has an api that can delete it. Even if you say you’re running on vms - there exist api that can delete the disk, the vm, the network config, etc.
They're hired to be responsible for some part of the product.
Introducing AI doesn't remove that responsibility.
Folks tend to focus on the code and the tools they're using (maybe I'm cynical from years in the industry). I don't think your boss wants to do your job, even if they could use AI to do it. I think your boss wants to have a headcount, and he wants the headcount to be responsible for the product.
Both he should learned his lesson AND AI should not be trusted.
User: I tried to cut some bread and it cut my finger instead.
AI companies: not my problem!
HN: The AI didn't cut your finger, you did, idiot.
It just goes to show, if you're a jerk, expect to be treated like one (even by an AI model)! Be polite, people.
Why can you delete a network load balancer that is still getting traffic?
Why can you delete a VM that is getting non-trivial network traffic?
Why can you delete a database that has sessions / requests in the last hour?
Why can you drop a table that has queries in the last hour?
I've seen this at work the most with slow rollouts. They said it was for prod only, then it became applied to staging and dev somehow. They said you can force push in emergencies, but approximately 0 people on any given team know how to do this reliably, and it still takes way longer even in --force --now --breakglass --yesimeanit mode. So the end result is longer MTTR. It maybe prevents some kinds of outages, but also you're less likely to manually monitor a rollout when it takes longer.
The basic setup has worked well for me at several companies with many large teams on-call.