Posted by MattIPv4 8 hours ago
Another site was constantly getting DDoS by Russians who were made we took down their scams on forums, that had to go through verisign back then, not sure who they're using now. They may have enough aggregate pipe it doesn't matter at this point
Currently consulting somwhere with 30 services per engineer. I cannot convince them this is hell. Maybe that makes it my personal hell.
In that every night you're playing murder mystery, and its never fun.
how is such service spam different from unix "small functions that do one thing only" culture?
why in unix case it is usually/historically seen as nice, while in web case it makes stuff worse?
You will basically need to employ solutions for problems only caused by your microservices arch. E.g. take reading the logs for a single request. In a monolith, just read the logs. For the many-service approach, you need to work out how you're going to correlate that request across them all.
Even the aforementioned network failures require a lot of design, and there's no standardization. Does the calling service retry? Does the callee have a durable queue and pick back up? What happens if a call/message gets 'too old'?
Also, from the other end, command line utils are typically made by entirely different people with entirely different philosophies/paradigms, so the encapsulation makes sense. That's not true when you're the one writing all the services, especially not at small-to-mid-size companies.
Plus, you already can do the single-concern thing in a monolith, just with modules/interfaces/etc.
One strategy to convince is to get someone less technical than you to sit by you while you try and trace everything from one error'd HTTP request from start to finish to diagnose the problem. If they see it takes half a day to check every call to every internal endpoint to 100% satisfy a particular request sometimes that can help.
Also sometimes they just think "this is a bunch of nerd stuff, why are you involving me?!" So it's not foolproof.
The real solution is probably to leave, but the market sucks at the moment. At least AI makes the 10-repos-per-tiny-feature thing easier.
99.99
99.90
99.00
90.00
https://gitlab.com/gabriel.chamon/ci-components/-/tree/main/...
That helps with Git not so much issues etc.
At any rate, it seems like GitHub is back up now, so we'll see how long that lasts.
https://www.forbes.com/sites/bernardmarr/2025/07/08/microsof...
To explain this one-word comment for those unfamiliar, see previously:
GitHub will prioritize migrating to Azure over feature development (5 months ago) https://news.ycombinator.com/item?id=45517173
In particular:
> GitHub has recently seen more outages, in part because its central data center in Virginia is indeed resource-constrained and running into scaling issues. AI agents are part of the problem here. But it’s our understanding that some GitHub employees are concerned about this migration because GitHub’s MySQL clusters, which form the backbone of the service and run on bare metal servers, won’t easily make the move to Azure and lead to even more outages going forward.
I'm sure the people with the purse strings didn't care, though, and just wanted to funnel the GH userbase into Azure until the wheels fell off, then write off the BU. Bought for $7.5B, it used to make $250M, but now makes $2B, so they could offload it make a profit. I wonder who'll buy it. Prob Google, Amazon, IBM, Oracle, or a hedge fund. They could choose not to sell it, but it'll end up a writeoff if the userbase jumps ship.
What? No, no it's not. The entire discipline of Infrastructure and Systems engineering are dedicated to doing these sorts of things. There are well-worn paths to making stable changes. I've done a dozen massive infrastructure migrations, some at companies bigger than Github, and I've never once come close to this sort of instability.
This is a botched infrastructure migration, onto a frankly inferior platform, not something that just happens to everyone.
Artificial intelligence, Azure integration, many other things.