Posted by ndhandala 10/29/2025
Good datacenters have redundant and physically separated power and communication from different providers.
Also, in case something catastrophic happens at one datacenter, the author mentions they are peered to another datacenter in a different country, as another layer of redundancy. Cloudflare handles their ingress, so such a catastrophic event wouldn't likely to be noticed by their customers.
However to steelman AWS use. Many businesses are STILL running mainframes. Many run terrible setups like Access as a production database. In 2025 there are large companies with no CICD platforms or IAC, and some companies where even VC is still a new concept or a dark art. So not every company is in the position to actually hire competent system administrators and system engineers to set up some bare metal machines and configure Ceph, much less Hadoop or Kubernetes. So AWS lets these companies just buy this capabilities while forcing the software stack to modernize.
That company had its own data center, tape archives, etc. It had been running largely the same way continuously since the 90s. When I left for a better job, the company had split into two camps. The old curmudgeonly on-prem activists and the over-optimistic cloud native AWS/GCP certified evangelist with no real experience in the cloud (because they worked at a company with no cloud presence). I'm humble enough to admit that I was part of the second camp and I didn't know shit, I was cargo culting.
This migration is still not complete as far as I'm aware. Hopefully the teams that resisted this long and never left for the cloud get to settle in for another decade of on-prem superiority lol.
This is the most infuriating pain point when using AWS.
We have premium support at work and the engineers basically read the documentation back to us, ignoring our complaints about cross-az bandwidth and the utter and complete lack of az-awareness in their services.
Example: we had some redises that we wanted to migrate to ElastiCache... All the aws engineers did was reading back the marketing pages and pushing for the serverless offering (where bandwidth was the main cost driver). The reader/writer endpoints are totally az-unaware.
Cross-AZ bandwidth for ElastiCache was costing us MORE than ElastiCache itself.
In the end we had to engineer AROUND ElastiCache in order to get it working, and working with predictable pricing.
"Invent and simplify" my ass...
I’m sorry but I don’t believe this for one second.
And unfortunately that makes me distrust the entirety of the article.
Specifically, they promise to provide a "small, rapidly growable slice of a very big thing". I.e.: You can create an empty S3 account for $0.00, fill it with a few megabytes of initial data for $0.00001, and then if you suddenly need petabytes then it scales smoothly and beautifully up past any reasonable scale. You get billed for that at an exorbitant rate, but the point is that you can do it without having to rearchitect anything.
"This Works Great(tm)" for three specific categories of customers:
- Tiny organisations that expect to grow big suddenly: Startups, and maybe the few orgs that have rare or annual events and nothing in between. Think electronic voting systems and the like.
- Small international companies that need global presence on the cheap. SaaS vendors, IoT, and a few niche organisations are pretty much the only ones in this category.
- Enormous organisations that need large scale but can't be bothered (or can't afford) managing that. There was an AWS talk about a customer that needed about 1 PB of storage which these days is "just" 500 x 20 TB disks, but they needed burst IOPS far in excess of that, on the order of 100,000 disks. The "thin slice" model of the cloud works great for this, because S3 has millions of disks behind it, and each customers' data is spread out over those disks.
Everybody else in the "medium" category is sold a bunch of bullshit.
Cloud VMs are between 5x and 10x as expensive as the on-prem equivalents, all costs factored in. I've seen the numbers from CIOs and CTOs, they all got told "the cloud is cheaper", and their costs skyrocketed as soon as they went to the cloud. You still need engineers. You still need "deployments". You still need sysops. You still need to update your VMs and their software somehow. Nothing changes really, except suddenly VMware starts looking cheap in comparison.
The "You can scale, the cloud is flexible, you can..." marketing is a load of bollocks.
First, Pay-as-you-Go pricing is on average 7x on-prem VM pricing. The only way to bring this down to merely 3x the on-prem cost is to LOCK IN the compute using "reservations" of some sort, typically for 3-year periods. This is NOT FLEXIBLE BY DEFINITION!
The whole marketing of the cloud revolves around the flexibility, but all of their pricing and cost optimisation revolves around getting customers to lock in spending for years and years.
Similarly, Spot-priced anything is so unreliable that it is completely unusable for almost all "enterprise" customers, even for non-production use.
Seriously, can you imagine telling a developer that costs $200/hour that they can't do their work because their DEV instance is gone for a day because some other tenant needed it more!?
This underlying issue with VM pricing means that any service that is built on top of VMs inherits the same pricing model with lock-in contracts, totally negating the scalability benefits and global presence of the public cloud. If you want S3 in every region with 10 MB each... that's cheap. If you want just one VM in every region... no longer cheap. If you want any VM-based service in every region... also not cheap. Oh... you wanted DEV/TST/UAT/GREEN/BLUE? With high availability? Get the CFO on the line, he'll need to approve your budget!
"Just engineer your software to be cloud native!" is what you inevitably hear from apologists. Sure, sure... I'll get right on that. I mean sure, the public cloud vendors failed to do so for like two thirds of their own first-party products, but I'm sure I'll have better luck! Let's see.. my "medium sized org" has... checks notes... about 1,000 unique pieces of software deployed on VMs, of which 800 are CotS vendor products, 600 of which require Windows Server and have a GUI configurator. This will go... smooth.
For 90% of the potential customers out there, the big businesses, the enterprises, the universities, governments, and the like... it's just a more expensive data centre that someone else runs for them.
The only significant advantages to public cloud vs on-prem I've seen are:
- Faster networking, with a well-engineered 100 Gbps or 200 Gbps core in most clouds. This includes Internet uplinks of similar spec. These are rare in private hosting.
- Three-way zone redundancy instead of the typical two-way.
- Zone redundant services that are just a "checkbox".
- Separation of duties where the layer 2 network and hypervisor are managed by a vendor instead of internal staff. (This can be difficult to arrange internally for medium sized orgs needing high security, there aren't enough IT admin staff for true separation.)
... that's about it.