Top
Best
New

Posted by aarondf 3 hours ago

Railway Blocked by Google Cloud(status.railway.com)
291 points | 126 comments
valgaze 36 minutes ago|
May 2024 UniSuper incident: https://cloud.google.com/blog/products/infrastructure/detail...

https://www.unisuper.com.au/about-us/media-centre/2024/a-joi...

A joint statement from UniSuper CEO Peter Chun and Google Cloud CEO Thomas Kurian

8 May 2024

UniSuper and Google Cloud understand the disruption to services experienced by members has been extremely frustrating and disappointing. We extend our sincere apologies to all members.

While supporting UniSuper to bring its systems back online, Google Cloud has been conducting a root cause analysis.

Thomas Kurian has confirmed that the disruption arose from an unprecedented sequence of events, where an inadvertent misconfiguration during provisioning of UniSuper’s Private Cloud services ultimately resulted in the deletion of UniSuper’s Private Cloud subscription.

This is described as an isolated, “one-of-a-kind occurrence” that has never before occurred with any Google Cloud client globally. This should not have happened. Google Cloud has identified the sequence of events and taken measures to ensure it does not happen again.

Why did the outage last so long?

UniSuper had duplication across two geographies as protection against outages and data loss. However, the deletion of the Private Cloud subscription triggered deletion across both geographies.

Restoring the Private Cloud required significant coordination and effort between UniSuper and Google Cloud, including recovery of hundreds of virtual machines, databases, and applications.

kvakvs 18 minutes ago|
The instant cascading worldwide deletion upon closing or deleting a subscription sounds like a recipe for disaster. Why not mark it for deletion and delete say... a day or a week later?
shye 11 minutes ago|||
From personal experience, as a customer who once did something stupid: Google Cloud does soft deletes. But you need to reach out to support fast enough. And really, if you deleted something important and discovered it only the next day, and not within minutes, you're having a bigger issue that a soft delete won't solve.
modernpacifist 12 minutes ago||||
Either mark-for-delete has the same impact as deleting in terms of shooting all the Cloud resources associated with the subscription, at which point the outage still happens but maybe the recovery is smoother or you've just delayed the inevitable by a week because no one will look at it unless there is actual impact.
manapause 7 minutes ago|||
It’s a good question. That said unless there are compliance or fallback concerns i would prefer a service that burns my data on departure.
dangoodmanUT 2 hours ago||
It has been 0 days since GCP has taken down a startup (again).

You see this at least once a year. Never heard of this from AWS or Azure.

In all seriousness, this is why we don't use them. They have the most ergonomic cloud of the big three, then absolutely murder it by having this kind of reputation.

Spooky23 26 minutes ago||
https://en.wikipedia.org/wiki/Timeline_of_Amazon_Web_Service...

Azure nerfed the front door of all Azure and O365 services last year.

All it these companies are great at what they and occasionally fuck up.

somewhatgoated 1 hour ago|||
On the other hand i can’t remember when there was a serious outage on GCP, unlike AWS/Azure who seem to go down catastrophically a couple of times per year.
adamtaylor_13 47 minutes ago|||
Perhaps you don't notice GCP outages because so few companies rely on them?
fragmede 26 minutes ago||
GCP has a lot of customers. But you wouldn't know the companies that do, unless you worked there and wanted to leak it, or it publicly comes out. Eg it's been publicly acknowledged that Apple uses GCP for iCloud, https://www.cnbc.com/amp/2018/02/26/apple-confirms-it-uses-g... , and Home Depot is another that's used as a case study, https://cloud.google.com/customers/the-home-depot but most customers don't want to make a big deal about being on GCP as it's none of our business who's hosting them.
shye 9 minutes ago|||
Apple also uses AWS, and I won't be surprised if they also use Azure. Big companies are multicloud, and not because it's a good idea (it rarely is), but because they inherited multiple environments on different CSPs, and maintaining those where they are is often cheaper than migrating them to a different CSP.
Imustaskforhelp 5 minutes ago|||
upvoted & favourited because you taught me a really interesting fact which I feel makes up for an amazing discussion (regarding icloud using GCP).

also, I can't help but imagine if instead of render, it was Apple's account which could've been auto-banned (Render is almost a billion dollar company or series-B, I am not sure)

I haven't read the articles and I admit that but can you please elaborate to me on why Apple uses GCP themselves for idrive, I would love to know the technical decisions behind it on a genuinely curious level.

From my (let's face it) limited understanding of GCP, it isn't particularly good or price performant and one of the wonders is that Google sells it directly with Google photos too and an competitive lineup at android.

So in some sense if Apple is using gcp's for icloud then aren't they just reselling google storage themselves and google can always beat them in pricing while also wanting to chew away at the percentage of iphones themselves too?

I mean, I can still try to understand the google search pays apple 10 billion dollars (right?) deal but I don't quite understand why apple would pick GCP when the hosting market is one of the more competitive ones with lots of companies.

I would love to get some explainations or theories as to why exactly is that the case

(Also given its HN, if anyone from apple is reading or knows the answer, I would love that too!)

abofh 1 hour ago||||
I've been in AWS for almost twenty years at this point. It's been a long time since I've seen a global outage of the data plane on anything. The control plane, especially the US-east-1 services? Yes - but if you're off of east-1, your outages are measured in missile strikes, not botched deployments.
andreareina 33 minutes ago||
Didn't the latest outage affect people not on us-east-1 because internal aws services depend on us-east-1?
erikerikson 22 minutes ago||
The impacts are usually partial. For example, scaling is impacted but everything already deployed contributes to work up to capacity. Or, you can't change configuration but the previous configuration works as configured. Often surprisingly not so impactful even if there can be limited work stoppage.
hasyimibhar 8 minutes ago||
The problem with the us-east-1 outage is that a lot of big companies are there, so even if you try your best not to depend on us-east-1, your third party providers are most likely there. In my previous company, we were completely down during us-east-1 outage because of other dependencies that are beyond our control.
plandis 1 hour ago||||
GCP has had outages. From a quick search it looks like they had a global outage less than a year ago:

https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...

JoRyGu 1 hour ago||||
AWS goes down catastrophically but are back up in minutes/hours most of the time (as long as they aren't down because Iran blew up their data center). That's obviously REALLY bad for certain industries, but I suspect for the vast majority of their customers it's not a big deal. We've been able to isolate the damage almost every time just by having AZ failover in place and avoiding us-east-1 where we can.
pixl97 1 hour ago||||
GCP never goes down because they banned all their customers.
corpoposter 1 hour ago||||
IIRC the Paris datacenter flood took down a whole “region” and some data was permanently unrecoverable.
nemothekid 40 minutes ago||||
>On the other hand i can’t remember when there was a serious outage on GCP

They had a really bad global outage a year ago. At least with AWS outages are contained to a single region.

blobbers 1 hour ago||||
Unfortunately, if everyone goes down people are understanding. If just _you_ go down, then its oddly less forgiveable.
manyatoms 45 minutes ago||||
How is blackhole-ing a customer not considered an outage?
devmor 1 hour ago||||
There was a pretty bad one last summer - their IAM system got a bad update and it broke almost all GCP services for an hour or so, since every authenticated API call reaches out to IAM.

It had lasting effects for us for a little over 3 hours.

danesparza 55 minutes ago||||
You can read the parent post, right?
Izikiel43 50 minutes ago|||
I still remember the one where they nuked all the storage of I think an Australian insurance company I think, luckily the it department had done a multi cloud setup for backups
overfeed 1 hour ago|||
> Never heard of this from AWS or Azure.

AWS does it more efficiently; it takes down many startups at a time when us-east-1 goes down.

stingraycharles 1 hour ago|||
That’s an entirely different type of problem, and avoidable by just using us-east-2 (I still don’t understand why people default to us-east-1 unless they require some highly specific services).
aloha2436 52 minutes ago|||
Is it that easily avoidable? A lot of AWS's control plane seems to have dependencies on us-east-1, or at least that's what it's looked like as a non-us-east-1 user during recent outages.
MattGaiser 53 minutes ago|||
Sympathy. Railway is going to have numerous people blaming them for this outage. When us-east-1 fails, it is headline news, so you are not to blame.
yandie 21 minutes ago||||
During my 5 years of my startup, we had only 1 outage due to AWS because we picked us-west-2 as the primary reason. If anyone starting a company and picks us-east-1 as the primary reason, they should be fired. There's absolutely no reason to be in that region.
tempest_ 15 minutes ago||
Why do people want to be in that region? Is it the default or something?

I know some workloads help to be colocated but all these places are connected by fiber and every cloud has a worldwide CDN it seems.

xavdid 15 minutes ago||||
If my cloud provider brings my startup down, it's my problem. If they bring all the startups down, that's their problem.
mgfist 44 minutes ago|||
And we all celebrate it since we can't do any work
rozap 1 hour ago|||
Yep, we also don't touch them for this same reason.
abrookewood 2 hours ago|||
Yep, agree 100%. Such a stupid move on their behalf.
jameson 2 hours ago|||
What was the reason GCP took down a startup previously?
__s 1 hour ago|||
hn.algolia.com gcp blocked

https://news.ycombinator.com/item?id=46731498 https://news.ycombinator.com/item?id=33360416

Then I recall https://news.ycombinator.com/item?id=45798827

https://news.ycombinator.com/item?id=33737577

busterarm 56 minutes ago|||
Hetzner and OVH also do this all the time.

It's AWS and Azure that are the outliers and tend not to care too much what their customers do with their infrastructure. AWS is perfectly fine with allowing me to run copies of 15 year old vulnerable AMIs copied from AMIs they've long since deprecated and removed. Even for removed features like NAT AMIs.

tjpnz 2 hours ago||
AWS normally contacts you first.
kevin_nisbet 2 hours ago|||
Do they?

The only anecdotal thing I've seen is we hired a vendor to do a pentest a few years ago, and they setup some stuff in an AWS account and that account got totally yeeted out of existence by AWS if memory serves.

dannyw 1 hour ago|||
You should not be conducting unauthorized penetration tests against third party infrastructure providers without permission. They have processes and systems and usually just wants a heads up of what you plan to test and t the duration / timestamps.

Cuz otherwise you look like a threat actor.

That’s assuming your vendor was pentesting AWS systems. If you meant you hired a vendor to pentest your own systems on AWS, that’s of course a totally different matter.

kevin_nisbet 1 hour ago||
>That’s assuming your vendor was pentesting AWS systems. If you meant you hired a vendor to pentest your own systems on AWS, that’s of course a totally different matter.

Sorry for being unclear, the vendor was attacking our organization only, and any other company was expressly forbidden in the contract. As I recall it was a fake SSO sign-in page to collect credentials that they would try and social engineer our employees with.

Shank 12 minutes ago||
At a minimum you should contact AWS before you launch a phishing page as a test that targets AWS customers.
alchemism 2 hours ago||||
I’m fairly certain you are supposed to contact any vendor before attempting to penetrate hosts with authorization, not the other way around.
coredog64 1 hour ago||
Having done this for both Azure and AWS, there's a specific ticket that needs to be filed with each provider that documents the scope of your pen test, where you're coming from, and a time frame over which you're doing it (which ISTR was "not more than 24 hours")
mixdup 1 hour ago||||
Responding to an unknown security tester like that is a selling point, not a cautionary tale
kevin_nisbet 1 hour ago||
Yup, I thought it was great. Although one concern I always had in the back of my mind was where is the line drawn. Such as if an adversary gains access to one of my orgs accounts and does something similar, do we get 100% taken out.
cherioo 2 hours ago|||
They better do. What is google doing?
Gigachad 2 hours ago||
It's all AI powered
binarycleric 2 hours ago||
How the heck do these things happen, especially with companies with huge monthly spend? At my last job we had some suspicious workloads running on AWS and our TAM reached out to us before taking any action. Who wants to bet this was some AI automation gone wrong and because GCP seems to be allergic to actually contacting a human to get a response, this just sits in some support queue that outsourced workers look at after a few hours just to give a canned response?
garciasn 2 hours ago||
Nothing surprises me with anything related to support on GCP. While we absolutely do not need them, I have been through no less than 12 different Account Executives over the last 6y and they're all ENTIRELY and COMPLETELY useless.

They all introduce themselves, beg me to setup a meeting w/them and some sort of engineering resource(s), and they come to a meeting with a canned slide deck that is so absurdly unrelated to us that I just laugh, and then the next time I hear from them it's because we have a new AE.

This is my most recent reply (right after Next '26):

> I really appreciate you reaching out; however, we have met with, I dunno at this point, more than a dozen GCP Account reps, execs, technical teams, etc over the years and there's little to no value for us or you, now or in the future. Please do feel free to invest your time on your other clients. We're good; truly.

I love GCP and its services; we have been very pleased with it over the years, but the human side of it? Fucking sucks and I just don't see why they even bother.

shye 5 minutes ago|||
That's exactly why I'm less pleased with GCP: to trust a CSP (or any service), I need to be assured that when (not if) things go wrong, I could escalate to a team that would have my back.
OptionOfT 1 hour ago||||
It's because they're measured on something, unsure which metric, but it's definitely not how helpful they are to you.
YuriNiyazov 1 hour ago||
Don't know about GCP, but our AE on AWS was also continuously rotating, and as best I can tell, their job was to figure out what we are planning to build, and to ensure that we should always use <INSERT AWS SERVICE DU JOUR> for that, rather than a competitor product or build it ourselves.
Rodeoclash 1 hour ago||
Exactly the same experience for us as well. I just don't bother with them.
garciasn 1 hour ago||
Before I just cut them off entirely, I used to tell them my primary concern was cost savings and that I wanted them to recommend ways I could cut 25% off my bill every month and watch the glorified salespeople fumble over trying to avoid that conversation.

It’s ok though, Claude helped us cut >45% of our monthly costs. I’m surprised they haven’t been beating down my door after we made that level-shift. Probably in AE transition. ¯\_(ツ)_/¯

dylanpyle 59 minutes ago||||
For what it's worth - I'm not sure what the criteria is (I assume we're "medium sized / not a big upsell opportunity"?) - our GCP rep quickly pushed us to switching to using a GCP reseller. They took over our billing so that we can pay via ACH, and provide both free first-line support/escalation and paid engagements for bigger projects; they don't charge a premium on top, apparently Google pays them for supporting us. Hasn't made much of a difference in how we operate, but at least we have a direct-ish line for issues when they come up.
idontwantthis 1 hour ago|||
It doesn’t worry you enough that someday you could have a serious problem and they wouldn’t be able to help you?
garciasn 1 hour ago||
On the list of things that worry me the most about our company's stuff, an issue I cannot solve w/o help from a human at GCP is around #900000042.
ndneighbor 1 hour ago|||
huh- I guess there are two HN submissions with meaningful replies...

I said this in the other thread, we got access to our account back, but even with a Account Rep. and a CSM on our account- it still took them a while to figure out what was going on.

I'm sure it could have been worse if we didn't have a rep on our account.

guluarte 1 hour ago||
It's Google. They let you use their services, but the moment you don't fit the norm, they suspend you.
rajeshvar 30 minutes ago||
What does blocked mean? Is there a different post that I am missing? There is shared infrastructure in GCP for networking (ex-googler here) and if only railway is affected, then it is not clear if it is only GCP or if there is something from Railway's perspective that needs to be addressed.
BitWiseVibe 2 hours ago||
As someone who runs some public APIs, the amount of spam from Railway IPs is insane. They have horrible abuse prevention. Hopefully this encourages them to improve their operations.
nikcub 45 minutes ago|
This is the conflict at the center of running a hosting company - make it easy to signup and you get a lot of new users but also a lot of abuse.

Implement anti-abuse measures and you will hit some loud false positives (this may be the case with GCP here).

I don't envy anybody running a hosting co - the internet is a really ugly place under the surface.

edit: to add - AWS are really good here. Must be the ~30 years of retail fraud and abuse experience.

chatmasta 1 hour ago||
I thought Railway was building their own data centers? [0]

> The fact of the matter is, you simply cannot build a cloud on someone else’s cloud.

Indeed…

[0] https://blog.railway.com/p/launch-week-02-welcome

bearjaws 2 hours ago||
I will never leverage GCP in an enterprise setting, it's honestly amazing how hard they fumble the bag. Will be interesting to see when GCP support started working with them, from the updates there was an hour and change from when they identified the issue and GCP support was confirmed.

In the cloud space it seems like AWS does nothing and wins.

UrbanNorminal 2 hours ago||
Is google allergic to humans or something? Cannot they just send an email or call the company before taking a wrecking ball to the entire company's infra? Are they stupid?
BarryMilo 1 hour ago|
Surely this is automated. They wouldn't waste precious dollars on employing humans just to keep other humans happy.
koolhead17 7 minutes ago||
Let's blame some rouge AI agent at GCP causing this.
brokenodo 1 hour ago||
Well, as a 2 week tenured and very happy Railway customer until now, I am now a Render customer. Somehow DNS cut over within 1 min(!) and live after about 30 minutes of work. Not bad!
DrewADesign 1 hour ago|
In my experience, DNS changes are a lot faster than they used to be. There’s some website that has a map that tries to resolve your domain with a bunch of name servers around the world that was pretty neat to look at last time I migrated something.
nbarbettini 21 minutes ago||
I became so conditioned to waiting hours(!) for DNS propagation that I'm always pleasantly surprised when it takes <5 min these days.
eezing 6 minutes ago|
“Deletion of private cloud subscription…”

Who deleted it?

More comments...