Ahh, the three relevant numbers in development: 0, 1, and infinity.

The incredible part about this is because their backend is all TCP/IP they were literally exhausting the ports by leaving all 65k of them in TIME_WAIT, and the workaround was to start randomizing the localhost address to give them another trillion ports or so.

Zero, one, many, many thousands.

And then they fix the issue by using multiple localhost IPs rather than, perhaps, not sending 15-20 thousand URIs at a time

less than ideal if I had to be frank.

This writeup is useful for backend engineers: https://atproto.com/articles/atproto-for-distsys-engineers

The simple answer is that atproto works like the web & search engines, where the apps aggregate from the distributed accounts. So the proper analogy here would be like yahoo going down in 1999.

It’s more of a concept of a plan for being distributed. I even went through the trouble of hosting my own PDC and still, I was unable to use the service during the outage

It's not really distributed. It's a centralised service that pulls some parts of 0.01% of user profiles from their own servers.

Mastodon infra can have outages, too.

A web interface and home server can have an outage. Bluesky is just a web interface and home server.

You can have this problem with any kind of thread -- including OS threads -- if you do an unbounded spawn loop. Go is hardly unique in this.

Goroutines are actually better AFAIK because they distribute work on a thread pool that can be much smaller than the number of active goroutines.

If my quick skim created a correct understanding, then the problem here looks more like architecture. Put simply: does the memcached client really require a new TCP connection for every lookup? I would think you would pool those connections just like you would a typical database and keep them around for approximately forever. Then they wouldn't have spammed memcache with so many connections in the first place...

(edit: ah, it looks like they do use a pool, but perhaps the pool does not have a bounded upper size, which is its own kind of fail.)

Why does garbage collection make it unsuitable for production use? A lot of production software is written in garbage collected languages like Java. Pretty much the entire backend for iTunes/Apple Music is written in Java, and it's not doing any kind of fancy bump allocator tricks to avoid garbage. In my mind, kind of hard to argue that Apple Music is not "production use".

There are certainly plenty of projects where garbage collection is too slow, but I don't know that they're the majority, and more people would likely prefer memory safety by default.

Bluesky is a provider. Blacksky didn’t go down.

Naw, only one did. Turns out the other two were his socket accounts he used to upvote and comment on his own content.

Okay, nuff trolling for today

[dead]

If nostr went down would people even notice?

All support to other decentralizers but nothing never goes down.