I guess that whoever maintains that infra simply hadn't thought of it or was not aware. It's not something you get for free in a monitoring system with some agent like disk usage for example. You need to know and remember you have a hard limit on IDs and be aware at which ID you are.

Ao3 doesn't have a dude getting slack alerts by a dozen monitoring agents. It's one of the last holdouts of the old, more personal internet. Hell, it's even certain that they forgot or even didn't know that the type was an unsigned int.

And that's perfect. Blame the wall too, because it was running just fine. It's a site to write (mostly porn), with better uptime and more daily users than most of the companies posted on HN daily.

    4,247,583: Teen And Up Audiences
    4,173,082: General Audiences
    2,816,083: Explicit
    2,271,446: Mature
    1,676,061: Not Rated

> This is like seeing a brick wall 40 miles down a straight road and yet still managing to drive into it, and then blaming the wall.

Not really, no. For example, if you drive into the wall, you may die.

Another experience that feels like death is working in a company that implements on-call rotations.

It would be too easy to draw out a parallel between how you approach a free fanfiction website (the website should mystically owe you five 9's uptime) and the mentality that metastased in the industry.

Instead, I'm gonna take this opportunity to point out that the AO3 downtime affected you, as a non-user, enough to vitrify the admin, where hardcore users laughed it off (because they're not entitled toddlers).

There's no SLA for Harry Styles porn. Run the migration, lock the table for two days and redo the same in 13 years when you get to 4 billion bookmarks.

The website appears to date from 2008. This was a _very_ common latent bug at that point, particularly because Rails would basically force you to implement it. I assume this got fixed at some point, but for a long time all ActiveRecord models had an autoincrementing ID, which had to be a signed 32 bit int. There were scary monkey-patching workarounds if you wanted something more sensible.

EDIT: And, yes, it is apparently Rails! https://fanlore.org/wiki/Archive_of_Our_Own#Timeline

Nah I made the same mistake back in 2009 for a system that was storing behavior events during malware analysis.

You don’t often expect to have two billion of something until you do.

Salesforce is a rather popular platform.

Its defaults are also either a 18-character ID, or a 32bit integer. So, unless you take the effort to actually fight Apex, you're gonna hit this problem sooner or later.

One of the first things I internalized about databases was "just always use BIGSERIAL for primary keys". There are very few good reasons not to.

or use UUID/GUIDS, many databases (eg PostgreSQL) and frameworks (eg Django) support them.

let's use 128bit integer and handle them like floats in php!

and maybe put a 32bit timestamp along and pretend it can somehow store more than a 32bit integer can.