"Here's what's going on."

Thank you

https://news.ycombinator.com/item?id=49571448

I had a feeling it was due to "AI" companies and developers using "agents"

Not surprised

Is it established that the scraping scourge of late is primarily driven by AI companies? Anyone aware of any relevant studies?

Beginning to think that the difficulty to browse most websites nowadays due to throttling, is yet another negative externality of AI development that society is forced to bear.

Anyone else feel like making a new internet and starting over

I wonder why doesn’t the Internet Archive require logging-in prior to accessing the Wayback Machine. It would probably help them distinguish humans from bots, at a very little cost to humans.

The anti virus industry needs to crack down on crawler and proxy malware, plus ISPs FINALLY need to replace CGNATs with iov6 to stop crawlers banning everyone behind a NAT.

I'd be happy to pay a 5$/month donation to get a higher rate limit/more lenient filter put on me

Shouldn't the solution be to gate bulk access for automated services for a price? Not just the wayback machine, any personal or corporate website?

My blogs are getting slammed and there are issues with cloudflare or captchas.

Wayback can’t be accessed through vpn, atleast on proton. Heck, most sites simply block you for using vpn.

Thank you for the Wayback Machine. It is immensely powerful for good.

Why can't they capture OS, Browser, and IP address at the time of error?

All that information is available at the point of failure, the user should not need to email it in.