We're somehow still stuck with CAPTCHAs (and other challenges), a 25 years old concept that wastes millions of human hours and billions in infra costs [0].
Cool... but I guess now we need a benchmark for such solutions. I don't know the author, I roughly know the problem (as I self host and most of my traffic now comes from AI scrapper bots, not the usual indexing bots or, mind you, humans) but when they are numerous solutions to a multi-dimensional problem I need a common way to compare them.
Yet another solution is always welcomed but without being able to efficiently compare it doesn't help me to pick the right one for me.
Allowed
https://web.archive.org/web/20250419222331if_/https://anubis...
https://web.archive.org/web/20250419222331if_/https://anubis...
https://web.archive.org/web/20250420152651if_/https://anubis...
https://web.archive.org/web/20250420152651if_/https://anubis...
Blocked
https://web.archive.org/web/20250424235436if_/https://anubis...
https://web.archive.org/web/20250510230703if_/https://anubis...
https://web.archive.org/web/20250511110518if_/https://anubis...
https://web.archive.org/web/20250630101240if_/https://anubis...
https://web.archive.org/web/20250808051637if_/https://anubis...
https://web.archive.org/web/20250909160601if_/https://anubis...
Allowed
https://web.archive.org/web/20250921062513if_/https://anubis...
OpenAI Atlas defeats all of this by being a user's web browser. They got between you and the user you're trying to serve content, and they slurp up everything the user browses to return it back for training.
The firewall is now moot.
The bigger AI company, Google, has already been doing this for decades. They were the middlemen between your reader and you, and that position is unassailable. Without them, you don't have readers.
At this point, the only people you're keeping out with LLM firewalls are the smaller players, which further entrenches the leaders.
OpenAI and Google want you to block everybody else.
Do you have any proof, or even circumstantial evidence to point to this being the case?
If chrome actually scraped every site ever you visited and sent it off to Google, it’d be trivially simple to find some indication of that in network traffic, or heck - even chromium code.
Who would dare block Google Search from indexing their site?
The relationship is adversarial, but necessary.
People who don't want to be indexed. Or found at all.
But for anyone whose main concern is their server staying up, Atlas isn't a problem. It's not doing a million extra loads.
Would you trust OpenAI if they told you it doesn't?
If you would, would you also trust Meta to tell you if its multibillion dollar investment was trained on terabytes of pirated media the company downloaded over BitTorrent?
Personally I would just believe what they say for the time being; there would be backlash in doing something else, possibly legal one.
That isn't a conspiracy theory, it's fundamentally how interfacing with 3rd party hosted LLMs works.
Unless the user asked something that just needs visiting many pages, I suppose. For example, Google Gemini was pretty helpful in finding out the typical price ranges and dishes a local shopping centre coffee shops have, as the information was far from being just in a single page..
It's definitely pointless if you completely miss the point of it.
> OpenAI Atlas defeats all of this by being a user's web browser. They got between you and the user you're trying to serve content, and they slurp up everything the user browses to return it back for training.
Cool. Anubis' fundamental purpose is not to prevent all bot access tho, as clearly spelled in its overview:
> This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies.
OpenAI atlas piggybacking on the user's normal browsing is not within the remit of anubis, because it's not going to take a small site down or dramatically increase hosting costs.
> At this point, the only people you're keeping out with LLM firewalls are the smaller players
Oh no, who will think of the small assholes?
The conclusion I have come to is more general: I just personally don't like nerd-culture. Having an anime girl (but the same would be the case for an star trek, my litte pony/furry, etc.-themed site) signifies a kind of personality that I don't feel comfortable about, mainly due to the oblivious social awkwardness, but also due to "personal" habits of some people you meet in nerdy spaces. I guess there is something about the fact of not distinguishing between a public presentation and personal interests that this is reminiscent of. For instance: A guy can enjoy model trains, sure, but he is your college at work and always just goes on about model trains (without considering if this interests you or not!), then the fact that this subsumes his personality becomes a bore or even just plain unpleasant. This is not to generalize that this is the case for everyone in these spaces, I am friends with nerdy-people on an individual basis, but I am painfully aware that I don't fit in perfectly like the last piece of a jigsaw puzzle -- and increasingly have less of a desire to do so.
So for me at least this is not offence, but in addition to the above also some kind of reminder that there is a fundamental rift in how decency and intersocial relations are imagined between the people who share my interests and me, which does bother me. Having that cat-girl appear every time I open some site reminds me of this fact.
Does any of this make sense? The way you and others phrase objections to the objections makes it seem like anyone who dislikes this is some obsessive or bigoted weirdo, which I hope I don't make the impression of. (Hit me up, even privately off-HN if anyone wants to chat about this, especially if you disagree with me, this is a topic that I find interesting and want to better understand!)
It is really bizarre how everybody tries to make it about politics. While I may or may not disagree with a developer's politics, it's their conduct that I care about, and I associate those who express appreciation for anime at every possible opportunity with especially poor conduct and have yet to have encounter an exception to the rule.
The amount of flags I'm seeing for posts simply expressing disagreement on the matter is quite worrying.
This is some real four-dimensional chess. "You're the childish one for not wanting Japanese cartoons on software projects!"
This whole comment chain solidifies my opinion that disgust is one of the driving human emotions. People feel initial, momentary disgust and only then explain it using the most solid justification that comes to mind, but the core disgust is unshakable and precedes all explanations. No one here has managed to procure any argument for why seeing a basic sketch in a certain style is objectively bad or harmful to someone, only that it's "weird" in some vague way. Basically, it goes against the primal instinct of how the person thinks the world "ought to work", therefore it's bad, end of story.
To me it seems obvious. The anime art style is in, especially in Western countries, especially^2 among younger people, and especially^3 among techy people. Ergo, you may see a mascot in that style once in a while in hobbyist projects. Doesn't seem like anything particularly objectionable to me.
And yet if you bring up that "Gimp" is an unserious name, or anything about RMS that's far more problematic than a cute cartoon, that same subset will defend it to the death.
I'm not talking about anything this narrow - disgust manifests itself in every facet of human life, this comment thread is just one minor example. Lots of laws were initially justified by some form of disgust. There's practically an infinite number of examples of people feeling immediately "icky" about something (absolutely anything, applicable at any point in history) and only then trying to create a justification for these feelings, basically working backwards to make their instincts seem more reasonable and palatable. You can easily spot it because when one justification is taken down, another one takes its place, and it can go on for eternity - justifications are temporary, the only thing that's permanent is the unshakable feeling of correctness and righteousness about the initial disgust. Notice how OP's argument about standards and professionalism was quickly swapped out for a more dignified version of "well, I just feel like everyone who likes XYZ is a sweaty anti-social manchild", as soon as arguing the original point became more difficult?
> The world is full of serious businesses that use "cute" icons or employ anime-styled elements
I can't think of any outside of Japan.
Go ahead, say it.
You're aggressively trying to imply I'm saying something here and I'm not sure what.
I would be OK with the sentence if it was "You're the childish one for not wanting Japanese cartoons on your software projects!".
As you wrote it, well, that's none of your business.
> an animated cock and balls
You don't see a difference between these things?
Wikipedia suggests that there's an association with queer and trans youth, is that what's meant to make the cock-and-balls comparison work? But it also says it has a history back to 17th century Japan...
I'm not looking for anyone's approval. If I was, I wouldn't be publicly disagreeing with people on an internet forum, would I? Relax with your armchair psychology.
> Personally, I wouldn't take seriously anyone who has a problem with Anubis but doesn't blink when presented with people drawn in the corporate Memphis style.
I don't like either and find them both ugly.
American corporate culture is dehumanizing and dystopian, not a standard for professionalism.
Nothing says "professional" like starting a debate on HN about the weirdness of the mascot of a free software project, likely for political reasons.
You're engaging in bad faith here. Nobody has brought up politics at all. If an almost identical clone of myself (with the same opinions on everything but mascots) developed a software project with an anime mascot I'd still disapprove.
I suppose it all comes down to what your definition of "professional" is.
I would argue that this statement is blatantly false. Currently, most people really do not care about anubis anime cat girl icon which is actually fairly tame and boring picture.
In history, people used all kind of images for professional things, including stuff they found funny or cute.
It is a shitty, and obviously bad solution for preventing scraping traffic. The goal of scraping traffic isn't to overwhelm your site, it's to read it once. If you make it prohibitively expensive to read your site even once, nobody comes to it. If you make it only mildly expensive, nobody scraping cares.
Anubis is specifically DDOS protection, not generally anti-bot, aside from defeating basic bots that don't emulate a full browser. It's been cargo-culted in front of a bunch of websites because of the latter, but it was obviously not going to work for long.
If the authors of the scrapers actually cared about it, we wouldn't have this problem in the first place. But today the more appropriate description is: the goal is to scrape as much data as possible as quickly as possible, preferably before your site falls over. They really don't care and side effects beyond that. Search engines have an incentive to leave your site running. AI companies don't. (Maybe apart from perplexity)
Only well-behaved application-level DDoS protection maybe.
A real network-level attack in the many-gigabits/sec+ will not be stopped by anubis itself.