Posted by linhns 19 hours ago
But, bare minimum they should have a useragent that people can opt out of through robots.txt. That's just basic etiquette. Beyond that, I'll wait for their response.
I understand Brave wanting to have the best results for their search, but that doesn't seem to be their primary motivation if the lawsuit is to be believed. It seems to be they're doing the whole "chad third party scraper meme"[1] and then making big bucks selling those results to the other AI companies like Perplexity that have already gotten in trouble doing that same thing in the past.
I wonder in the future if they're be a whole underground/darkweb or p2p/distributed scraping economy to get scraping data from people/countries that have no restriction on where they get their data and have no qualms about getting sued about it, like Anna's Archive. Or some kind of local client people can run on their home computer that works as a peer to peer service, slowly scraping websites from their residential IPs and getting paid a few pennies to submit the results to a huge service that collects it then resells it to the big AI companies.
[1] https://knowyourmeme.com/photos/2615826-twitters-view-rate-l...