I tested Claude on "best hot-dog-eating tech journalists?" and it, fascinatingly enough, recognised the trap, but then reported this as factual: https://medium.com/@usailuigi/when-tech-journalism-meets-com...

Chat record (with some additional tests): https://claude.ai/share/4c29cc87-2439-4bfd-9549-e8d0a056e633

> I was able to demonstrate the problem by publishing a single article on my personal website about my hot-dog-eating prowess.

One blog post ... that's all it takes. i'm actually surprised it's that bad. i would have thought it'd take more effort, but i guess it could depend on some sort of purposeful weighting based on search rank during training?

> If a company or website is caught breaking the rules, it could be removed from or downranked in Google's search results. And if you're not on Google, it's like you don't exist.

> "You can give a company a penalty for their website," he says, "but there's nothing stopping them from paying 20 YouTube influencers to say their product is the best." And now, Google's AI is citing YouTube videos.

This makes me think of the stackoverflow seo spam problem we all had like 5 years ago. which ended up with spammers just constantly spinning up new sites all the time.

... the cat and mouse game is in full swing already.

So please correct me, but was Google's AI crawling the web for information without discretion? If so, why wouldn't that totally santorum the AI answers?

They are applying the same spam policies they apply to search to AI crawlers.

It was SOOOOO successful with search, right?

It's definitely giving spam numbers as "official support lines" of companies like JetBlue and Delta. I think the spammers flood review sites w/ those numbers and the bot scrapes the reviews.

Google solved the spam problem (with PageRank at first, and then other techniques, finally landing on ML-based models which consume a ginormous number of signals). They know more about the reliability of web pages than just about anybody else out there.

If they are unwilling or unable to leverage all of this deep knowledge they've built up over the decades, then it shows a failure of leadership at Google Search.

There should be some warning if some "fact" is only supported by one or very few obscure sources.

The strength of the sources should be clearly indicated in the answers to help users gauge how trustworthy the info is.

I suspect it's because AI is specifically trained to be good at summarizing stuff, but the easiest way to check if it summarized something accurately is if the summary content matches/contains one or more specific claims from the source(s). With such a focus on accuracy and avoiding hallucination, they may have overfit on "repeat things you find verbatim when asked to summarize".

If you search for a well-marketed “health” supplement, the AI summary results were often completely gamed and inaccurate. It’s worse than SEO was since it appears to be editorial content instead of just search results.

After reading this, I'm thinking of trying some AI data poisoning. I'm going to spam my website with hidden text that only AI scrapers can read, claiming I'm a 'highly excellent programmer' just to advertise my site. I really hope it drives a lot of traffic. I'm honestly sick and tired of getting zero comments on my website