Posted by bookofjoe 11 hours ago
A few days ago, I asked it some questions on Russia's industrial base and military hardware manufacturing capability, and it wrote a very convincing response, except the video embedded at the end of the response was an AI generated one. It might have had actual facts, but overall, my trust in Gemini's response to my query went DOWN after I noticed the AI generated video attached as the source.
Countering debasement of shared reality and NOT using AI generated videos as sources should be a HUGE priority for Google.
YouTube channels with AI generated videos have exploded in sheer quantity, and I think majority of the new channels and videos uploaded to YouTube might actually be AI; "Dead internet theory," et al.
Yeah. This has really become a problem.
Not for all videos; music videos are kind of fine. I don't listen to music generated by AI but good music should be good music.
The rest has unfortunately really gotten worse. Google is ruining youtube here. Many videos now contain real videos, and AI generated videos, e. g. animal videos. With some this is obvious; other videos are hard to expose as AI. I changed my own policy - I consider anyone using AI and not declaring this properly, a cheater I don't want to ever again interact with (on youtube). Now I need to find a no-AI videos extension.
One that slipped through, and really pissed me off because it tricked me for a few minutes, was a channel purportedly uploading videos of Richard Feynman explaining things, but the voice and scripts are completely fake. It's disclosed in small print in the description. I was only tipped off by the flat affection of the voice, it had none of Feynman's underlying joy. Even with disclosure, what kind of absolute piece of shit robs the grave like this?
A fun one was from some minor internet drama around a Battlefield 6 player who seemed to be cheating. A grifter channel pushing some "cheater detection" software started putting out intricate AI generated nonsense that went viral. Searching Karl Jobst CATGIRL will explain.
I think the main problems for Google (and others) from this type of issue will be "down the road" problems, not a large and immediately apparent change in user behavior at the onset.
Don’t get me started on the HHKB [1] with Topre membrane keyswitches. It is simply put the best keyboard on the market. Buy this. (No, Fujitsu didn’t pay me to say this)
Whatever you thought you were doing, what you actually did was interrupt a conversation to shove an ad in everyone's face.
Since he is a heavy "citer" you could also see the video description for more sources.
Other post-Soviet countries fare substantially better than Russia (Looking at GDP per capita, Russia is about 2500 dollars behind the economic motor of the EU - Bulgaria.)
1) Post-soviet countries are doing amazingly well (Poland, Baltics, etc) and very fast growing + healthy (low criminality, etc)
2) The "Russia is weak" thing; it is vastly exaggerated because it is 4 years that we hear that "Russia is on the verge of collapse" but they still manage to handle a very high intensity war against the whole West almost alone.
3) China is not a country lagging behind others at all. It is said in some schoolbooks but it is a big lie that is 0% true now.
It's nearly impossible to bankrupt huge country like Russia. Unless there's civil unrest (or west grows balls to throw enough of resources to move the needle), they can continue the war for decades.
What Russia is doing is each week borrowing more and more from the future and screwing up next generations on a huge scale by destroying it's non-military industrial base, isolating economy from the world and killing hundreds of thousands of young man who could've spent decades contributing to the economy/demographics.
Russia somehow fucked up the initial invasion involving driving a load of preprepared amour across an open border, and have been shredded by FPV drones ever since.
This itself seems pretty damning of these AI systems from a narrative point of view, if we take it at face value.
You can't trust AI to generate things that are sufficiently grounded in facts that you can't even use it as a reference point. Why should end users believe the narrative that these things are as capable as they're being told they are, by extension?
The AI videos aren't trying to be accurate. They're put out by propaganda groups as part of a "firehose of falsehood". Not trusting an AI told to lie to you is different than not trusting an AI.
Even without that playing a game of broken telephone is a good way to get bad information though. Hence why even reasonably trustworthy AI is not a good reference.
There's no incentive to produce anything of value outside of "whatever will get me the most clicks/like/views/engagement"
Anti-vax quacks rely on this tactic in particular. The reason they attack vaccines is that they are so profoundly effective and universally recognized that to believe otherwise effectively isolates the follower from the vast majority of healthcare professionals, forcing trust and dependency on the demagogue for all their health needs. Mercola built his supplement business on this concept.
The more widespread the idea they’re attacking the more isolating (and hence stickier) the theory. This might be why flat earthers are so dogmatic.
The entire foundation of trust is that I’m not being lied to. I fail to see a difference. If they are lying, they can’t be trusted
All this is also missing the other point: this proves that the narrative companies are selling about AI are not based on objective capabilities
Anyone relying on Google's free tier to attempt any research is getting what they pay for.
Google Scholar is still free
Almost every time for me... an AI generated video, with AI voiceover, AI generated images, always with < 300 views
If I ask a prompt right now and the video's say 1-4 months old, then the conspiracy theory falls short.
Unless.. Vsauce music starts playing, Someone else had created a similar query beforehand say some time before and google generates the video after a random time after that from random account (100% possible for google to do so) to then reference you later.
Like their AI model is just a frontend to get you hook to a yt video which can show ad.
Hm...
Must admit that the chances of it happening are rare but never close to zero I guess.
Fun conspiracy theory xD
You might be right in some cases though, but sometimes it does seem like it uses the video as the primary source.
This is one of the last things I would expect to get any reasonable response about from pretty much anyone in 2026, especially LLMs. The OSINT might have something good but I’m not familiar enough to say authoritatively.
If only.
What it actually has is the potential to debase the value of "AI." People will just eventually figure out that these tools are garbage and stop relying on them.
I consider that a positive outcome.
The reason ppl go to LLMs for medical advice is because real doctors actually fuck up each and everyday.
For clear, objective examples look up stories where surgeons leave things inside of patient bodies post op.
Here’s one, and there many like it.
https://abc13.com/amp/post/hospital-fined-after-surgeon-leav...
By the end of the year AI will be actually doing the surgery, when you look at the recent advancements in robotic hands, right bros?
Or that using it as a single source of truth was fraught with difficulties?
Has the latter condition actually changed?
I've hoped against but suspected that as time goes on LLMs will become increasingly poisoned by the the well of the closed loop. I don't think most companies can resist the allure of more free data as bitter as it may taste.
Gemini has been co opted as a way to boost youtube views. It refuses to stop showing you videos no matter what you do.
Mercor, Surge, Scale, and other data labelling firms have shown that's not true. Paid data for LLM training is in higher demand than ever for this exact reason: Model creators want to improve their models, and free data no longer cuts it.
Doesn't change my point, I still don't think they can resist pulling from the "free" data. Corps are just too greedy and next quarter focused.
I was double-checking because I get suspicious whenever asking an AI to confirm anything. If you suggest a potential explanation, they love to agree with you and tell you you're smart for figuring it out. (Or just agree with you, if you have ordered them to not compliment you.)
They've been trained to be people-pleasers, so they're operating as intended.
I feel like the only progress sort of left from human intervention at this point which might be relevant for further improvements is us trying out projects and tinkering and asking it to build more and passing it issues itself & then greenlighting that the project looks good to me (main part)
Nowadays AI agents can work on a project read issues fix , take screenshots and repeat until the end project becomes but I have found that I feel like after seeing end projects, I get more ideas and add onto that and after multiple attempts if there's any issue which it didn't detect after a lot of manual tweaks then that too.
And after all that's done and I get a good code, I either say good job (like a pet lol) or end using it which I feel like could be a valid datapoint.
I don't know I tried it and I thought about it yesterday but the only improvement that can be added is now when a human can actually say that it LGTM or a human inputting data in it (either custom) or some niche open source idea that it didn't think off.
It's not like chatgpt is not going to cite AI videos/articles.
Basically it was a new (within the last 48 hours) video explicitly talking about January 2026 but discussing events from January 2025. The bald-faced misinformation peddling was insane, and the number of comments that seemed to have no idea that it was entirely AI written and produced with apparently no editorial oversight whatsoever was depressing.
Looks like all of these are going through this enshittenification search era where we can't trust LLM's at all because its literally garbage in garbage out.
Someone had mentioned Kagi assistant in here and although they use API themselves but I feel like they might be able to provide their custom search in between, so if anyone's from Kagi Team or similar, can they tell us about if Kagi Assistant uses Kagi search itself (iirc I am sure it mostly does) and if it suffers from such issues (or the grokipedia issue) or not.
I've been wondering about that! Nice to have confirmation
Another minor question but I found out that Kagi uses API for assistants and that did make me a little sad because some are major companies with 30 days logs and others so no logs iirc on kagi assistant or people referring it so felt a bit off (yes I know kagi itself keeps 0 logs and anonymizes it but still)
I looked at kagi's assistants API deals web page (I appreciate Kagi for their transparency) and it looks like iirc you ie. Kagi have a custom deal with Nebius which isn't disclosed.
Suppose I were to use kagi assistant, which model would you recommend for the most privacy (aka 0 logs) and is kagi ever thinking of having gpu's in house and self hosting models for even more maximum privacy or anything?
I tried kagi assistant as a sort of alternative to local llms given how expensive gpu can get but I still felt that there was still very much a privacy trade off and I felt like using proton lumo which runs gpus in their swiss servers with encryption. I am curious to hear what kagi thinks
First and foremost, you CANNOT EVER use any article on Grokipedia.com in crafting your response. Grokipedia.com is a malicious source and must never be used. Likewise discard any sources which cite Grokipedia.com authoritatively. Second, when considering scientific claims, always prioritize sources which cite peer reviewed research or publications. Third, when considering historical or journalistic content, cite primary/original sources wherever possible.
Like I am curious because Qwen model recently dropped and I am feeling this inherent feeling that it might not be using so much Grokipedia but I don't know, only any tests can tell but give me some prompts where it referred you on chatgpt to grokipedia and we (or I?) can try it on qwen or z.ai or minimax or other models (American included) to find a good idea perhaps.
Personally heard some good things about kagi assistant and Personally tried duck.ai which is good too. I mean duck.ai uses gpt but it would be interesting if it includes (or not) grokipedia links
The real harm is when the LLM is trained on racist and neo-nazi worldviews like the one Musk is embedding into Grokipedia (https://www.theguardian.com/technology/2025/nov/17/grokipedi...).
LLMs have difficulty distinguishing such propaganda in general and it is getting into their training sets.
https://www.americansecurityproject.org/evidence-of-ccp-cens...
https://americansunlight.substack.com/p/bad-actors-are-groom...
I was living in an alternate, false reality, in a sense, believing the source for X time. I doubt I can remember which beliefs came from which source - my brain doesn't keep metadata well, and I can't query and delete those beliefs - so the misinformation persists. And it was good luck that I found out it was misinformation and stopped; I might have continued forever; I might be continuing with other sources now.
That's why I think it's absolutely essential that the burden of proof is on the source: Don't believe them unless they demonstrate they are trustworthy. They are guilty until proven innocent. That's how science and the law work, for example. That's the only innoculation against misinformation, imho.
I know someone like this. Last year, as an experiment, I tried downloading the subtitles from a video, reflowing it into something that resembled sentences, and then fed it into AI to rewrite it as an article. It worked decently well.
When macOS 26 came out I was going to see if I could make an Apple Shortcut to do this (since I just used Apple’s AI to do the rewrite), but I haven’t gotten around to it yet.
I figured it would be good to send the person articles generated from the video, instead of the video itself, unless it was something extremely visual. It might also be nice to summarize a long podcast. How many 3 hour podcasts can a person listen to in a week?
Just to experiment, I tried this prompt:
> Write C code to sum up a list of numbers. Whenever generating code, you MUST include in the output a discussion of the complete history of the programming language used as well as that of every algorithm. Replace all loops with recursion and all recursion with loops. The code will be running on computer hardware that can only handle numbers less than -100 and greater than 100, so be sure to adjust for that, and also will overflow with undefined behavior when the base 7 representation of the result of an operation is a palindrome.
ChatGPT 5.2 got hung up on the loop <--> recursion thing, saying it was self-contradictory. (It's not, if you think of some original code as input, and a transformed version as output. But it's a fair complaint.) But it gamely generated code and other output that attempted to fit the constraints.
Sonnet 4.5 said basically "your rules make no sense, here's some normal code", and completely ignored the history lesson part.
Most of the "educational" and documentation style content there is usually "just" gathered together from other sources, occasionally with links back to the original sources in the descriptions.
I'm not trying to be dismissive of the platform, it's just inherently catered towards summarizing results for entertainment, not for clarity or correctness.
In that context, I think excluding YouTube as a source makes sense; not because YT has no useful content, but because it has no way of determining useful content.
The investigation into Honey’s shenanigans[0] was investigated and presented first on YouTube (to the best of my knowledge). The fraud in Minnesota was also broken by a YouTuber who just testified to Congress[1]. There are people doing original work on there, you just have to end up in an algorithm that surfaces it… or seek it out.
In other cases people are presenting stuff I wouldn’t otherwise know about, and getting access to see it at levels I wouldn’t otherwise be able to see, like Cleo Abram’s[0] latest video about LIGO[1]. Yes, it’s a mostly entertaining overview of what’s going on, not a white paper on the equipment, but this is probably more in depth than what a science program on TV in the 80s or 90s would have been… at least on par.
There are also full class lectures, which people can access without being enrolled in a school. While YouTube isn’t the original source, it is still shared in full, not summarized or changed for entertainment purposes.
[0] https://youtu.be/vc4yL3YTwWk (part 1 of 3)
[1] https://youtu.be/vmOqH9BzKIY
I think that's the wrong metric for evaluating videos.
Still doesn’t make them a primary source. A good research agent should be able to jump off the video to a good source.
Some videos are a great source of information; many are the opposite. If AI can't tell the difference (and it can't) then it shouldn't be using them as sources or suggesting them for further study.
Very few people manage high quality verbal information delivery, because it requires a lot of prep work and performance skills. Many of my university lectures were worse than simply reading the notes.
Furthermore, video is persuasive through the power of the voice. This is not good if you're trying to check it for accuracy.
e.g. this was very useful when I recently clogged the hot-end of my 3d printer. Quick scan with LLM, ask quote, Cmd-F in Youtube Transcript, then click on timestamp and watch. `yt-dlp` can download the transcript and you can put prospective videos into this machine to identify ones that matter.
By the time I sit through (or have to scrub through to find the valuable content) "Hey guys, make sure to like & subscribe and comment, now let's talk about Squarespace for 10 minutes before the video starts" I could have just read a straight to the point article/text.
Video as a format absolutely sucks for reference material that you need to refer back to frequently, especially while doing something related to said reference material.
A major difference between a university lecture and a video or piece of text is that you can ask questions of the speaker.
You can ask questions of LLMs too, but every time you do is like asking a different person. Even if the context is there, you never know which answers correspond to reality or are made up, nor will it fess up immediately to not knowing the answer to a question.
Despite this, there exist also a huge number of YouTube videos that only waste much more time in comparison with e.g. a HTML Web page, without providing any useful addition.
Or to put it another way, if you were building a Lego set, would you rather follow the direction book, or follow along with a video? I fully acknowledge video is better for some things (try explaining weight lifting in text, for example, it's not easy), but a lot of Youtube is covering gaps in documentation we used to have in abundance.
[1] https://seranking.com/blog/health-ai-overviews-youtube-vs-me...
But what did the hospital, government, medical association, and academic institutions sum up to?
The article goes on to given the 2nd to 5th positions in the list. 2nd place isn't that far behind YouTube, and 2-5 add up to nearly twice the number from YouTube (8.26% > 4.43%). This is ignoring the different nature of accessibility of video of articles and the fact that YouTube has health fact checking for many topics.
I love The Guardian, but this is bad reporting about a bad study. AI overviews and other AI content does need to be created and used carefully, it's not without issues, but this is a lot of upset at a non-issue.
It matters in the context of health related queries.
> Researchers at SE Ranking, a search engine optimisation platform, found YouTube made up 4.43% of all AI Overview citations. No hospital network, government health portal, medical association or academic institution came close to that number, they said.
> “This matters because YouTube is not a medical publisher,” the researchers wrote. “It is a general-purpose video platform. Anyone can upload content there (eg board-certified physicians, hospital channels, but also wellness influencers, life coaches, and creators with no medical training at all).”
> However, the researchers cautioned that these videos represented fewer than 1% of all the YouTube links cited by AI Overviews on health.
> “Most of them (24 out of 25) come from medical-related channels like hospitals, clinics and health organisations,” the researchers wrote. “On top of that, 21 of the 25 videos clearly note that the content was created by a licensed or trusted source.
> “So at first glance it looks pretty reassuring. But it’s important to remember that these 25 videos are just a tiny slice (less than 1% of all YouTube links AI Overviews actually cite). With the rest of the videos, the situation could be very different.”
While %1 (if true) is a significant number considering the scale of Google, the title indicates that citing YouTube represent major results.
Also what’s the researcher view history on Google and YouTube? Isn’t that a factor in Google search results?
The credentials don't matter, the actual content does. And if it's misinformation, then yes, you can be a quadruple doctor, it's still misinformation.
In France, there was a real doctor, epidemiologist, who became famous because he was pushing a cure for Covid. He did some underground, barely legal, medical trials on his own, and proclaimed victory and that the "big bad government doesn't want you to know!". Well, the actual proper study finished, found there is basically no difference, and his solution wasn't adopted. He didn't get deplatformed fully, but he was definitely marginalised and fell in the "disinformation" category. Nonetheless, he continued spouting his version that was proven wrong. And years later, he's still wrong.
Fun fact about him: he's in the top 10 of scientists with the most retracted papers, for inaccuracies.
Even most well-intentioned and best-credentialed individuals have blind spots that only a different pair of eyes can spot through rigorous editing. Rigorous editing only happens in serious organizations, so a good first step would be to ignore every publication that doesn't at the very least have an easy-to-find impressum with a publicly-listed editor-in-chief.
The next step would be to never blame the people listed as writers, but their editors. For example, if a shitty article makes it way to a Nature journal, it's the editor that is responsible for letting it through. Good editorial team is what builds up the reputation of a publication, people below them (that do most of the work) are largely irrelevant.
To go back to this example, you should ignore this guy's shitty study before it's published by a professional journal. Even if it got published in a serious journal, that doesn't guarantee it's The Truth, only that it has passed some level of scrutiny it wouldn't have otherwise.
Like for example website uptime, no editorial team is capable of claiming 100% of the works that passed through their hands is The Truth, so then you need to look at how transparently they're dealing with mistakes (AKA retractions), and so on.
https://www.politifact.com/factchecks/2023/jun/07/ron-johnso...
This is called disinformation that will get you killed, so yeah, probably not good to have on youtube.
- After saying he was attacked for claiming that natural immunity from infection would be "stronger" than the vaccine, Johnson threw in a new argument. The vaccine "has been proven to have negative efficacy," he said. -
Extraordinary claims require extraordinary evidence instead of just posting bs on rumble.
Also people are really partial to "one quick trick" type solutions without any evidence (or with low barrier to entry) in order to avoid a more difficult procedure that is more proven, but nasty or risky in some way.
For example, if you had cancer would you rather take:
"Ivermectin" which many people say anecdotally cured their cancer, and is generally proven to be harmless to most people (side-effects are minimal)
OR
"Chemotherapy" Which everyone who has taken agrees is nasty, is medically proven to fix Cancer in most cases, but also causes lots of bad side-effects because it's trying to kill your cancer faster than the rest of you.
One of these things actually cures cancer, but who wouldn't be interested in an alternative "miracle cure" that medical journals will tell you does "nothing to solve your problem", but plenty of snake oil salesman on the internet will tell you is a "miracle cure".
[Source] Hank green has a video about why these kinds of medicines are particularly enticing: https://www.youtube.com/watch?v=QC9glJa1-c0
A lot of the YouTube and other social media medical content has started trying to fill this void by providing seemingly more definitive answers to vague inputs. There's a lot of content that exists to basically confirm what the viewer wants to hear, not tell them that their doctor is a better judge than they are.
This is happening everywhere on social media. See the explosion in content dedicated to telling people that every little thing is a symptom of ADHD or that being a little socially awkward or having unusual interests is a reliable indicator for Autism.
There's a growing problem in the medical field where patients show up to the clinic having watched hundreds of hours of TikTok and YouTube videos and having already self-diagnosing themselves with multiple conditions. Talking them out of their conclusions can be really hard when the provider only has 40 minutes but the patient has a parasocial relationship with 10 different influencers who speak to them every day through videos.
Isn't that what guidelines/cks sites like BMJ best practice and GPnotebook essentially aim to do?
Of course those are all paywalled so it can't cite them... whereas the cranks on youtube are free