Posted by rbanffy 4 days ago
It's good to see this studied, but this should really be more obvious.
More likely in my opinion it's the context that matters here. If people know they're trying to be convinced of something and know they're talking to an AI, they may feel like they can trust (what they consider to be) an unbiased and rational AI. There's probably a fallacy associated with the assumption that an AI convincer is more rational and fact based, while it's more probable that the AI in the real world is, in fact, funded and trained by a think tank that wants you to vote against your own self-interests.
I suspect that LLMs not being human allows people to not just anthropomorphise the robot, but they project themselves upon it. When someone speaks to the LLM, they're kind of, or actually just literally talking to themselves. But then something new! 'Themselves' suggests new information to themselves. And now without the scary / icky meat and blood human on the other side, said person accepts the argument on its own merits.
Of course, the concern now is arguments based on false or statistically hacked data.
At least to me this is a given.
You can take the same set of factual information and give it to one person that studders, has poor presentation, and otherwise poor vocal cadence and people are going to have a hard time with it.
Now, if you took the same facts, maybe even the exact same paragraphs and gave it to Richard Feynman, even if you didn't know who he was, the presentation itself is likely to hook you.
Anecdotally I've noticed a trend of far-right bots/trolls moving on from just spouting hatred and really lean into cherry picked and misrepresented statistics to give themselves an aura of credibility. That strategy seems moderately effective because the argument consists of verifiable facts even though it forms a faulty conclusion.
So all of this is to say that this is a double edged sword where malicious actors will have the advantage.
It sounds like the arguments were basically AI slop hallucinations.
My son wanted to wear his undies on the outside a la Captain Underpants and I tried war gaming this with him - hmm, so your undie on the outside is going to require another one inside your pants and we are out of fresh pairs etc etc. In the end, he gave up on the idea of mimicking Captain Underpants.
Human intelligence is also powerful because when the assumptions we put into those shortcuts are incorrect things go terrifically wrong.
https://www.researchgate.net/publication/325733310_The_Influ...
The remark is risible because it is an example of why it it false. Everyone is subject to influence by words, that's what words are for. Being "confidently incorrect" is also called being "mistaken", and everyone does it every day, including the commenter by the very act of saying it never happens.
It only takes a small modicum of thought to prove that the thesis is obviously completely false, yet it is stated with absolute sneering confidence and contempt.
By the commenter's logic, only someone with "low IQ" would believe their own words.
These are also not the only ways that people can be misled about things, and susceptibility to them is likely not exclusively correlated to intelligence, but they might well be heuristics that LLMs can leverage effectively.
Also that correlation I mention likely can be quantified, but I doubt anyone is doing the research on it.
I'm going to assume the actual truth behind this is complex, varied, and social.
When I was younger I had moved from a school that slightly above average to one that was below average in grades. The area had a really strong cult of ignorance feeling and I saw a lot of different behaviors in people that reflect themselves in political patterns. People that embrace ignorance seem much less likely to follow people that outwardly signal intelligence, quite often the opposite happens and they outright reject them. Instead of intelligence they are much more apt to believe those that broadcast success.
This quite often leads to power seeking individuals "playing dumb" and bolstering their success signal. "Awe shucks, that guy that got top scores at Harvard is just like me who didn't graduate high school" would be a good analogy to frame it.
You mean you don't independently know one way or the other?
So you are taking it on his authority?
Because it was stated with confidence?
According to him, you must be one of those low IQ people.
Does that help to see the irony here, and why the original claim is not only clearly wrong but inherently elitist and insulting towards its own intended audience?
The alternative, the world that all intelligent people must live in at all times accordinng to the comment, is epistemic solipsism, that everything is presumed false, everyone is lying or erroneous, because there is no guaranteed way a priori to separate true statements from false ones.
> I doubt anyone is doing the research on it.
Not only is there research on it, this is like Psych/Journalism/Philosophy/Law/Communication 101 stuff. There are textbooks on it written over a century ago. It is the topic that the field of "Social Psychology" was invented to study.
The dominant theory is the Elaboration Likelihood Model, which describes and predicts based on situations when a person is more likely to doubt a claimed statement of fact, taking into account past experience with the source, cultural norms, motivation, time, concentration/nondistraction, and domain knowledge.
There is also tons of research specifically on the impact of intelligence on susceptability to persuasion by authoritative tone, and the reality is the *opposite* of what was claimed: smarter people who enjoy deep thinking are more likely to believe falsehoods (Price & Stone, 2004), because they perceive the overconfidence as exciting new information to think about, just like you are doing now.
So congrats, by uncritically believing the guy who incorrectly said you were low IQ for believing him, you actually provided evidence that suggests you are high IQ, because science shows that gullibleness is a trait of high-IQ people.
You could also be right-wing authoritarian, that's the other thing that strongly predicts gullibleness but I'm gonna guess you're smart instead.
I'm guessing you didn't read what I wrote, or maybe I flubbed the communication in some ways because I was trying to be cooperative and complimentary towards you.
Edit: ok, you might object to me saying you're gullible, but the point I'm making is that that's not a bad thing and is actually evidence in favor of intelligence, the opposite of what the other guy was saying.
I'm gullible too. I'm also short, nearsighted, bad at communication, unable to bench over 90 lbs, and tend to spend way too much time researching fields irrelevant to my own work. Susceptibility to over-confidently incorrect messaging is a scientifically proven achilles heel of smart people. It's important for us to know that so that we can metacognitively account for it and hopefully avoid falling victim to it by way of scams, cults, and hallucinating AIs.