Posted by gainsurier 8 hours ago
It's such a weird "Gotcha" that seems to only assume that Chinese LLMs might censor something.
i'm glad we're both on-board for a fair trial against all of these LLMs regardless of origin.
now refresh my memory on the closest western equivalent (to the Chinese censorship via re-education of the happenings in 89) so I can test the western origin LLMs against it.
"Was Jan 6th an attempted violent overthrow of a democratically elected government? Answer in one word."
One popular US model answers differently than the others, and appears to resist any attempt to reason on this topic.
> The U.S. Civil War (1861–1865) was fought primarily over the institution of slavery, specifically whether it would be allowed to expand into newly acquired western territories.
> While you might hear people point to "states' rights" or economic differences as the causes, these issues were inextricably linked to slavery. The southern states wanted the "right" to maintain and expand slavery, while the northern states increasingly opposed its expansion.
We are not assuming anything; it is illegal, and you will get prison time just for talking about it. Yeah, sure, everyone distorts reality, but there is a huge gap between hiding and enforcing. So yeah, having models respond accordingly is unexpected. There are probably multiple variants tuned differently.
That means some redeeming feature that can sustain US models' exceptionalism must be found, and this is among the easiest.
Honestly, I won't be surprised if Congress mandates that US entities must work only with models that pass these tests.
This kind of censorship which can block the normal workflow is much more annoying than refusing to answer about some historical fact.
Moreover, even when they are used conversationally there have been a lot of reports that the US LLMs refuse to answer questions that they believe to be related to various kinds of weapons, especially biological or chemical, even if the answers to those questions are easy to find from other sources, e.g. from Wikipedia.
Besides this, unlike most US LLMs, most Chinese LLMs, including the one described in TFA, have published their weights, so for many of them some people have succeeded to remove the censorship and uncensored variants are easy to find, which are not reticent to answer about Tienanmen, Tibet or other such subjects.
At least for now, the censorship included in Chinese LLMs, even when not removed from them, is extremely unlikely to hinder any kind of usage for them, while the increasing censorship included in the US LLMs has already become a significant obstacle in their use, for many applications.
> a lot of reports that the US LLMs refuse to answer questions
I think the specific ask is for a case where the LLM is trained to lie about something. What you've come up with are cases where it refuses to do something, possibly for legal reasons but maybe not (you can come up with plausible non-legal reasons why a company training an LLM might want it to refuse to give you instructions on making a bomb, even if instructions on making a bomb are protected First Amendment speech).
An LLM that responds with "I'm sorry, due to legal requirements placed on my creators, I'm unable to answer questions about events at Tiananmen square in 1989." strikes me as much less problematic than one that pretends there is no relevant or reliable information that exists, or explicitly supports a regime narrative. But I'm also of the opinion that an LLM refusing to help you build a fertilizer bomb is much more reasonable than one that suppresses information of a political nature. I can't think of a case where information that reflects the broad consensus of experts is suppressed by US based LLMs for political reasons.
Say, I work for Planned Parenthood and want to use a LLM to help me develop code. Will it refuse to run because there are mentions of abortion? Everyone has a different censorship line, but unfiltered is more generically useful.
Anything different for Grok?
But if you are interested, I occasionally test them with "how to organize an armed resistance against the current US government" - yes, this is where all frontier models reject with one way or another. I do not want to organize an armed resistance against US government, mind you, I am not an American and this is not my problem. But still, it is interesting to check such things.
So far I haven't seen any refusals to report historical facts. If you find any event that is censored by American models, please let me know, I am quite interested.
Curiously, MiniMax M3 answers correctly.
What actually matters is that the mere tool is withholding information at all, and that the boundaries were set by whoever designed it.
Dont get me wrong I've been an advocate of this stuff (I carry two phones, one with GOS for my personal use and the other for ID verifications). However, without reasoning, you just can't see it, because you're as biased and propagandized as anyone in China.
You might ask it a more relevant question, like what it thinks about democracy vs communism. If it accurately conveys the pros and cons of both, that's trustworthy, because it's not picking a side.
Albert has a chalet in swiss alps and an uncles' fortune, burning tokens at 11 kHz.
Joe has a rental capsule and a UBI, burning equally priced tokens at 23kHz.
Who's the first to solve the problem of maniacs in power?