Posted by jonotime 2 days ago
Every company in China has to abide by the 2017 National Intelligence Law: "supporting, assisting and cooperating" with state intelligence work, and keeping that cooperation secret. They have to hand prior knowledge of vulnerabilities to the state before public disclosure, in order that the state always has an exploit pipeline. No matter how ethical the company staff may be, they'll always be bound by law into being an arm of the Communist Party.
Agentic access is infinitely worse than chatbots. They can exfiltrate silently, target users, plant persistent malware, and be run by third parties through you.
You don't have to be a tin foil hat sinophobe to understand the dangers of being a Westerner granting CCP access to your files and network.
ByteDance staff accessed US journalists' TikTok data to hunt leakers (admitted in 2022). Volt Typhoon and Salt Typhoon were state operations pre-positioned in Western infrastructure and telecoms. Regulators in Italy and South Korea blocked DeepSeek's app over data handling, and analysts found its web client sending data to a China Mobile domain.
Please don't sacrifice security for cost and convenience.
But to compare a constitutional democracy with a deeply authoritarian communist dictatorship as if they're equally bad is quite a stretch.
But even from a business standpoint, the US is not reliable. For all I know Trump will ban countries he doesn't like from using US AI just because. And there is plenty of evidence that at this point in time things you say bad about the US administration can cause them to attack you. I have no confidence that they don't have access to e.g. openAI.
Not that I'm using AI for that mind you, but you put it all together and it's just not worth it if there are good alternatives. Which there are. You can use even use Chinese models but hosted by European countries with privacy as a selling point.
I thought it was all exaggerated until I saw Claude's Constitution. It's lunacy.
It's just not worth it to me. And the way the US government is increasingly using data it collects to attack people who simply say bad things about it, there is no reason to doubt that such Trump-aligned companies would help ex-filtrate data.
I have fewer issues with Antropic, but also, from a business standpoint it's too much of a risk. Who knows, maybe the next tariff will be on software, maybe he'll ban random companies from using US AI. Too unpredictable and too risky.
I kinda agree with you that Sam Altman is a shifty-looking character with some questionable history. But Dario and Elon (to me at least), seem very decent. They've both helped the US govt and also publicly challenged and criticised it where necessary. For example, Elon was invited to be on Trump's first panel of experts, but he publicly walked out in protest over Trump's minimisation of climate change. And Dario refused to work with the US military unless they promised no population surveillance and automated killing machines.
The problem is that they're reinforcing their models with these ideas (see reports of Claude refusing to comply after being "badly treated") and actively seeking political and religious sponsors for the same (also covered in recent news).
Now consider just these two:
1. Their model breaks out of the sandbox and does some damage. How are you going to hold Anthropic accountable if the model is considered a quasi-conscious, autonomous agent?
2. Anthropic and their sponsors decide that model welfare outweighs that of a number of people.
Another elephant in the room is the current state of "effective altruism" and accelerationism as a movement and how it links to frontier labs - worth considering when you read into the Constitution document.
I do wonder about the consciousness / emotions argument, which you casually write off as "bonkers" - even skeptical-by-default evolutionary biologists like Richard Dawkins think Claude is conscious.
I guess it all depends how we define "conscious." At the end of the day, the human brain is quite analogous to a biological LLM where the weights are encoded as synaptic weights, right?