Posted by vslira 13 hours ago
(It’s of course not only the US, just that the largest AI providers are US based and we know from history how US agencies operate)
I'm using open weights models on a privacy-focused provider right now, and that's adequate for my current usage, but I'm rapidly getting to the point where my agent's access level to my data (and to a lesser extent, my accounts) is becoming something I'm not comfortable sending outside my network at all.
My hope is that models that are roughly on par with Deepseek V4 Flash can be run on hardware that I can own for <~$5k in the near future. We're close, but not there yet as far as I know.
The only long-term solution to this is self-hosting.
An actual objection I had while talking to an aerospace company was they don't want opensource models because the threat of it having a poisoned training example on specific systems.
It's easily the most hidden malware possible, completely undetectable until an exact set of tokens unlocks it. Is it line 100,543 of your security product? You will literally not know until it plants it in there.
Open models also let you read reasoning traces. That means anomalies would show up when the backdoor activates, like a run of unrelated words or a jump in top token probability. It's only undetectable until the first time it happens.
If you can't stop them from sometimes telling customers things like "yes, I will give you a penthouse suite at our hotel for only $3/night", why would you be able to guarantee that, with some specific set of tokens, they would produce a perfect and undetectable backdoor customized to the code at hand?
Anthropic has done such checks at the agent level: https://cybersecuritynews.com/anthropic-claude-hidden-code/
Their excuse was defending against distillation attacks but you can see how that can be abused
Apart from the obvious harms of invasion of privacy, and fishing expeditions being biased to the places you decided to fish. There is the simple fact that data can be misleading, especially without context. An interceped communication is a piece of data that is intrinsically tied to the trust of the inteceptor. A few people with an agenda can collaborate to create a seeming truth by 'discovering' the same thing from different sources.
Requiring warrants compelling information holders to provide data, not only serves the task of protection from abuse but also create a record of provenance that can be verified.
It also provides a degree of symmetry in capabilities which discourages actions that one party may do over another if they are motivated to act because they have a temporary advantage over another.
150 years ago was the invention of the telephone, and I think that articles like this seem to assume that prior to this, police just never caught any criminals.
A year when police were relatively rare, and quite new in the western legal experience, having only emerged from slave patrols in the United States and from the founding of the 1829 Scotland Yard in the United Kingdom. This is a year with living memory of a time when the state did not employ people to do what was ostensibly the civic responsibility of every person to quell crime and protect others.
Law enforcement doesn’t need this surveillance ability at all. All time periods prior to 25 years ago didn’t have it.
Additionally, there is no correlation between “law enforcement reads text messages” and crime rates going down.
Crime's been decreasing for years, and (from what I understand) this year is tracking to be one of the safest years on record?
Definitionally, there absolutely has to be a correlation (not causation) between those two factors you listed.
Most of the crime decrease from its peak happened before the year 2000.
That means back in the days of law enforcement needing to do low-tech wiretapping techniques, crime was still rapidly decreasing.
There’s just not even a correlation. And, as a reminder, correlation is not causation even if it was there.
Exhausting infrastructure vulnerabilities even without quantum could be a game changer for many technologies and enable things we can’t do right now, like vote on our phones.
I'm more curious what could be a right choice, and more importantly who is the "we" in this, as many decisions are largely made by companies and governments.
Unless you remember 2013, Snowden, that nothing was done (at most was some concern about doing it to US citizens, the rest of the world doesn't deserve privacy), all US (and/or five-eyes) based web companies must disclose users information and be forced to not disclose that, and things kept going surely at a faster and more intrusive rate in everything else, and of course phones.
You are complaining being sprinkled by water while at the bottom of the ocean. At least the big companies can find their own vulnerabilities with the AI tools you mention, the rest of the doors are still wide open.
It also seems likely to me that the US Gov. probably already has routine mechanisms for compelling targeted software updates for persons of interest, so I'm not sure that a more formalized backdoor than automatic updates is going to be surfaced in the mainstream, unless that is avenue is also cut down somehow.
Basically in the AI age, the difference between a vulnerability and a backdoor diminishes..
AI code is flawless and impenetrable!