Posted by emptybits 17 hours ago
Anthropic (in discussion with Pentagon) claimed mass surveillance is their red line.
Yet, they do automated mass surveillance of their users on behalf of police.
1791144575 | Florida woman used Claude as a diary, then Anthropic reported an entry to police | https://www.techspot.com/news/114091-florida-woman-used-clau... | https://news.ycombinator.com/item?id=49957340 | 0 comments
1791147034 | Florida woman used Claude as a diary, then Anthropic reported an entry to police | https://www.techspot.com/news/114091-florida-woman-used-clau... | https://news.ycombinator.com/item?id=49957692 | 0 comments
1791149399 | Florida woman used Claude as a diary, then Anthropic reported an entry to police | https://www.techspot.com/news/114091-florida-woman-used-clau... | https://news.ycombinator.com/item?id=49958089 | 3 comments
1791213096 | Florida woman arrested for allegedly making threats in an AI chat | https://www.theverge.com/ai-artificial-intelligence/1004747/... | https://news.ycombinator.com/item?id=49965895 | 2 comments
Is this an invasion of their privacy (reporting to police)? Yes, but possibly warranted?
Should a social worker have contacted them rather than the police? Probably, if for no other reason than to ask if they were serious about harming someone.
Difficult questions, I'm still undecided on whether it's OK to always ignore someone's rants, even if it may be (or they think it may be) a private diary.
Actually charging them with a felony seems pretty quick to accuse. (Maybe I missed a hint about how long the investigation took before the felongy charge?)
I am going to flag you though.
If they were to offer total privacy, is it ok for the public to use chat to get advice on _how_ to commit a crime? Basically everyone agrees that crime-committing advice is inappropriate…but if it is not ok to get advice, that means there must be a portal for law enforcement to step in when that may have happened. Then the question becomes what is the line for when to report? In other words, the issue needs to be adjudicated.
But we don’t want OpenAI/Claude to have some $20/hour reviewer making decisions that are this high stakes…we need the courts to do the judicial work because they (1) have a public charter, (2) have meaningful expertise and specialization at interpreting the law and (3) we can hold them accountable.
> If they were to offer total privacy, is it ok for the public to use chat to get advice on _how_ to commit a crime?
Yes it should be, but it should be illegal for a company providing chat service to respond with anything other than a refusal when doing so.
That detection and refusal should be a private closed loop though, anonymizing any data that will be passed into a training pipeline, or ads targeting. This requirement for closed loop private chats should be mandated by law sooner than later. Otherwise we're getting into very tricky territory where the temptation of alerting on things like pre-crime grows too close.
Over in Europe they want to read all ofd our private messages, yet these chatbots, pretending to be our friends, will snitch on us just for our thoughts.
It's getting pretty orwellian out there.
What would happen if it had scanned a file it didn’t have permission to look at and found this threat?
I honestly don’t know how I feel about this. On the one hand if you’re using claude as a diary you have no expectation of privacy and she was talking about committing a very serious crime.
This still makes me feel queasy though.
For the sake of even more argument, imagine if she was writing her thoughts with a pencil, on a good old fashioned paper diary, and she had a phone nearby and the phone took a picture of her diary, OCR'd the words, and reported it to the police?
I think the defendant could successfully defend themselves by claiming they did not know (or intend!) the message could be viewed by another person, as they were plainly using it as a private diary.