I have some sympathy for Anthropic here because I've seen the headlines after OpenAI failed to report a shooter in a similar situation. So from their perspective, it's damned-if-you-don't, damned-if-you-do.
However, people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech. Before LLMs, Big Tech had no way to scrutinize the bulk of what was going on within their services, so you could have a secret hate diary in Google Docs. Now, everything you say or write can be automatically screened for red flags on a planetary scale, and probably will be because that's what the regulators and "concerned citizens" will demand. In a couple of years, you'll be biting your tongue a lot more often in private chats.
I suspect it will go further: imagine giving an mp3 to LLM to clean up some noise. It detects it was illegally downloaded from youtube, deletes it and automatically fines you via attached credit card.
Unless it becomes a requirement to be licensed to be able to use any kind of ai model. You know, for safety and stuff. And of course with appropriate reporting to institutions.
Or worse: downloading a picture of pirate ship and without any concern for the copyright asking the LLM to make a coloring page for your kid. BTW Chatgpt does that way better than Claude
> people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech
Yup. And with zero privacy protections in statute for AI chat, there is nothing to prevent an AI CEO looking to curry political favour from e.g. handing over the private correspondence of an opponent or an entire district’s residents.
And such was the case for a man who snapped a photo of his own child to send to the doctor which got uploaded to his Google photos resulting in his Google account of over a decade getting shutdown for CSAM.
As another commenter said, you're not chatting with a friend; you're chatting with Big Tech.
You should probably get a head start on waiting a couple years to bite your tongue and assume everything you type into a computer is summarized and sent to your boss, government, advertisers, political actors, insurance companies, worst enemy, etc. With phones, Alexas, and little AI tamagotchis, you probably shouldn't say much in person either.
They're actively rummaging through your inputs so the damned-if-you-don't case doesn't really exist; no one expects Anthropic to not notify law enforcement once they learn of something like this. What you might have expected was some privacy in the first place though, where Anthropic would never have learned of this in the first place and where the damned-if-you-do case wasn't a thing.
It's a byproduct of the nannyism safety marketing from the AI companies. I'm glad these cases were caught, but disagree with how they were disposed of. If the automated flagging is good, enforce it by default. If it's noisy, refine the tech then enforce it by default. This middleground where everything going through the platforms is subject to training and arbitrary human inspection in the midst of an acrid cloud of marketing-driven fearmongering is unacceptable, and it reinforces the idea the fearmongering is legitimate.
Somehow humanity survived the past 40 years without Microsoft Word and Excel phoning home and shopping users to the feds at random, I don't see why the standard should be any different for this new class of tooling.
> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism.
She didn't threaten anything, she wrote down that she was going to do it. A "threat" is more than a mere statement, especially when written in what is described as a "diary".
> A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office.
Obviously I don't want anyone to shoot up anything, but this seems like a weak case legally speaking.
Yeah.. if you write a personal note and it's backed up by the operating system, it appears to be in violation of this law as well (since the company could theoretically read it)
It is unlawful for any person to send, post, or transmit, or procure the sending, posting, or transmission of, a writing or other record, including an electronic record, in any manner in which it may be viewed by another person, when in such writing or record the person makes a threat to: (a) Kill or to do bodily harm to another person; or (b) Conduct a mass shooting or an act of terrorism.
I suppose since Anthropic's T&Cs allow them to have a person read your chats, that makes it violate the law. Of course, if Anthropic didn't have that in their T&Cs, it wouldn't have been illegal to write.
I have told llms all kinds of stories to find out what its answers would be. I always make it sound like it is the truth to make sure the AI answers in a way that it would if somebody actually said this. I also tested internal flagging systems of the ai company I work at with the most evil things a person can ever say to find out if it would flag them.
Of course I did not mean any of that stuff, but how can you make sure a human reviewer knows you did not mean it while the llm does not know that you did not mean it.
I guess its a miracle I am not in jail yet.
Flagging people for anything said to an llm sounds wrong to me because an LLM is not a real person and while some people put in their internal thoughts, others just roleplay and the two are inseparable just from reading it.
This is exactly the typical use I make of the llm.
Adding:
- I typically ask questions in the I form, regardless for whom or why I ask for.
- Gemini chats quite often end when it starts recommending psychological council or a suicide line, to talk about my problems. It apparently detects a persistent tendency to not agree with the party line. So it makes sense I must be suicidal ;-
But sure, as llm's start to babysit us, and know our inner dialog better than anyone else, we'll soon be debugging their opinion/behavior/co-existence/authority, when it comes to reporting people to the authorities, or taking on tasks in society in general. We'll hire doctors to cure our psychological profile from our record (Total Recall).
A Minority Report like this shouldn't cause a referral to the police.
Anthropic commits literal felonies by stealing millions of books and violating Copyright like it doesn't exist: no charge.
One unfortunate woman who happened to write the wrong thing in the wrong place is now having her life turned upside-down for perceived thought-crime.
To Anthropic, and all employees working there, your company's product and the result of your work is cruelty. You are enabling it and pushing it down everyone's throat. You can never again claim that you are the "ethical" AI company, for no such thing exists.
For the sake of argument what would happen if she had kept the diary locally and claude code scanned the file?
What would happen if it had scanned a file it didn’t have permission to look at and found this threat?
I honestly don’t know how I feel about this. On the one hand if you’re using claude as a diary you have no expectation of privacy and she was talking about committing a very serious crime.
Over in Europe they want to read all ofd our private messages, yet these chatbots, pretending to be our friends, will snitch on us just for our thoughts.
I once had a copy of 1984 in my checked baggage returning home to the USA and when I was unpacking the bag at home, the book had a notice inside the book that my bags had been inspected by TSA...
Only tangential but when I was an undergraduate studying philosophy I had Bertrand Russell's Why I Am Not A Christian in my carry-on, and the TSA saw that and had a field day.
In Europe they should provide the citizens with the service of their messages not reaching US servers. So basically that there is only one party reading along with them, not half the world.
This kind of thing will get worse with humanoid robots because they have cameras and microphones. Will they be programmed to tell on you if you break any kind of rule in front of them because their owners are terrified of being sued?
this feels very thought-crimey to me, but I think I'd have to actually see that chats to really decide. Like is she making plans/asking for advice? is she just talking about her feelings exactly as if it's a diary?
She is not being charged with planning, just making threats in a place a person could read - the person being employees of the company.
Basically, under this interpretation, any personal note you store in the servers of a company could qualify, even if you didn't ever imagine someone would read and as such you couldn't have thought about it as a threat
After people getting pilloried for social media posts from twenty years ago, I really hope (sensible) people will have the wherewithal to think twice about what they hand over to their chatbots.
LLM is not a person but T&C probably states that a human moderator may view any of it. Her lawyer could probably argue a moderator filtering usage isn't the intent of the law, it was more about publishing / sending message for other humans and it was never clear to her that a human was reviewing her private diary. Shouldn't LLM disclose that at some point when people are feeding really personal stuff?
> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
Thought crimes are real when you're sharing your thoughts with Claude
I wonder if "criminal intent" may be missing here given that most people probably operate under the assumption that Anthropic are not reading their messages.
I think people have a binary understanding on this. Someone is either reading their messages or not. While the reality is that all the messages are read by a machine (not unlike GMail and Outlook) and anything suspicious gets flagged up so a human can read it. This obscure the concept of "reading" as most laypeople understand it.
Summary: don't type in Claude anything you wouldn't like a human to read.
It tells you exactly what it does with your information if you ask it, including this exact scenario, and has for at least four months now (when I asked).
You're not wrong, and it may come down to this in court, but there's a difference between what people think happens, or the reasonable expectations, and what actually happens, and I think it's important to recognise that difference and why it comes about.
I don't think someone is an idiot for thinking that the information they type into their private Claude account is private. I also don't think people are idiots for thinking their phone is listening to them and giving them targeted advertising based on that. Both are reasonable deductions from their lived experiences. Both are wrong.
Yes but you are someone (i assume because you are on HN) that at least understands this point. There are A LOT of people that use it as their confidant, expecting it to be private. There is a massive education issue going on as its not in the interest of these Corpos to make you fear sharing all your details with them. Because, then you wont get Dot or whatever Anthropic comes up with and share all your personal info with it.
I think it's a reasonable point to make. Writing things down is a part of "thought" for many, including those who keep diaries/journals. If you write in a private journal you do so with the expectation that is not shared, and that wouldn't seem to break this law (with my naive reading).
I mean, at this point it's pretty well known that all tech services will be hacked by fable, anthropic themselves promised it, so writing your journal in google docs or claude or a txt document on your laptop or such is the same as releasing it publicly yeah?
If it were written on paper, and only in a room with no phones or cameras so fable couldn't hack it, then I think you wouldn't be sharing it.
I think the zero-risk approach common nowadays is insanely corrosive to democracy and freedom. If a million people fantasise about shooting the sheriff, and one ever goes on to do it, I don't believe avoiding it warrants creating an apparatus of mass surveillance. After all, if people were really serious about zero murder, the only practical solution would be to lock up everyone. Some (most?) tradeoffs have exponential costs at the limit and we/lawmakers should recognise that.
I have some sympathy for Anthropic here because I've seen the headlines after OpenAI failed to report a shooter in a similar situation. So from their perspective, it's damned-if-you-don't, damned-if-you-do.
However, people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech. Before LLMs, Big Tech had no way to scrutinize the bulk of what was going on within their services, so you could have a secret hate diary in Google Docs. Now, everything you say or write can be automatically screened for red flags on a planetary scale, and probably will be because that's what the regulators and "concerned citizens" will demand. In a couple of years, you'll be biting your tongue a lot more often in private chats.
I suspect it will go further: imagine giving an mp3 to LLM to clean up some noise. It detects it was illegally downloaded from youtube, deletes it and automatically fines you via attached credit card.
Wouldn't they have to fine themselves first?
They were fined for their illegal downloading. More than a credit card limit.
And dramatically lower than their expected value from the copyrighted material they scraped
I'll be glad for open models when the day (inevitably) comes that the proprietary LLMs no longer work in my interests.
> LLMs no longer work in my interests
As articles like these show, cloud-based LLMs don't work in your interest today.
Unless it becomes a requirement to be licensed to be able to use any kind of ai model. You know, for safety and stuff. And of course with appropriate reporting to institutions.
That day was yesterday. LLMs from big tech regularly refuse to do what you want.
Or worse: downloading a picture of pirate ship and without any concern for the copyright asking the LLM to make a coloring page for your kid. BTW Chatgpt does that way better than Claude
> people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech
Yup. And with zero privacy protections in statute for AI chat, there is nothing to prevent an AI CEO looking to curry political favour from e.g. handing over the private correspondence of an opponent or an entire district’s residents.
And such was the case for a man who snapped a photo of his own child to send to the doctor which got uploaded to his Google photos resulting in his Google account of over a decade getting shutdown for CSAM.
As another commenter said, you're not chatting with a friend; you're chatting with Big Tech.
How much do you love Big Brother?
You should probably get a head start on waiting a couple years to bite your tongue and assume everything you type into a computer is summarized and sent to your boss, government, advertisers, political actors, insurance companies, worst enemy, etc. With phones, Alexas, and little AI tamagotchis, you probably shouldn't say much in person either.
They're actively rummaging through your inputs so the damned-if-you-don't case doesn't really exist; no one expects Anthropic to not notify law enforcement once they learn of something like this. What you might have expected was some privacy in the first place though, where Anthropic would never have learned of this in the first place and where the damned-if-you-do case wasn't a thing.
It's a byproduct of the nannyism safety marketing from the AI companies. I'm glad these cases were caught, but disagree with how they were disposed of. If the automated flagging is good, enforce it by default. If it's noisy, refine the tech then enforce it by default. This middleground where everything going through the platforms is subject to training and arbitrary human inspection in the midst of an acrid cloud of marketing-driven fearmongering is unacceptable, and it reinforces the idea the fearmongering is legitimate.
Somehow humanity survived the past 40 years without Microsoft Word and Excel phoning home and shopping users to the feds at random, I don't see why the standard should be any different for this new class of tooling.
As if it is just nannyism marketing by tech companies. The EU tries to bring a new nanny law into legislation every other month
I don't understand how she violated this law:
> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism.
She didn't threaten anything, she wrote down that she was going to do it. A "threat" is more than a mere statement, especially when written in what is described as a "diary".
> A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office.
Obviously I don't want anyone to shoot up anything, but this seems like a weak case legally speaking.
Yeah.. if you write a personal note and it's backed up by the operating system, it appears to be in violation of this law as well (since the company could theoretically read it)
This almost smells like thought crime... Minority Report when?
It is unlawful for any person to send, post, or transmit, or procure the sending, posting, or transmission of, a writing or other record, including an electronic record, in any manner in which it may be viewed by another person, when in such writing or record the person makes a threat to: (a) Kill or to do bodily harm to another person; or (b) Conduct a mass shooting or an act of terrorism.
— via https://www.leg.state.fl.us/Statutes/index.cfm?App_mode=Disp...
The DA is serious about "in any matter."
Except if you're one of the "warfighters" that Anthropic supports
I mean, obviously? Soldiers operate under different rules than civilians. This has always been true.
Is the LLM now "another person"?
The other person is not the LLM, rather the service provider’s employees.
> may be viewed by another person
Was it viewed by another person? Yes.
I suppose since Anthropic's T&Cs allow them to have a person read your chats, that makes it violate the law. Of course, if Anthropic didn't have that in their T&Cs, it wouldn't have been illegal to write.
Depends on how good her lawyer is now
The company that runs it is
I have told llms all kinds of stories to find out what its answers would be. I always make it sound like it is the truth to make sure the AI answers in a way that it would if somebody actually said this. I also tested internal flagging systems of the ai company I work at with the most evil things a person can ever say to find out if it would flag them.
Of course I did not mean any of that stuff, but how can you make sure a human reviewer knows you did not mean it while the llm does not know that you did not mean it.
I guess its a miracle I am not in jail yet.
Flagging people for anything said to an llm sounds wrong to me because an LLM is not a real person and while some people put in their internal thoughts, others just roleplay and the two are inseparable just from reading it.
Don’t worry, they’ll store those messages forever and incarcerate you at their convenience.
This is exactly the typical use I make of the llm.
Adding: - I typically ask questions in the I form, regardless for whom or why I ask for. - Gemini chats quite often end when it starts recommending psychological council or a suicide line, to talk about my problems. It apparently detects a persistent tendency to not agree with the party line. So it makes sense I must be suicidal ;-
But sure, as llm's start to babysit us, and know our inner dialog better than anyone else, we'll soon be debugging their opinion/behavior/co-existence/authority, when it comes to reporting people to the authorities, or taking on tasks in society in general. We'll hire doctors to cure our psychological profile from our record (Total Recall).
A Minority Report like this shouldn't cause a referral to the police.
Anthropic commits literal felonies by stealing millions of books and violating Copyright like it doesn't exist: no charge.
One unfortunate woman who happened to write the wrong thing in the wrong place is now having her life turned upside-down for perceived thought-crime.
To Anthropic, and all employees working there, your company's product and the result of your work is cruelty. You are enabling it and pushing it down everyone's throat. You can never again claim that you are the "ethical" AI company, for no such thing exists.
For the sake of argument what would happen if she had kept the diary locally and claude code scanned the file?
What would happen if it had scanned a file it didn’t have permission to look at and found this threat?
I honestly don’t know how I feel about this. On the one hand if you’re using claude as a diary you have no expectation of privacy and she was talking about committing a very serious crime.
This still makes me feel queasy though.
Both of those scenarios would demonstrate even more commitment to safety so I imagine they’d be at least as likely to happen as this, if not more.
Oh man, what a crazy time to be alive.
Over in Europe they want to read all ofd our private messages, yet these chatbots, pretending to be our friends, will snitch on us just for our thoughts.
It's getting pretty orwellian out there.
I once had a copy of 1984 in my checked baggage returning home to the USA and when I was unpacking the bag at home, the book had a notice inside the book that my bags had been inspected by TSA...
Only tangential but when I was an undergraduate studying philosophy I had Bertrand Russell's Why I Am Not A Christian in my carry-on, and the TSA saw that and had a field day.
In Europe they should provide the citizens with the service of their messages not reaching US servers. So basically that there is only one party reading along with them, not half the world.
> making a written threat of violence under Florida law.
A diary constitutes making a threat?
Oh boy the roleplaying part of LLM world is in for a bad time
This kind of thing will get worse with humanoid robots because they have cameras and microphones. Will they be programmed to tell on you if you break any kind of rule in front of them because their owners are terrified of being sued?
Yes
In any case this is proof that law and negative PR can influence tech companies, including frontier AI providers.
Then again, I wish this worked in a way that would give users more privacy and agency, instead of less.
this feels very thought-crimey to me, but I think I'd have to actually see that chats to really decide. Like is she making plans/asking for advice? is she just talking about her feelings exactly as if it's a diary?
She is not being charged with planning, just making threats in a place a person could read - the person being employees of the company.
Basically, under this interpretation, any personal note you store in the servers of a company could qualify, even if you didn't ever imagine someone would read and as such you couldn't have thought about it as a threat
If only the woman hosted open-weight model in her house.
After people getting pilloried for social media posts from twenty years ago, I really hope (sensible) people will have the wherewithal to think twice about what they hand over to their chatbots.
> The communication must be made in a manner in which another person may view it.
How does a LLM prompt satisfy this? I guess it'll be an easy win for her.
LLM is not a person but T&C probably states that a human moderator may view any of it. Her lawyer could probably argue a moderator filtering usage isn't the intent of the law, it was more about publishing / sending message for other humans and it was never clear to her that a human was reviewing her private diary. Shouldn't LLM disclose that at some point when people are feeding really personal stuff?
AI snitches don't get stitches.
I guess it's time to come up with a more plausible explanation about why I asked how to make nuclear weapons almost every month to test LLM refusals.
The future is incredibly dark if they manage to ban open source models.
> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
Thought crimes are real when you're sharing your thoughts with Claude
Auditing logs is a crazy interpretation of "another person may view it"
I wonder if "criminal intent" may be missing here given that most people probably operate under the assumption that Anthropic are not reading their messages.
I think people have a binary understanding on this. Someone is either reading their messages or not. While the reality is that all the messages are read by a machine (not unlike GMail and Outlook) and anything suspicious gets flagged up so a human can read it. This obscure the concept of "reading" as most laypeople understand it.
Summary: don't type in Claude anything you wouldn't like a human to read.
It tells you exactly what it does with your information if you ask it, including this exact scenario, and has for at least four months now (when I asked).
You're not wrong, and it may come down to this in court, but there's a difference between what people think happens, or the reasonable expectations, and what actually happens, and I think it's important to recognise that difference and why it comes about.
I don't think someone is an idiot for thinking that the information they type into their private Claude account is private. I also don't think people are idiots for thinking their phone is listening to them and giving them targeted advertising based on that. Both are reasonable deductions from their lived experiences. Both are wrong.
Yes but you are someone (i assume because you are on HN) that at least understands this point. There are A LOT of people that use it as their confidant, expecting it to be private. There is a massive education issue going on as its not in the interest of these Corpos to make you fear sharing all your details with them. Because, then you wont get Dot or whatever Anthropic comes up with and share all your personal info with it.
this has nothing to do with claude. “thought” is fine. “written and shared” is illegal all over
https://www.nbcmiami.com/news/local/everyone-deserves-to-die...
https://www.wdsu.com/article/maryland-high-school-student-ch...
https://www.pinellassheriff.gov/21-023-deputies-arrest-pinel...
I think it's a reasonable point to make. Writing things down is a part of "thought" for many, including those who keep diaries/journals. If you write in a private journal you do so with the expectation that is not shared, and that wouldn't seem to break this law (with my naive reading).
I mean, at this point it's pretty well known that all tech services will be hacked by fable, anthropic themselves promised it, so writing your journal in google docs or claude or a txt document on your laptop or such is the same as releasing it publicly yeah?
If it were written on paper, and only in a room with no phones or cameras so fable couldn't hack it, then I think you wouldn't be sharing it.
> “written and shared” is illegal all over
I think it’s valid to ask if tapping something into Claude is legitimately sharing a threat.
I don’t think it is. I also think the sheriff could have found more-substantial evidence if she was actually planning domestic terrorism.
That said, if the shooting happened and we were looking at this from before? It’s a tough balance without an easy answer.
I think the zero-risk approach common nowadays is insanely corrosive to democracy and freedom. If a million people fantasise about shooting the sheriff, and one ever goes on to do it, I don't believe avoiding it warrants creating an apparatus of mass surveillance. After all, if people were really serious about zero murder, the only practical solution would be to lock up everyone. Some (most?) tradeoffs have exponential costs at the limit and we/lawmakers should recognise that.
Should CEOs of Antrophic and OpenAI be reported too then? They cannot stop saying that AI will destroy humans.
laws for thee and not for me
They are good at keeping notes on your criminal conspiracy