> Grok AI generated large amounts of CSAM and nonconsensual intimate imagery
Grok Imagine has been considerably locked down in terms of intimate imagery over the last few weeks. E.g. Harley Quinn used to be one of the easiest characters to manipulate, with or without any resemblance to Margot Robbie. No more. X still serves up explicit hardcore, and Imagine used to get at least in that neighborhood, but that has been squelched. For prurient purposes, nerfed. Not at all limited to CSAM or real people. The pressure they're getting from all over seems to explain it.
I think the terminology here can confuse the issue a bit. And because its such a socially pernicious and stigmatized topic, it's hard to even talk about the phrasing without without raising doubts as to why you would get caught up in the weeds on such an issue. But, I would say there is use in making the distinction between something that is CSAM, where its a record of abuse that has happened to a real child, and sexualized depictions of children, or content promoting the sexualization of children. The social and personal harms are distinct, and if we are to firmly understand the arguments for/against guardrails on generative AI, then it's a distinction that needs to be made I think.
A problem you'd probably run into here is that it would be rather difficult to prove that no real CSAM was involved in the process of making the somehow-okay fake CSAM. Image generation models require training sets, after all. Do the companies training these models have the necessary data and evidence to prove every individual in every training image was over 18? Is generating fake CSAM okay if you trained the model on non-CSAM photos of real kids? I don't think so.
There are of course situations where being aggressive about this can hinder people's freedoms - like an adult who looks 'too young' having their freedoms curtailed because any photos or videos would Look Like CSAM - but I don't think they're common enough harms to justify holding back on regulation here.
Personally I think that even if the training data, is made of images of people who are over 18, the state of the adult industry is that there is a lot of harm and exploitation involved in that even. Do we have the evidence necessary to prove every over 18 person who's images are in the training data are okay with that use? So there is no somehow-okay anything. But if were dealing with models that can combine concepts .. then "adult" + "sex" = "sexualized adult" so "child" + "sex" = "sexualized child" .. this is just a fundamental capability of the technology. Without even getting into nudity where its easy to see how non CSAM, non-sexualize medical images etc, can fill in the gaps for the model.
This is why I think just calling it CSAM is confusing the arguments. I see the base harm as being the promotion of sexualization of children. And this is more a harm and risk to society and to people consuming these images. Of course if you extend this to "real childs image" + "sex" to get a sexualized deepfake then there is a kind of harm to that individual too, but its still a disctint thing from physical abuse..
This is just my instincts on this but I'd be really interested to see some real discussion between people with actual extensive understand and experience of all these areas.
> like an adult who looks 'too young' having their freedoms curtailed because any photos or videos would Look Like CSAM - but I don't think they're common enough harms
It is extremely common in some parts of the world. Age from visual appearance basically doesn't work inter-culturally, with or without AI, and ... there are people amassing something like 1/3 of the world that look like kids until they must retire from all work including use of gas stoves to boil water. Said people just literally look like kids, attracts wrong kinds of the rest 2/3, wrongly reject them if others come over, and gets banned on social media as being underaged, all the time.
In an ideal world, generated non-consensual imagery should be illegal through invasion of privacy through misappropriation of name or likeness, but I think only a limited number of states have those laws.
Is it private data if it was scraped from my social media profile marked private but leaked through a shared party? My expectation was that image would only be shared with those I wanted to see it (form of privacy).
"CSAM" is a codeword for anime. Users of this term routinely reject focusing on real kids and abuses. I assume "privacy" must be therefore a codeword too, especially considering that nonconsensual shocking images can be handled by defamation laws than privacy laws and principles.
I would hope we can agree that aggressive policing of anime and cartoons is a bad thing without denying the real existence of CSAM - the actual thing - or denying the bad things that have to occur for it to exist
I never denied the real existence of CSAM, I'm just saying that there are people who use "CSAM" to specifically mean anime and anime related photos and avoids discussion of actually detecting or preventing sexual abuses. I think I've seen it blatantly said on organizational blog posts somewhere that trying to move focus to kids is disingenuous or something.
But they did resist locking it down, recall Musk making fun of concerns? They clearly don't take governance seriously, its whatever Musk is gravitating to in his filter bubble.
Reddit did the same. Tumblr died when it banned porn. There seems to be a very perverse incentive for social media platforms to be as permissive as possible.
Personally, I'd be in favor of banning all sexual content on X, but it really feels like a legislative solution applying to all social media platforms might be the best solution.
And yes I realize the slippery slope that could put us on.
The issue wasn't specifically allowing NSFW content, it was allowing anyone to get grok to openly make NSFW deepfakes of anyone without even an attempt at policing things.
> Grok AI generated large amounts of CSAM and nonconsensual intimate imagery
on it own? on Musk's orders?
it's "people used Grok AI to generate large amounts of CSAM and nonconsensual intimate imagery". people use all sorts of things to do all sorts of shit.
There was a small number of crazy anime nerds rotating accounts tokenmaxxing Grok image generation to generate anime and cosplayer porn shorts with egregiously low yields and no intention of even publishing the results let alone monetizing, something like <1 result per daily quota across few accounts. Grok was briefly focused on adult usage, and they took advantage of that, until it was too much for even xAI. It seems British online advocacy groups tend to use "CSAM" as circumlocution for "anime", perhaps inspired by the fact that both imagined and real figures seen in anime related content always look to be below legal ages to some to the point that said some thinks it can be banned as willfully depicting underaged entities, so maybe this is a push coming from that direction.
your iPhone can be used to film non-imaginary CSAM and securely distribute it with Signal over the Internet. think of the children and throw it into a blender.
Our response[^1] to X’s petition debunks many claims the company uses in its arguments. For example, there’s little evidence the order placed an undue financial burden on X. In our letter, we note that the compliance cost is merely “a rounding error against the $200 billion valuation of X Corp. following the xAI merger.”
The letter is more interesting than the cover, undersigned by Center for Digital Democracy, Check My Ads Institute, Constitutional Alliance, Consumer Action, Consumer Federation of America, Consumers for Auto Reliability and Safety, Demand Progress Education Fund (“DPEF”), Electronic Frontier Foundation (“EFF”), Electronic Privacy Information Center (“EPIC”), National Consumers League (“NCL”), Oregon Consumer Justice, Oregon Consumer League, Public Citizen, Travelers United and Virginia Citizens Consumer Council, and drafted by DPEF's Special Advisor Kate Oh (kate@demandprogress.org),
EFF's Senior Staff Technologist William Budington (bill@eff.org), EPIC's Senior Counsel Sara Geoghegan (geoghegan@epic.org), and NCL's Senior Public Policy Manager Eden Iscil (edeni@nclnet.org).
The EFF does not blindly take the stance that "anything should be allowed as long as you do it with a computer". Their input here is very reasonable, and in standing with their principles.
Are you claiming that privacy can never be a prerequisite for freedom and/or justice?
It's trivially easy to see cases where freedom+justice+innovation can conflict with each other (it's even trivially easy to see where they can conflict with each other specifically for innovations involving the reduction of privacy, ye olde panopticon.)
So it's also trivially simple to understand that at some point you're gonna have to pick one over another. And note that freedom is the first word in that list.
> Grok AI generated large amounts of CSAM and nonconsensual intimate imagery
Grok Imagine has been considerably locked down in terms of intimate imagery over the last few weeks. E.g. Harley Quinn used to be one of the easiest characters to manipulate, with or without any resemblance to Margot Robbie. No more. X still serves up explicit hardcore, and Imagine used to get at least in that neighborhood, but that has been squelched. For prurient purposes, nerfed. Not at all limited to CSAM or real people. The pressure they're getting from all over seems to explain it.
Whatever you think about X's image generation models, I don't see how it is related to the petition that the EFF is opposing.
Is generation of non-consensual imagery really a privacy issue?
If someone publishes a real naked photo of you, that was acquired without your consent, that would be a privacy issue.
If someone generates a naked photo of you, even if it looks identical to a real photo, it's not your private data.
I think the terminology here can confuse the issue a bit. And because its such a socially pernicious and stigmatized topic, it's hard to even talk about the phrasing without without raising doubts as to why you would get caught up in the weeds on such an issue. But, I would say there is use in making the distinction between something that is CSAM, where its a record of abuse that has happened to a real child, and sexualized depictions of children, or content promoting the sexualization of children. The social and personal harms are distinct, and if we are to firmly understand the arguments for/against guardrails on generative AI, then it's a distinction that needs to be made I think.
A problem you'd probably run into here is that it would be rather difficult to prove that no real CSAM was involved in the process of making the somehow-okay fake CSAM. Image generation models require training sets, after all. Do the companies training these models have the necessary data and evidence to prove every individual in every training image was over 18? Is generating fake CSAM okay if you trained the model on non-CSAM photos of real kids? I don't think so.
There are of course situations where being aggressive about this can hinder people's freedoms - like an adult who looks 'too young' having their freedoms curtailed because any photos or videos would Look Like CSAM - but I don't think they're common enough harms to justify holding back on regulation here.
Personally I think that even if the training data, is made of images of people who are over 18, the state of the adult industry is that there is a lot of harm and exploitation involved in that even. Do we have the evidence necessary to prove every over 18 person who's images are in the training data are okay with that use? So there is no somehow-okay anything. But if were dealing with models that can combine concepts .. then "adult" + "sex" = "sexualized adult" so "child" + "sex" = "sexualized child" .. this is just a fundamental capability of the technology. Without even getting into nudity where its easy to see how non CSAM, non-sexualize medical images etc, can fill in the gaps for the model. This is why I think just calling it CSAM is confusing the arguments. I see the base harm as being the promotion of sexualization of children. And this is more a harm and risk to society and to people consuming these images. Of course if you extend this to "real childs image" + "sex" to get a sexualized deepfake then there is a kind of harm to that individual too, but its still a disctint thing from physical abuse.. This is just my instincts on this but I'd be really interested to see some real discussion between people with actual extensive understand and experience of all these areas.
> like an adult who looks 'too young' having their freedoms curtailed because any photos or videos would Look Like CSAM - but I don't think they're common enough harms
It is extremely common in some parts of the world. Age from visual appearance basically doesn't work inter-culturally, with or without AI, and ... there are people amassing something like 1/3 of the world that look like kids until they must retire from all work including use of gas stoves to boil water. Said people just literally look like kids, attracts wrong kinds of the rest 2/3, wrongly reject them if others come over, and gets banned on social media as being underaged, all the time.
In an ideal world, generated non-consensual imagery should be illegal through invasion of privacy through misappropriation of name or likeness, but I think only a limited number of states have those laws.
Is it private data if it was scraped from my social media profile marked private but leaked through a shared party? My expectation was that image would only be shared with those I wanted to see it (form of privacy).
That's fair. Is there any indication as to if xAi trains on private profiles?
"CSAM" is a codeword for anime. Users of this term routinely reject focusing on real kids and abuses. I assume "privacy" must be therefore a codeword too, especially considering that nonconsensual shocking images can be handled by defamation laws than privacy laws and principles.
I would hope we can agree that aggressive policing of anime and cartoons is a bad thing without denying the real existence of CSAM - the actual thing - or denying the bad things that have to occur for it to exist
I never denied the real existence of CSAM, I'm just saying that there are people who use "CSAM" to specifically mean anime and anime related photos and avoids discussion of actually detecting or preventing sexual abuses. I think I've seen it blatantly said on organizational blog posts somewhere that trying to move focus to kids is disingenuous or something.
You'd probably enjoy life a lot more if you go touch some grass once in a while. See a squirrel or something.
But they did resist locking it down, recall Musk making fun of concerns? They clearly don't take governance seriously, its whatever Musk is gravitating to in his filter bubble.
Reddit did the same. Tumblr died when it banned porn. There seems to be a very perverse incentive for social media platforms to be as permissive as possible.
Personally, I'd be in favor of banning all sexual content on X, but it really feels like a legislative solution applying to all social media platforms might be the best solution.
And yes I realize the slippery slope that could put us on.
The issue wasn't specifically allowing NSFW content, it was allowing anyone to get grok to openly make NSFW deepfakes of anyone without even an attempt at policing things.
> Grok AI generated large amounts of CSAM and nonconsensual intimate imagery
on it own? on Musk's orders?
it's "people used Grok AI to generate large amounts of CSAM and nonconsensual intimate imagery". people use all sorts of things to do all sorts of shit.
There was a small number of crazy anime nerds rotating accounts tokenmaxxing Grok image generation to generate anime and cosplayer porn shorts with egregiously low yields and no intention of even publishing the results let alone monetizing, something like <1 result per daily quota across few accounts. Grok was briefly focused on adult usage, and they took advantage of that, until it was too much for even xAI. It seems British online advocacy groups tend to use "CSAM" as circumlocution for "anime", perhaps inspired by the fact that both imagined and real figures seen in anime related content always look to be below legal ages to some to the point that said some thinks it can be banned as willfully depicting underaged entities, so maybe this is a push coming from that direction.
Yes and we should stop at elsewhere too. " it happened elsewhere n times" is a terrible excuse for inaction.
your iPhone can be used to film non-imaginary CSAM and securely distribute it with Signal over the Internet. think of the children and throw it into a blender.
Politely, what on earth point are you trying to make? Whatever it is it really is not coming across well.
"We only build and operate the orphan crushing machine, it's people that line up to turn the crank".
your phone, your browser, and the Internet itself are orphan crushing machines too, then.
Source: https://www.eff.org/deeplinks/2026/06/eff-and-allies-xs-ftc-...
TY!
The EFF featured update / press release at https://www.eff.org/deeplinks/2026/06/eff-and-allies-xs-ftc-... links to the letter, with color:
Our response[^1] to X’s petition debunks many claims the company uses in its arguments. For example, there’s little evidence the order placed an undue financial burden on X. In our letter, we note that the compliance cost is merely “a rounding error against the $200 billion valuation of X Corp. following the xAI merger.”
[^1]: public interest advocates opposing x petition 2026: https://www.eff.org/files/2026/07/02/public_interest_advocat...
The letter is more interesting than the cover, undersigned by Center for Digital Democracy, Check My Ads Institute, Constitutional Alliance, Consumer Action, Consumer Federation of America, Consumers for Auto Reliability and Safety, Demand Progress Education Fund (“DPEF”), Electronic Frontier Foundation (“EFF”), Electronic Privacy Information Center (“EPIC”), National Consumers League (“NCL”), Oregon Consumer Justice, Oregon Consumer League, Public Citizen, Travelers United and Virginia Citizens Consumer Council, and drafted by DPEF's Special Advisor Kate Oh (kate@demandprogress.org), EFF's Senior Staff Technologist William Budington (bill@eff.org), EPIC's Senior Counsel Sara Geoghegan (geoghegan@epic.org), and NCL's Senior Public Policy Manager Eden Iscil (edeni@nclnet.org).
Can update the link to the blog post; no need for this to be an arstechnica PDF
It didn't let me edit, I tried.
Why is the EFF arguing for less freedom on how computers can be used? The EFF should be against the government restricting computing freedom.
Their are two kinds of freedoms, freedom to and freedom of.
And they often contradict.
People should have freedom of abuse of their images, your freedom to abuse them
The EFF does not blindly take the stance that "anything should be allowed as long as you do it with a computer". Their input here is very reasonable, and in standing with their principles.
The letter is more about privacy, not freedom.
They're arguing X is a massive privacy risk and should not get any exemptions.
>EFF's mission is to ensure that technology supports freedom, justice, and innovation for all people of the world.
The EFF's mission statement supports them prioritizing freedom and innovation over privacy.
Are you claiming that privacy can never be a prerequisite for freedom and/or justice?
It's trivially easy to see cases where freedom+justice+innovation can conflict with each other (it's even trivially easy to see where they can conflict with each other specifically for innovations involving the reduction of privacy, ye olde panopticon.)
So it's also trivially simple to understand that at some point you're gonna have to pick one over another. And note that freedom is the first word in that list.
As far as I know, EFF has always championed privacy.
i don't know how you're equating "computing freedom" with regulation of privacy guards. should FTC not care about that? can you elaborate?
Why is the EFF arguing for less freedom on how computers can be used? The EFF should be against the government restricting computing freedom.
Basic human decency?
So not privacy?
But Trump just asked Musk for SpaceX stock to “seed US kids’ savings accounts” [1]. That trading of favors is almost explicit.
[1] https://arstechnica.com/tech-policy/2026/06/spacex-may-donat...
Musk spent $300 million in the 2024 campaign exactly for this kind of situation. He already bought himself the result.
The US presidency is for sale. At least it costs real amounts of money compared to bribery elsewhere. Well not sure if that’s good or bad.