So we finally tackle the issue of ai-text pollution and probably found a way to clean up the internet (from now on), yet people start complaining that "their" output is marked as spam.
Well, the solution is quite easy: start thinking on your own again and write the lines yourself.
I welcome this watermarking. Finally it's an easy detectable signal that someone just generated some request/answer to waste my time by forcing me thinking for the other person too. Now that time is over.
(It never was hard to spot this type of texts but now there is evidence)
The reason why Anthropic did it is imho the different - they don’t want to ingest their own output again (or somehow differentiate from already stored data) so they try to sign it.
The data volume that is generated daily is staggering so every petabyte counts :)
> So we finally tackle the issue of ai-text pollution and probably found a way to clean up the internet
We are very, very far from that! So far we have a single ai vendor introducing a statistical bias to their generation that can make it simpler to identify genAI in some (longer) texts. We don’t know yet how effective that will be in practice, and if other models will follow suit
Yep. Also, the vast majority of online content is very short (twitter, Reddit comments, HN comments, etc). It’s not something where SynthID type approach will be effective i assume. Still a good first baby step
Not sure why Anthropic is getting all the attention here. Google been doing this since at least late 2024[1] and openai I believe is doing this around 9 days ago[2].
Realized your right about OpenAI but they plan to. In the link they state "our goal is to expand provenance signals to all modalities including text" so its comming up regardless. I pretty sure Google does it though.
So, it seems the watermark is less what it sounds like (a stamp) and more an "imperceptible statistical pattern woven into the choice of words and sentence structures." Does that mean AI responses will sound even more "AI?" Like, will it become even easier to detect on a read-through because of the word choices and patterning? I see the word "imperceptible" there, but what does this mean in this context? My non-tech brain is kinda breaking here.
> Like, will it become even easier to detect on a read-through because of the word choices and patterning?
It's turned on right now. Can you tell a difference? I can't.
How many ways could I write this paragraph and still convey the same idea? Way more than we're aware of. Hundreds? Thousands? Maybe a lot more? The number of semantically similar variants increases exponentially with each word.
I suspect anthropic could turn their fingerprinting up or down if they want. If it were turned way up, claude would use weird phrasing but it would take very little text to tell if something were AI generated. If they turned it down, it would seem imperceptible to humans, but you would need a large sample to determine (with high accuracy) that a passage was AI generated. There's probably a very large middle ground where humans can't tell, and where it doesn't take a large text sample to know (with high probability) that some text was AI generated.
The information being encoded (the watermark) is the _relative ranking of each token compared to other possibilities_. If our prompt was "Write a positive review for a restaurant" and the response began:
"The restaurant "
Our next set of predictions might be:
[was, had, offers]
So we append the rank/index of the next token (0, 1, or 2) onto the secret. Given a long enough response, that secret becomes unique enough to use as a watermark. This obviously relies on having full deterministic access to the LLM itself, i.e. I don't believe it will be possible for users to derive the fingerprint from text that they've generated, only Anthropic will be able to.
The immediate objection is that this runs the risk of degrading the quality of the response. I think that's totally valid and I'll be curious how Anthropic handles it.
That's my very rough understanding! If someone with more knowledge wants to expand, feel free.
Supposedly you can avoid degradation by using synonyms. But not all words have synonyms. The more concrete and factual the prose, the harder it is to watermark. "Cow" is not a synonym for "cat" and "dark matter" is not a synonym for "galaxy."
So the watermark words will be biased towards filler and fluff where invisible substitutions are easier, and the content is less (cough...) load-bearing.
The likely outcome is the development of AI watermark strippers which filter out all the twitches and tells that make default AI writing so annoying.
Google seem to have given up on SynthID for text for now, so this is likely a harder problem than it looks. My guess is Anthropic announced this to meet regulatory requirements. But they don't have a robust detector, and I seriously doubt they have a robust system that can survive trivial rewriting by a different model.
From my understanding, AI outputs have patterns due to its text prediction algorithm and AI detectors just recognize those patterns as watermarks. If there used to be 1,000 different ways to write a paragraph, watermarks may reduce it to 300 and that’s still a lot. Humans shouldn’t be able to perceive these changes except the output is really short.
I think its somewhat the opposite of easy to detect patterns in the watermarked text. Regular authors can be statistically fingerprinted and I think we do a rough version of this ourselves. An LLM with dense watermarking may sound less like one author we have a low opinion of and more like an encyclopedia set made by a mix of authors we have low opinions of.
Sure, but if my experience (I know, I know) is anything to go by, it's increasingly difficult to get Claude to write in anything other than it's own house style.
A “watermark” is as an author’s mark. I struggle to understand how a myriad of different texts will produce the same watermark output. How big does a text have to be to generate this sign? What is the false positive rate (where my own authentic prose—gasp—is falsely accused of being AI). How do you “prove” it’s true? Will Anthropic offer some kind of service?
I find ChatGPT to be overly loquacious, and my preference for Claude is the brevity of output. Does this mean I will now have to suffer Claude’s gibbering, too?
> We'll support users and other third parties to detect Claude’s marks
This is the part that I fear most. What happens with false positives? Will I need to change arbitrary parts of my text if they are flagged as AI generated? Will I be forced to add mistakes?
And then, since LLMs are trained on existing, human-written, quality content, it may be that well-written text gets accidentally flagged more often?
And, if all the providers do this, and they use different watermarks, will I be forced to change things at random until I get a negative from all of those?
I very much think this will yield the wrong signal. There were already some academic plagiarism checkers around that would return false positives, and were blindly trusted by teachers and professors. I suppose AI checkers are all around now, and there will be more of those.
Proofreading is one of the best uses of LLMs that I can think about - I still write my text, then an LLM fixes some of my "errors" since I'm not a native speaker. It usually improves clarity. But now I risk my text to be classified as AI written with potentially bad outcomes.
I don't know whether I should blame the EU, though :-)
Third-party “AI checkers” and LLM watermarking at generation are a bit different. I’d suggest you read some of the methods; they’re approachable to someone on this forum. False positives aren’t really a failure case with good params. False negatives, sure.
The actual original dunk from that page is great enough to be reproduced here again verbatim:
> during the months of autumn [...] Work is left to feebler hands. ... In those months the great oracle becomes —what at other times it is not—simply silly. In spring and early summer, the Times is often violent, unfair, fallacious, inconsistent, intentionally unmeaning, even positively blundering, but it is very seldom merely silly. ... In the dead of autumn, when the second and third rate hands are on, we sink from nonsense written with a purpose to nonsense written because the writer must write either nonsense or nothing.
Sounds like "The Saturday Review of Politics, Literature, Science, and Art" where this came from was a wonderful little venture. Is there anything like it today?
Feels like "we sink from nonsense written with a purpose to nonsense written because the writer must write either nonsense or nothing" accurately describes HN sometimes :D
What would really be crazy is if there were a platform on which people provided commentary about other people's commentary on other people's commentary.
I wonder how it impacts code generation. It shouldn't impact prose in general because of synonyms and whatnot, but code requires exact reproduction. That is, what happens if you ask an LLM to recite a large, human-written excerpt as is, without modifications? Wouldn't the modified token sampler try to change some tokens here and there (for the watermark to work)?
For example, what if I say, "Repeat this text verbatim: %long_human_written_text%"? Would the output be recognized as AI-generated or human-generated?
It's not a case of needing a crutch. It's how you value your time. "Shall I spend 30 mins grammar checking this product marketing copy, or shall I get AI to do it while I start work coding a new feature?"
Will anyone even really care (outside of detecting cheaters in academia)? I see people talking about LLM based posts all the time even at work in Slack conversations where a manager verbatim pastes a clearly AI output as a response for some question and while people seem to care internally they don't actually seem to call them out.
Amusingly, the 'histrionic' Reddit post linked to in the article smells very AI generated to me. [0]
"And it doesn't stop there. There's the power question."
So not only is the Reddit user in question writing impassioned angry screeds about being caught using AI in their work, but they are presumably asking for Claude's help in writing said prose. I wonder what that prompt looked like?
>Amusingly, the 'histrionic' Reddit post linked to in the article smells very AI generated to me.
The whole text was a smorgasbord of the worst, most boring AI clichés possible. It was hard to finish for me, even though it is not that long, just because it was so badly written.
The only reason why I think it was partly written by a person who is just way too used to AI (as opposed to a simple prompt) is that the gaps in its logic feel more human than artificial.
Yeah this definitely seems like a case of "hey AI, take this mess of incoherent conspiracy-laden rambling points and turn it into something" and the AI had to try and stitch something together that even resembled something making sense. The logic doesn't even hold together.
Most vendors probably do it covertly, as watermarking is a tried and true method to defend intellectual property. I’m more interested in the motivations behind Anthropic admitting it publicly.
Well, of course. The stigma surrounding AI use will only get worse with stuff like this. I'm not interested in having people single me out for using AI. So glad I switched away from Anthropic.
The stigma surrounding AI use will get worse if... People know AI is being used? Sorry how does maintaining heightened paranoia by omitting clear signals help exactly?
Unfortunately the witch hunts don't stop whether you use AI or not. Even if you don't use it, (unavoidable) false positives mean the anti-ai crusaders come for you eventually anyway. Happened in plenty of purity-seeking art communities already, will happen in code communities as well. The people who get hurt are the people just trying to make things. Best not to empower self righteous people with flawed sin-detecting tools, that's guaranteed to turn out badly for everyone.
Agreed... How did the art community deal with the witch hunts?
In my case the consequences are mounting. I've already essentially abandoned two communities over it. I'm sitting on human reviewed and tested mesa patches I can't bring myself to submit to upstream because of the AI stigma. I was using AI to revive some of my GCC work as well but abandoned it after their policy announcement. There seems to be no end in sight.
It was bad in one that I was part of. First they had people put their work through those ai detectors and there were a bunch of false positives, so they started requiring “process videos” showing you actually working on the project. But then people started calling those AI generated. Nothing could satisfy them. So you’d have artists making handmade work, putting it through ai detectors, submitting process videos of them literally working on the piece, and still they were driven from the community because enough people just got toxic about it. Felt very Red Scare. “Are you now, or have you ever been, a user of AI?"
As an aside, I’ve never seen any pro ai people treat anyone not using ai like this.
Hiding your use of AI will only make things worse when people find out. Look at how badly the CEO of Saber Interactive is crashing out after they lied about firing a writer to replace them with AI.
But in the absence of a reliable signal (which I doubt this is anyway, so I'm speaking hypothetically), lots of people will get their work dismissed as AI-generated when it isn't.
People should not be dismissing other people's work at all, AI generated or not.
If you're looking for signals, it's because you want to do something based on the result. Maybe people should work on getting rid of that impulse instead.
> People should not be dismissing other people's work at all, AI generated or not.
It depends on what the nature of the work is. If it's a work where technical soundness is the most important thing, then I agree. The criteria should be whether or not the work is sound.
But not everything is like that. Take fine art, for instance, where being human-made is a core part of its value.
It's fine to appreciate human art for what it is. I can empathize with the idea that the art is an external expression of the author's mind. The output of the weights of the author's brain, if you will.
I absolutely draw the line at thoughtlessly dismissing AI art as "slop" though.
If you're a gamer: Have you seen the amount of AI slop on Steam? Or the amount of poorly created translations banged out with AI? Or the amount of shitty decomps appearing daily developed with Claude? People are looking for signals because AI usage is a strong signal of poor quality trash. You are not going to overcome that barrier by complaining it's being made obvious of it or complaining that people are being 'prejudiced', the only way you will overcome it is by proving you have something of actual worth. And if you can't prove that, then maybe you need to reevaluate what you are creating.
> Yes, I understand you want to dismiss works based purely on provenance without actually evaluating it on its merits.
I have gone over almost 20,000 games on Steam because I focus on indie game curation. If you have actually paid attention around you to the kinds of things people are throwing out there then you would understand. I have seen games that literally do not function with a bunch of generated assets [1], games whose sole providence is combining generated hollow art with a generated hollow story [2]. You are dealing with thousands of people who are doing exactly the same thing as you, with the same quality and the same reasoning. Your work can and will get filtered out along with the rest of them because you are using the same signal in the same way.
Isn’t this about turning a weakness into a feature? Claude is already unable to write original prose that does not trigger an AI detector like Pangram.
We can already tell when people write with AI, let's not worry!
But I guess people are worried the watermark stays even if they completely change what AI wrote
Legitimately we should have had this day one for a whole host of reasons (not the least of which is the issue of inadvertently using output from generative AI as training data for Generative AI).
But bigger than that is that while there are folks that want to be able to have a machine do their textual output for them with as little energy on their part as possible, the rest of us have to deal with the impact of that call on their part.
The purpose of communicating with other humans is not utilitarian, psychologically it is the pathway to connection and forms a large basis of how we are able to get along without eradicating ourselves as a species. Using generative AI to replace that humanity seems unthinkable to me, and having markers that help us differentiate serves as a way to keep others from gaslighting us, which undermines trust and communication.
I realize as a stereotype technical folks see communication as a means to a utilitarian end, but it's so much more important than that, so much so that even this small action of watermarking generative AI output can keep untold disasters from occurring because humans didn't know other humans were using it.
Well, um... I fail to see how this changes anything? Give me two random pieces of text, one written by Claude, one not. I'm pretty sure I will be able to tell with close to 100% accuracy which is which, as long as as there will be a non-trivial amount of text.
Claude outputs are already very easily distinguishable. The watermark is already pretty much there, even if it's not explicitly put in the outputs.
Based on Hacker News, other parts of the internet and even some job experience, I've come to the realization that many people legitimately have no idea that their pants are down and everything is on full display.
If you thought you could have AI "reorganize a paragraph" and not have anyone know, you're probably wrong. You can't because Claude can't fucking write even a paragraph without ruining it half the time. But apparently, a lot of people are either literally blind to writing style, or just don't care, because even most "Show HN" posts these days are extremely clearly AI slop with little care, marked and sometimes not marked. If you are fortunate enough to be blind to this (trust me it's not particularly fun, the style is truly grating) next time you're on a cool new project, just search/grep for "load bearing" and have a good laugh. or cry.
Of course, not everyone is necessarily trying to be misleading, but:
- I find this writing style to be truly miserable. It's like equivalently annoying to if the entire world started talking and typing in a slick quick-witted cynical way like we were all living in a Marvel movie script or something. Not exactly like that, but it's an analogy. I now realize some people may not notice if they got swapped into such a world and their parents and friends all just started talking like this out of nowhere.
- I find it disappointing when people don't seem to think they could do better, and would prefer to have an LLM speak in their own voice rather than them. It almost makes me wonder if people are even still alive behind the keyboard. Am I utterly insane or is it weird that people are deciding to have something else speak in their voice? For anything short of a speech or PR it just feels weirdly inappropriate. Even if your English isn't great, I probably prefer your English over Claude's. At worst... I'd probably prefer its direct translation attempt over its own attempt to write something.
I've been having a great time with LLMs but seeing the way people use them has introduced me to the fact that my taste is a lot higher above average than I realized. I'm not really thrilled about that realization, because all it really means is the bar is a lot lower than I thought it was, not that I'm going to win an award for having good taste. (It's not that good anyways. It's just that the depths you can go are apparently deeper than I thought.)
Whenever people respond with "well how do I know you're not just dumping out AI outputs" I just sigh. I've definitely seen evidence that people will accuse things of being AI output when they're not, yet I am completely unworried about either false negatives or false positives. If you really can't tell the difference between "the load-bearing smoking gun" and natural human prose, its embarassing for you, not me.
LLMs in this way deepen my already existing dispair, because they're yet again increasing the disconnect between me and the rest of the world.
This is a dumb idea by Claude. It's currently a binary switch and doesn't differentiate between "fix the grammar on this paragraph" and two pages of AI Slop.
There should at least be some cut off where it applies and where it doesn't.
I understand their reasoning, but imagine at some point in the future, watermarks eventually lead to a level 9 vulnerability in your system. Theoretically, it is possible
So we finally tackle the issue of ai-text pollution and probably found a way to clean up the internet (from now on), yet people start complaining that "their" output is marked as spam.
Well, the solution is quite easy: start thinking on your own again and write the lines yourself.
I welcome this watermarking. Finally it's an easy detectable signal that someone just generated some request/answer to waste my time by forcing me thinking for the other person too. Now that time is over.
(It never was hard to spot this type of texts but now there is evidence)
The reason why Anthropic did it is imho the different - they don’t want to ingest their own output again (or somehow differentiate from already stored data) so they try to sign it.
The data volume that is generated daily is staggering so every petabyte counts :)
> So we finally tackle the issue of ai-text pollution and probably found a way to clean up the internet
We are very, very far from that! So far we have a single ai vendor introducing a statistical bias to their generation that can make it simpler to identify genAI in some (longer) texts. We don’t know yet how effective that will be in practice, and if other models will follow suit
If other AI vendors don't follow suit, then people will move over to them to escape detection. Then of course Claude will stop watermarking.
Watermarking is probably done by other vendors covertly as well. See my other comment.
Yep. Also, the vast majority of online content is very short (twitter, Reddit comments, HN comments, etc). It’s not something where SynthID type approach will be effective i assume. Still a good first baby step
Not sure why Anthropic is getting all the attention here. Google been doing this since at least late 2024[1] and openai I believe is doing this around 9 days ago[2].
[1]https://www.nature.com/articles/s41586-024-08025-4 [2]https://help.openai.com/en/articles/8912793-provenance-signa...
That’s what I can’t put my finger on, see my other comment : why would Anthropic tell the public.
The EU AI Act requires them to do this. It would be pretty weird for them to say "we're not going to do this" while secretly doing it anyway.
Thanks
They don't do it for text I think that is the core issue.
Somehow it is acceptable for us to have photo, video or audio watermarked but not text or worse code.
Realized your right about OpenAI but they plan to. In the link they state "our goal is to expand provenance signals to all modalities including text" so its comming up regardless. I pretty sure Google does it though.
So, it seems the watermark is less what it sounds like (a stamp) and more an "imperceptible statistical pattern woven into the choice of words and sentence structures." Does that mean AI responses will sound even more "AI?" Like, will it become even easier to detect on a read-through because of the word choices and patterning? I see the word "imperceptible" there, but what does this mean in this context? My non-tech brain is kinda breaking here.
> Like, will it become even easier to detect on a read-through because of the word choices and patterning?
It's turned on right now. Can you tell a difference? I can't.
How many ways could I write this paragraph and still convey the same idea? Way more than we're aware of. Hundreds? Thousands? Maybe a lot more? The number of semantically similar variants increases exponentially with each word.
I suspect anthropic could turn their fingerprinting up or down if they want. If it were turned way up, claude would use weird phrasing but it would take very little text to tell if something were AI generated. If they turned it down, it would seem imperceptible to humans, but you would need a large sample to determine (with high accuracy) that a passage was AI generated. There's probably a very large middle ground where humans can't tell, and where it doesn't take a large text sample to know (with high probability) that some text was AI generated.
https://arxiv.org/html/2510.20075v6
It is quite counterintuitive, but you can hide texts the same size as the original text in imperceptible statistics of a text.
Compared to that feat, hiding a watermark is very easy.
The information being encoded (the watermark) is the _relative ranking of each token compared to other possibilities_. If our prompt was "Write a positive review for a restaurant" and the response began:
"The restaurant "
Our next set of predictions might be:
[was, had, offers]
So we append the rank/index of the next token (0, 1, or 2) onto the secret. Given a long enough response, that secret becomes unique enough to use as a watermark. This obviously relies on having full deterministic access to the LLM itself, i.e. I don't believe it will be possible for users to derive the fingerprint from text that they've generated, only Anthropic will be able to.
The immediate objection is that this runs the risk of degrading the quality of the response. I think that's totally valid and I'll be curious how Anthropic handles it.
That's my very rough understanding! If someone with more knowledge wants to expand, feel free.
Supposedly you can avoid degradation by using synonyms. But not all words have synonyms. The more concrete and factual the prose, the harder it is to watermark. "Cow" is not a synonym for "cat" and "dark matter" is not a synonym for "galaxy."
So the watermark words will be biased towards filler and fluff where invisible substitutions are easier, and the content is less (cough...) load-bearing.
The likely outcome is the development of AI watermark strippers which filter out all the twitches and tells that make default AI writing so annoying.
Google seem to have given up on SynthID for text for now, so this is likely a harder problem than it looks. My guess is Anthropic announced this to meet regulatory requirements. But they don't have a robust detector, and I seriously doubt they have a robust system that can survive trivial rewriting by a different model.
Do we need the original prompt to recover the watermark?
From my understanding, AI outputs have patterns due to its text prediction algorithm and AI detectors just recognize those patterns as watermarks. If there used to be 1,000 different ways to write a paragraph, watermarks may reduce it to 300 and that’s still a lot. Humans shouldn’t be able to perceive these changes except the output is really short.
https://www.seangoedecke.com/text-ai-watermarks/
Nope. AI labs have agreed to start inserting those watermarks purposefully.
I think its somewhat the opposite of easy to detect patterns in the watermarked text. Regular authors can be statistically fingerprinted and I think we do a rough version of this ourselves. An LLM with dense watermarking may sound less like one author we have a low opinion of and more like an encyclopedia set made by a mix of authors we have low opinions of.
Is it possible for them to sound even more AI?
Don’t forget the survivor bias is at play, you don’t see the ones that are good at passing for human written
Sure, but if my experience (I know, I know) is anything to go by, it's increasingly difficult to get Claude to write in anything other than it's own house style.
> "imperceptible statistical pattern”
This is how I’m feeling right now.
A “watermark” is as an author’s mark. I struggle to understand how a myriad of different texts will produce the same watermark output. How big does a text have to be to generate this sign? What is the false positive rate (where my own authentic prose—gasp—is falsely accused of being AI). How do you “prove” it’s true? Will Anthropic offer some kind of service?
I find ChatGPT to be overly loquacious, and my preference for Claude is the brevity of output. Does this mean I will now have to suffer Claude’s gibbering, too?
Even more em-dash.
Unlikely, but I really hope so!
[dead]
From: https://support.claude.com/en/articles/16266773-how-claude-m...
> We'll support users and other third parties to detect Claude’s marks
This is the part that I fear most. What happens with false positives? Will I need to change arbitrary parts of my text if they are flagged as AI generated? Will I be forced to add mistakes?
And then, since LLMs are trained on existing, human-written, quality content, it may be that well-written text gets accidentally flagged more often?
And, if all the providers do this, and they use different watermarks, will I be forced to change things at random until I get a negative from all of those?
I very much think this will yield the wrong signal. There were already some academic plagiarism checkers around that would return false positives, and were blindly trusted by teachers and professors. I suppose AI checkers are all around now, and there will be more of those.
Proofreading is one of the best uses of LLMs that I can think about - I still write my text, then an LLM fixes some of my "errors" since I'm not a native speaker. It usually improves clarity. But now I risk my text to be classified as AI written with potentially bad outcomes.
I don't know whether I should blame the EU, though :-)
Third-party “AI checkers” and LLM watermarking at generation are a bit different. I’d suggest you read some of the methods; they’re approachable to someone on this forum. False positives aren’t really a failure case with good params. False negatives, sure.
Summarising a reddit discussion counts as news nowadays?
https://en.wikipedia.org/wiki/Silly_season
The actual original dunk from that page is great enough to be reproduced here again verbatim:
> during the months of autumn [...] Work is left to feebler hands. ... In those months the great oracle becomes —what at other times it is not—simply silly. In spring and early summer, the Times is often violent, unfair, fallacious, inconsistent, intentionally unmeaning, even positively blundering, but it is very seldom merely silly. ... In the dead of autumn, when the second and third rate hands are on, we sink from nonsense written with a purpose to nonsense written because the writer must write either nonsense or nothing.
Sounds like "The Saturday Review of Politics, Literature, Science, and Art" where this came from was a wonderful little venture. Is there anything like it today?
Feels like "we sink from nonsense written with a purpose to nonsense written because the writer must write either nonsense or nothing" accurately describes HN sometimes :D
[dead]
Been the case for more than a decade
What would really be crazy is if there were a platform on which people provided commentary about other people's commentary on other people's commentary.
It's turtles all the way down
0 upvotes too
I wonder how it impacts code generation. It shouldn't impact prose in general because of synonyms and whatnot, but code requires exact reproduction. That is, what happens if you ask an LLM to recite a large, human-written excerpt as is, without modifications? Wouldn't the modified token sampler try to change some tokens here and there (for the watermark to work)?
For example, what if I say, "Repeat this text verbatim: %long_human_written_text%"? Would the output be recognized as AI-generated or human-generated?
Anthropic seems to be on a mission to drive away as many users as possible from Claude
Yeah, one would think they'd stop their nonsense given how much competition they're getting. Thank god I've already switched away from Anthropic.
They are probably working on a $500 subscription without watermarks. The majority of their users have an IQ below 100 and need the crutch.
It's not a case of needing a crutch. It's how you value your time. "Shall I spend 30 mins grammar checking this product marketing copy, or shall I get AI to do it while I start work coding a new feature?"
Will anyone even really care (outside of detecting cheaters in academia)? I see people talking about LLM based posts all the time even at work in Slack conversations where a manager verbatim pastes a clearly AI output as a response for some question and while people seem to care internally they don't actually seem to call them out.
I wondered how watermarks are in the interests of the AI companies, but now it is more clear to me as there are two benefits:
first, they can skip them for training and
second, the AI output can be licensed to reduce, or at least detect, that if output is used as training data for other (more open) models.
Could probably make a 1B model that removes watermarks. Chooses synonyms, trains on user style, changes order around etc.
Amusingly, the 'histrionic' Reddit post linked to in the article smells very AI generated to me. [0]
"And it doesn't stop there. There's the power question."
So not only is the Reddit user in question writing impassioned angry screeds about being caught using AI in their work, but they are presumably asking for Claude's help in writing said prose. I wonder what that prompt looked like?
[0] https://www.reddit.com/r/artificial/comments/1vlzjc8/about_t...
>Amusingly, the 'histrionic' Reddit post linked to in the article smells very AI generated to me.
The whole text was a smorgasbord of the worst, most boring AI clichés possible. It was hard to finish for me, even though it is not that long, just because it was so badly written.
The only reason why I think it was partly written by a person who is just way too used to AI (as opposed to a simple prompt) is that the gaps in its logic feel more human than artificial.
Yeah this definitely seems like a case of "hey AI, take this mess of incoherent conspiracy-laden rambling points and turn it into something" and the AI had to try and stitch something together that even resembled something making sense. The logic doesn't even hold together.
This is aimed at academia with all the cheating kids and youngsters. I really dont have a big problem with it but I doubt it will be successful.
Can’t you just paste the copy into an editor that doesn’t support weird characters like EMacs or BBEdit?
Most vendors probably do it covertly, as watermarking is a tried and true method to defend intellectual property. I’m more interested in the motivations behind Anthropic admitting it publicly.
Well, of course. The stigma surrounding AI use will only get worse with stuff like this. I'm not interested in having people single me out for using AI. So glad I switched away from Anthropic.
The stigma surrounding AI use will get worse if... People know AI is being used? Sorry how does maintaining heightened paranoia by omitting clear signals help exactly?
I'm not interested in "helping" people call me "slop fetishist" or "clanker lover", thanks.
Did you write this with AI? jk
Unfortunately the witch hunts don't stop whether you use AI or not. Even if you don't use it, (unavoidable) false positives mean the anti-ai crusaders come for you eventually anyway. Happened in plenty of purity-seeking art communities already, will happen in code communities as well. The people who get hurt are the people just trying to make things. Best not to empower self righteous people with flawed sin-detecting tools, that's guaranteed to turn out badly for everyone.
Agreed... How did the art community deal with the witch hunts?
In my case the consequences are mounting. I've already essentially abandoned two communities over it. I'm sitting on human reviewed and tested mesa patches I can't bring myself to submit to upstream because of the AI stigma. I was using AI to revive some of my GCC work as well but abandoned it after their policy announcement. There seems to be no end in sight.
It was bad in one that I was part of. First they had people put their work through those ai detectors and there were a bunch of false positives, so they started requiring “process videos” showing you actually working on the project. But then people started calling those AI generated. Nothing could satisfy them. So you’d have artists making handmade work, putting it through ai detectors, submitting process videos of them literally working on the piece, and still they were driven from the community because enough people just got toxic about it. Felt very Red Scare. “Are you now, or have you ever been, a user of AI?"
As an aside, I’ve never seen any pro ai people treat anyone not using ai like this.
Hiding your use of AI will only make things worse when people find out. Look at how badly the CEO of Saber Interactive is crashing out after they lied about firing a writer to replace them with AI.
I don't hide my AI use at all.
Then why care about a watermark?
Because it's yet another signal prejudiced people get to use against me to dismiss my projects as slop.
But in the absence of a reliable signal (which I doubt this is anyway, so I'm speaking hypothetically), lots of people will get their work dismissed as AI-generated when it isn't.
People should not be dismissing other people's work at all, AI generated or not.
If you're looking for signals, it's because you want to do something based on the result. Maybe people should work on getting rid of that impulse instead.
> People should not be dismissing other people's work at all, AI generated or not.
It depends on what the nature of the work is. If it's a work where technical soundness is the most important thing, then I agree. The criteria should be whether or not the work is sound.
But not everything is like that. Take fine art, for instance, where being human-made is a core part of its value.
It's fine to appreciate human art for what it is. I can empathize with the idea that the art is an external expression of the author's mind. The output of the weights of the author's brain, if you will.
I absolutely draw the line at thoughtlessly dismissing AI art as "slop" though.
It's a thought-terminating cliche at this point.
If you're a gamer: Have you seen the amount of AI slop on Steam? Or the amount of poorly created translations banged out with AI? Or the amount of shitty decomps appearing daily developed with Claude? People are looking for signals because AI usage is a strong signal of poor quality trash. You are not going to overcome that barrier by complaining it's being made obvious of it or complaining that people are being 'prejudiced', the only way you will overcome it is by proving you have something of actual worth. And if you can't prove that, then maybe you need to reevaluate what you are creating.
> Have you seen the amount of AI slop on Steam?
I think I might have purchased some of those.
> Or the amount of shitty decomps appearing daily developed with Claude?
I'm developing one of those "shitty" decomps.
https://news.ycombinator.com/item?id=49267102
I'm still annotating the game myself though. Because I care about it and actually want to learn how it works.
> People are looking for signals because AI usage is a strong signal of poor quality trash.
Yes, I understand you want to dismiss works based purely on provenance without actually evaluating it on its merits.
Pre-judgement, as I said. Prejudice.
No, I will not help you do it.
> the only way you will overcome it is by proving you have something of actual worth
Ah yes, please continue. Do tell me about the worth of my own projects. GitHub and website links are on my profile.
> Yes, I understand you want to dismiss works based purely on provenance without actually evaluating it on its merits.
I have gone over almost 20,000 games on Steam because I focus on indie game curation. If you have actually paid attention around you to the kinds of things people are throwing out there then you would understand. I have seen games that literally do not function with a bunch of generated assets [1], games whose sole providence is combining generated hollow art with a generated hollow story [2]. You are dealing with thousands of people who are doing exactly the same thing as you, with the same quality and the same reasoning. Your work can and will get filtered out along with the rest of them because you are using the same signal in the same way.
[1] https://store.steampowered.com/app/2832640/DEAD_GET_REKT/
[2] https://store.steampowered.com/app/3694060/Manos_The_Hands_O...
Sounds like you want a quality filter, not an AI filter.
> You are dealing with thousands of people who are doing exactly the same thing as you, with the same quality and the same reasoning.
So you've just concluded I'm one of those people you keep reviewing on Steam. Very nice.
Did you actually look at my projects?
Isn’t this about turning a weakness into a feature? Claude is already unable to write original prose that does not trigger an AI detector like Pangram.
We can already tell when people write with AI, let's not worry! But I guess people are worried the watermark stays even if they completely change what AI wrote
Shouldn’t this count as ‘edited headline’ ? At least when I click on the link it says “Some Claude users…”
Do these watermarks show up in code as well?
Sauce for the goose.
Legitimately we should have had this day one for a whole host of reasons (not the least of which is the issue of inadvertently using output from generative AI as training data for Generative AI).
But bigger than that is that while there are folks that want to be able to have a machine do their textual output for them with as little energy on their part as possible, the rest of us have to deal with the impact of that call on their part.
The purpose of communicating with other humans is not utilitarian, psychologically it is the pathway to connection and forms a large basis of how we are able to get along without eradicating ourselves as a species. Using generative AI to replace that humanity seems unthinkable to me, and having markers that help us differentiate serves as a way to keep others from gaslighting us, which undermines trust and communication.
I realize as a stereotype technical folks see communication as a means to a utilitarian end, but it's so much more important than that, so much so that even this small action of watermarking generative AI output can keep untold disasters from occurring because humans didn't know other humans were using it.
Then a whole industry will rise at tracking and removing those watermarks.
Already happening. Just have a quick search for llm watermark remover and discovered multiple repos already available.
It’s very easy, just change each third word into something different than what you primarily wrote.
It’s very simple, just change every third word for something different to what you originally wrote.
It’s very straightforward, just change each and every third word for something different instead of what you actually wrote.
And so on ad infinitum.
Well, um... I fail to see how this changes anything? Give me two random pieces of text, one written by Claude, one not. I'm pretty sure I will be able to tell with close to 100% accuracy which is which, as long as as there will be a non-trivial amount of text.
Claude outputs are already very easily distinguishable. The watermark is already pretty much there, even if it's not explicitly put in the outputs.
Based on Hacker News, other parts of the internet and even some job experience, I've come to the realization that many people legitimately have no idea that their pants are down and everything is on full display.
If you thought you could have AI "reorganize a paragraph" and not have anyone know, you're probably wrong. You can't because Claude can't fucking write even a paragraph without ruining it half the time. But apparently, a lot of people are either literally blind to writing style, or just don't care, because even most "Show HN" posts these days are extremely clearly AI slop with little care, marked and sometimes not marked. If you are fortunate enough to be blind to this (trust me it's not particularly fun, the style is truly grating) next time you're on a cool new project, just search/grep for "load bearing" and have a good laugh. or cry.
Of course, not everyone is necessarily trying to be misleading, but:
- I find this writing style to be truly miserable. It's like equivalently annoying to if the entire world started talking and typing in a slick quick-witted cynical way like we were all living in a Marvel movie script or something. Not exactly like that, but it's an analogy. I now realize some people may not notice if they got swapped into such a world and their parents and friends all just started talking like this out of nowhere.
- I find it disappointing when people don't seem to think they could do better, and would prefer to have an LLM speak in their own voice rather than them. It almost makes me wonder if people are even still alive behind the keyboard. Am I utterly insane or is it weird that people are deciding to have something else speak in their voice? For anything short of a speech or PR it just feels weirdly inappropriate. Even if your English isn't great, I probably prefer your English over Claude's. At worst... I'd probably prefer its direct translation attempt over its own attempt to write something.
I've been having a great time with LLMs but seeing the way people use them has introduced me to the fact that my taste is a lot higher above average than I realized. I'm not really thrilled about that realization, because all it really means is the bar is a lot lower than I thought it was, not that I'm going to win an award for having good taste. (It's not that good anyways. It's just that the depths you can go are apparently deeper than I thought.)
Whenever people respond with "well how do I know you're not just dumping out AI outputs" I just sigh. I've definitely seen evidence that people will accuse things of being AI output when they're not, yet I am completely unworried about either false negatives or false positives. If you really can't tell the difference between "the load-bearing smoking gun" and natural human prose, its embarassing for you, not me.
LLMs in this way deepen my already existing dispair, because they're yet again increasing the disconnect between me and the rest of the world.
This is a dumb idea by Claude. It's currently a binary switch and doesn't differentiate between "fix the grammar on this paragraph" and two pages of AI Slop.
There should at least be some cut off where it applies and where it doesn't.
Yet another (accidental) win for the open-weight models.
I understand their reasoning, but imagine at some point in the future, watermarks eventually lead to a level 9 vulnerability in your system. Theoretically, it is possible