Top
Best
New

Posted by mfiguiere 15 hours ago

How Claude marks AI-generated content(support.claude.com)
211 points | 172 commentspage 4
orbital-decay 8 hours ago|
Does it mean their models will always write slop? Making the writing non-collapsed to specific patterns seems to break any injected/learned fingerprinting.
matheusmoreira 7 hours ago||
This is terrible news given the stigma against AI in general. I really don't want people singling me out for it.
Computer0 15 hours ago||
So this won't be happening in the US, but in the EU:

"When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response.

Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing. Watermarking will be applied at the model level, which means it will be present no matter which Claude product or surface the text comes from."

travisgriggs 14 hours ago||
I would love to see what this looks like in practice. Especially in generated code. I assume this is more than insertion of non visible special unicode whitespace characters, but more in the pattern of the text content itself?
toufka 14 hours ago|||
I'm guessing - probably some textual variation on Benford's law? [1]. Trivial for compute, painful for a human.

- "Ensure distribution of vowels is in >99th percentile of human work"

- "Ensure the distribution of the letter "s" is within 99th percentile of human work"

- "Ensure the distribution of the letter "L" is periodic with periodicity within 5% of 1/N characters.

- "Ensure there is a cross-linguistic 'typo' (colour vs color) at 1/N words, where N: 1000 = Model1, 2000 = Model2, 3000 = Model3.

- "Ensure the distribution of tense error is within 99th percentile of human work"

If more than 3 dimensions have a score >99% percentile of human, let's call it watermarked...

- 1) https://en.wikipedia.org/wiki/Benford%27s_law

Der_Einzige 14 hours ago|||
Models can't reliably follow instructions involving their own logprobs unless they can take agentic control and use quite sophisticated dynamic grammars/structures/constraints to force this behavior in one shot (which can be slow and the dynamic grammar modification feature isn't supported in closed model APIs for safety reasons) or repeated attempts at rewriting which is expensive/slow.

Yes they can do this, but it's more likely closer to the original "red token, green token" paper: https://arxiv.org/abs/2301.10226

i.e. take half of your LLMs vocabulary, and upweight its probabilities by ~55% to the other half's ~45%, and scan for overuse of this half of all tokens. You can even choose a different half/slice for every individual user, for every individual action. You can implement this under the hood cheaply with logit-biasing.

Terretta 1 hour ago||
Considering how weirdly detuned tokens selections have become in Anthropic's LLM prose in recent models, there is a chance this goes unnoticed in everyday use.
Computer0 14 hours ago|||
I would hate to have any of these rules effecting my output
limbicsystem 2 hours ago||
I see what you did there!
rcxdude 4 hours ago||||
It essentially looks like the difference between two different runs of the model with the same prompt but different seeds. The watermark is essentially a small bias in the model such that when there's multiple different tokens that could conceivably follow the previous token, the model will only pick some subset of them (the subset is derived from a hash of the previous token). This bias can then be checked for statistically (without needing access to the model and without needing the whole prompt), and for longer text where there's enough freedom in word choice you can show that it would be vanishingly improbable to accidentally follow the rules in the watermark.
wolfy1993 14 hours ago||||
IIRC, watermarking text could be as simple as training the model to use specific words/phrases more frequently than what you would expect to find in human-written text, to the point where it's highly statistically improbable that it wasn't AI generated. I assume similar logic could apply to code in the form of functions/code styling.

That's probably an over simplification. Also a solid defence that can be used against complaints about the way AI writes text.

AtHeartEngineer 14 hours ago|||
non visible text is extremely easy to filter with a git hook, a post tool call hook, or just a script. I doubt they are doing that
sixtyj 14 hours ago||
Or grep, in a skill. /clean-cc-watermark just entered the chat…
tech234a 13 hours ago||
Article specifically says "worldwide"
Computer0 10 hours ago||
I agree with your reading, I initially misread it.
colesantiago 14 hours ago||
Good.

They should make it easier, to detect slop so we can ignore it quickly.

I hope Pangram makes an API or an extension to analyze a page to detect slop on a page and then closes the tab immediately.

Nobody should be wasting time on garbage LLM output in code, text, image and videos.

pixl97 13 hours ago|
Panagram is a scam.
colesantiago 13 hours ago||
(This is the part where you provide extensive extraordinary evidence to your claim)
rcxdude 4 hours ago|||
Scam might be too strong a word but it certainly has far higher false positive rates than they are claiming, and their output is at best misleadingly presented: https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...
colesantiago 3 hours ago||
This is on Pangram 3 which is very very old now and the founder responded below

https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

What about on Pangram 4?

https://www.pangram.com/blog/pangram-4-technical

pixl97 13 hours ago||||
No, they are the ones making claims, especially their CEO saying things like a 1/10000 false positive rate. Their own testing showed a 2% rate, which is insanely high when you talk about the number of papers students turn in. Worse their testing methodology compared it with pre-llm documents and not post llm documents that were human written (much harder and more expensive to verify), by treating language as static.
colesantiago 3 hours ago||
You're saying because it has some false positives that Pangram is 100% a scam?

Is their research also a scam too?

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

https://www.pangram.com/blog/pangram-4-technical

If so, what is the best one out there other than Pangram then?

platinumrad 3 hours ago|||
Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.
colesantiago 3 hours ago||
Then what is the best way to know that something is AI generated slop then?
platinumrad 3 hours ago||
The question you're asking has nothing to do with who has the burden of proof when it comes to claims about Pangram, but I'll answer it anyway.

Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up.

You might have to make peace with the fact that there may not always be a tool that does what you want.

colesantiago 3 hours ago||
So Pangram is the best one right now, that all I need to know, and I can safely assume that the Claude AI marks will make it even stronger.

Thanks!

> But the burden of proof is on them...

I mean is this enough proof?

https://www.pangram.com/blog/pangram-4-technical

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

Or is this marketing, a public stunt or not real research?

I think this is enough for me to know they are actually improving their AI slop detector.

beambot 14 hours ago||
Yet another reason to support open-weight alternatives, I guess.
KoolKat23 1 hour ago||
[flagged]
quantumeon 2 hours ago||
[flagged]
pella 7 hours ago||
[dead]
beyondscaletech 2 hours ago|
[flagged]