Posted by ropbear 18 hours ago
If this didn't have a detectable effect on the quality of the token probability calculation, the watermark wouldn't be detectable. It may be a small degradation in the quality of the output relative to the neds of many users in many situations, but it's not zero. It's literally sometimes choosing different words that it otherwise would specifically for watermarking purposes.
If this was not the case, the encryption would be broken, and most everyone agrees that good encryption does exist.
The "quality of the probability calculations" as you put it is 100% in this case and any less would be a huge deal (as in - breaks all of the internet).
So, now you just take those same random bytes and use them as the seed for your LLM token choices. The output has the _cryptographically_ proven exact same quality as if you were using a true RNG (which you likely weren't using anyways).
You just need to know your LLM distribution and the encryption key, then with each new token you exponentially increase the chance of knowing whether it fits your encryption key. Without actually affecting the token choice in a perceivable manner.
> choosing different words that it otherwise would
The "that is otherwise would" is carrying all the weight here. "Otherwise" is sampling from a distribution. You just sample from the same distribution but with a cryptographically secure, seeded RNG. https://en.wikipedia.org/wiki/Cryptographically_secure_pseud...
"Knowing the LLM distribution" seems to me like the only hard part because you don't know the context of any random snippet.
I struggle to understand the relevance of that comment.
The blue/green token list biasing process literally does cause different tokens to be occasionally chosen. Not only that, but because a different token was chosen at one point, this changes the probabilities of all subsequent tokens, and the resulting later token stream every time it happens. If you had access to the token stream as it would have been, and the watermarked one by the end of the text they will be very noticeably different.
So, you don't own the generated text, and can't use it freely then. What if I copy paste a section, or rewrite a section of text to my liking? What if I rewrite some lines of code that contains the mark?
Security theater, and vague enough to be used as a weapon against who the government wishes.
I hope it's left off for non-EU customers.
I wonder if this is why Opus 5 keeps writing excessively long comments, even though I keep instructing it not to (both in chat, CLAUDE.md, and in its memories)
Everyone who invests in AI companies wants to see the value of their investment increase.
I'd give it about 3-5 years until an AI company claims copyright over code their LLM produces. This is a crucial step in that path.
> Also, what happens if another major global market makes it unlawful for AI to secretly watermark generated text?
> We’re applying watermarking globally at launch because we don't yet have a durable way to scope it by region. However, we will continue to evaluate different approaches, and will share updates when we have them.
So unless they figure it out, would that 'major global market' essentially need to be the US?
Try running an llm like qwen 3.8 27B in Q8 locally with an intentionally very low temperature setting, it will write like a caveman crossed with a robot. You may find that an extremely literal output does not look pleasant to read for humans.
Yeah, that about sums it up!