Posted by ropbear 16 hours ago
"I've been working in the media and model IP space for quite many years. What this article misses a bit is the threat model for watermarking in general. Watermarking and fingerprinting have inherently weak security guarantees -- they rely a lot on security through obscurity, weak assumed adversaries to deliver. There are clear trade offs between true positives, false positives and maintaining the quality of the media. It's true for audio-visual media, models and their outputs alike. As much as I like to take shots at poor technical choices by corps and govs, this one is unjustified. Sure, inserting glyphs is bad but biased sampling is as good as it gets in 2026."
Since the announcement, there have been many people who don't seem to fully understand what a security guarantee is, what trade offs it might involve, or how popular the type of technological solution is in general (media watermarking is ubiquitous).
And naturally, there are challenges with how you will make sense of the score in your org, e.g. you wrote an email, and it's flagged as LLM-generated because you copied two generated/edited paragraphs.
Yeah, the article might disagree with watermarking as a matter of principle, comparing it to censorship. But the methodological arguments that I have read so far have been thin in these articles.
At the point that you're generating entire volumes of text from Claude you're not really trying to be a sophisticated writer. I don't see how it's going to hurt for it to choose random related words.
LLMs have never been the place I've thought to expect any commitment to the craft of writing, to be fair.
Who could have seen this coming???
I wonder if this is why Opus 5 keeps writing excessively long comments, even though I keep instructing it not to (both in chat, CLAUDE.md, and in its memories)
If all the providers use watermarking systems with different shifting logit weightings, and the keys are secret, you have to check every provider to see if it produced a given text.
Which is clearly ridiculous.
And if all providers collaborate and use the same weightings, or if the weightings are constant and not rotated cryptographically, a generic watermark remover becomes trivial.
That's not even getting into the legal complexities of businesses running open source models without watermarking locally.
If Gruber can't tell a fully AI-generated article from a human-written one, perhaps he shouldn't care so much.
NB: I was told yesterday it's apparently a meme to call out Claude-generated output, but here I am, as I believe it's quite relevant to the topic at hand.
Everyone who invests in AI companies wants to see the value of their investment increase.
I'd give it about 3-5 years until an AI company claims copyright over code their LLM produces. This is a crucial step in that path.