Top
Best
New

Posted by tosh 6 hours ago

Our approach to EU text provenance rules(openai.com)
57 points | 42 comments
LudwigNagasena 19 minutes ago|
It’s obvious that such boneheaded methods don’t work. So what’s next? First, we need to deal with basic word substitution, which seems theoretically feasible. Then we need to deal with encodings such as Caesar cipher, replacing spaces with zero-width spaces, Base64, etc. Your account will be flagged and reported for outputting obfuscated text. What’s next after that? Ooops, you output too many vim commands instead of outputting text directly, your account is flagged and reported to Europol. You think you can bypass that with Deepseek? No, it will be banned alongside VPN.
socketcluster 13 minutes ago||
This approach feels wrong. For code, it's obvious now that Claude is adding watermarks through comments because they are way too long. I keep asking Claude to remove and reduce its comments.

Also, I don't think it's useful because AI involvement in the work is a spectrum, not a binary true or false. If I ask Claude to go over my code and fix up typos and clean up and it starts adding comments with hidden watermarks everywhere it makes the tiniest adjustment, it's essentially appropriating my code. Its contribution may be trivial and superficial.

When I use AI on my own projects, the code it generates basically looks like auto-complete to me; exactly what I would have written by hand. So I don't see why it needs to be marked as AI. It's just saving me time. It's very much my own work.

aucisson_masque 9 minutes ago|
Beside the added comments, why does it matter that everyone know you used ai to write your code ? You’re not ashamed of that, so let everyone know.
Aerroon 15 minutes ago||
Could this technique be theoretically used to track users themselves? It would be quite ironic if the EU forced tech companies to implement extra tracking, wouldn't it?
aucisson_masque 6 minutes ago|
I don’t see how you could implement individual fingerprinting for millions of users, and then make it recognizable. The added processing cost would be insane.

Beside there is nothing in eu réglementation suggesting that.

mgax 4 hours ago||
This is such a waste of time. If someone wants to bypass this it will be rather simple. Just change the words. If someone wants to avoid fingerprinting they will. Can’t we just focus on building rather than spending brainpower on these ridiculous sidequests
gonzalohm 3 hours ago||
Focus on building what? A future in which only big companies get credit for their work?
bsoqk 3 hours ago||
Isn't that what this watermarking allows for? To allow LLM companies to detect that certain text was authored by their tools?
SpicyLemonZest 3 hours ago|||
There's lots of regulations that are easy to bypass. It's absolutely trivial to, say, pull 30 amps from a home circuit rated for only 15 amps. But most people will just trip the fuse because they don't know what they're doing, and the existence of the regulation makes it easy to prove ill intent for anyone who does know what they're doing but causes harm anyway.
bsoqk 3 hours ago|||
The way I interpret your comment is that, in the future, it will be enough to prove that someone tried to remove an LLM watermark to put them in jail.
IsTom 2 hours ago|||
Imagine that somebody sold you text that they mislead you to think is not generated by AI and then trying to convince court that they didn't mislead you intentionally. If they've removed the watermark it stops this kind of defense.
bsoqk 2 hours ago|||
Why not have our devices generate watermarks for everything? That way, we won’t have any kind of defence.
okanat 2 hours ago||
That's what is going on with images taken. iPhone and other cameras now have a per device cryptographic signature. https://c2pa.org/

Soon images taken without this signature will be unpresentable to the court.

braiamp 2 hours ago|||
That's already on the books... misrepresentation of the product sold... or does the EU doesn't have that?
IsTom 1 hour ago||
Generally doing something intentionally or unintentionally might have different penalties.
SpicyLemonZest 2 hours ago||||
These are corporate regulations, it's not really about putting individuals in jail. I do expect that there could be companies where trying to remove an LLM watermark is a fireable offense, especially in the EU where many kinds of decisions must be made by an accountable human being and may not be delegated to an AI system.
someonebaggy 2 hours ago|||
If someone causes financial loss by posting LLM text and is then found to have removed a watermark. Everything in civil law is implicitly inside a if(someone harmed && they sue you) {} block.
akersten 3 hours ago|||
Of course, the implication by comparison to the building code example, that text absent some homeopathic suggestion of provenance causes harm, is absurd at best.

Sent from my iPhone (harm prevention watermark)

Analemma_ 4 hours ago|||
I think some EU regulations have merit and some don't, but going "can't we just focus on building instead of spending brainpower on following the law" is not going to win you much affection. There might be a connection between this pervasive attitude in tech and why now every proposed datacenter construction project is being met with ferocious public opposition.
iamnothere 3 hours ago||
This isn’t just a “techbro” attitude. Although they disagree on the specific problem laws in question, basically every political group (at least in the US) takes issue with the law as written. And most agree (72% as polled in 2023) that the law is written to favor the interests of the wealthy. The pushback against data centers isn’t in contrast to this—data centers provide the bulk of their benefits to wealthy investors, and many localities are permitting them against the will of local citizens.

You might also want to consider that many believe the current AI regulation push is a trojan horse for regulatory capture and subsequent domination of the industry by large, well-connected players who are hostile to privacy and individual freedom.

So when you see US people on a US site complaining about a law that affects US industry, especially scrappy startups, consider if maybe there’s more to it.

andriamanitra 4 minutes ago||
1% false positive rate is completely unacceptable, and if you can bypass it by changing some of the words what's even the point? This is only going to catch low effort slop.
m-hodges 4 hours ago||
> Starting today, API customers globally will be able to opt in to text watermarking for select models. Text watermarking will remain off by default in the API.

> Over the coming weeks, we will add an invisible watermark to eligible ChatGPT and Codex text output in the European Union.

GardenLetter27 17 minutes ago||
I wish we could vote out the EU!
athrowaway3z 3 hours ago||
>> Editing can weaken the watermark. In an evaluation of 400-token passages, replacing 10% of words with synonyms reduced detection from about 92% to 66%. Replacing 25% of words reduced it to 17%.

> Claude/codex/deepseek, please replace 25% of words with synonyms or slight rephrasing because i dont like the current version.

Not sure if that counts as: `a solution that's robust against "common alterations and adversarial attacks"`. Is there a sort of adversarial attack that is more common?

yorwba 3 hours ago|
If you use a watermarking model to do the synonym replacement, it will rephrase it in a way that is compatible with the watermark...
smokel 3 hours ago||
Why use watermarking, and not simply add a signature?
Gigachad 34 minutes ago||
Because the feature is to combat deception. The person generating the text is malicious in this scenario. They will just not include a signature.
SpicyLemonZest 3 hours ago||
The guidelines (https://digital-strategy.ec.europa.eu/en/library/guidelines-...) require a solution that's robust against "common alterations and adversarial attacks".
k__ 3 hours ago|
Is watermarking part of a model architecture or is it something added by the inference engine?
someonebaggy 2 hours ago|
Added during sampling by adding a bias to token probabilities in places where several tokens are equally likely.
yorwba 2 hours ago||
Reading their technical report https://cdn.openai.com/pdf/e9508624-d767-41b6-a26d-e34ca798a... it's a bit more complicated than that, but yes, the watermark is added during sampling.
More comments...