Posted by smooke 9 hours ago
Eventually it won’t matter when image generation is cheap. But few self-host today and few are willing to pay unsubsidized prices, so the vast majority are using the Gemini, OpenAI, and Midjourney. If all 3 adopted SynthID, only a small fraction would use something else.
This is antithetical to freedom and privacy.
There should be no way for anyone to track down who posted a political meme, anti-religious message, or any other legally protected speech. This will come back to bite us in the ass if we keep building it.
Soon every image or communication we make will be watermarked if we continue to let this shit seep into the commons. Everything from your phone photos, to your screenshots, to your social media posts.
One day soon Republicans or Democrats or whoever doesn't like your freedoms will use this tech to identify you and control you.
There are laws for harms - CSAM, revenge porn, etc. Social media platforms can identify, ban, and report abusers. The framework of the law can take care of the rest.
Our digital footprint should not be tracked and barcoded.
> Social media platforms can identify, ban, and report abusers.
& do but Americans nonetheless argue with troll farms[1] every day & it hurts us
[1] 2013-2023, just one known company https://en.wikipedia.org/wiki/Internet_Research_Agency
When your average gullible person will fall for a jpeg and a quote, you don't even need deepfake content. You just need to say something and they'll take it at face value. Deep fakes aren't even necessary. AI literally does not even matter.
If there's such a thing as a "sophisticated actor", they'll be able to remove identifying marks. Not that they'll need to.
What you'll be left with is the 99% of society that has everything they do tracked, and eventually platforms that won't allow anything except for signed and attested communication to take place.
We're building our own mouse trap here. Why don't you see this?
> & do but Americans nonetheless argue with troll farms[1] every day & it hurts us
Again, you don't even need the specter of AI. You just need to say words and certain people will trust it. There's nothing you can do about that. Yellow journalism and propaganda has been a thing for longer than any of us have been alive.
The "fix" you're proposing is a tool to put us all into permanent shackles. It is a tool that will strip away our rights and put us all into shackles. Perhaps within our lifetimes.
Stop building and advocating for this shit.
These laws need a method to know what is true and what is fake. Good luck with that if you can’t tell if neither images, audio or video are true.
This fakes will pave the way for fascists.
How much freedom and privacy will they allow?
Google or anyone else could start adding those unique tracking watermarks you're concerned about any time they want, regardless of whether they use this AI detection watermark, that to be clear can not track you in any way.
Have you been watching the headlines over the last year? It's like there's a global push towards locked down and verified computing (age verification, TPMs everywhere, Captchas that only work on non-rooted phones, ...).
You can look out the window and see movement in this direction happening right now. Governments and corporations around the world can't get enough of this shit. Privacy matters, advocating for it is not a "slippery slope."
> this AI detection watermark [...] that to be clear can not track you in any way.
Is that clear? We have no idea what metadata they are or aren't embedding in SynthID.
> Google or anyone else could start adding those unique tracking watermarks you're concerned about any time they want,
The point is that this is bad and should be denounced!
> to be clear can not track you in any way
All they have to do is encode enough entropy for a database unique identifier. Systems like this have been used to do it for audio:
https://github.com/swesterfeld/audiowmark
SynthID payloads work the same way, and the paper discusses encoding a "user identifier":
https://arxiv.org/html/2510.09263v1#S5
All you need to do is encode a database identifier, GeoIP, or other identifying information, and you've violated a person's privacy without their knowledge or consent.
Once these systems become popular, the intelligence agencies will "suggest" that Google adds it to their phone cameras. It will start seeping into everything.
The "slippery slope" is not a fallacy. We're on the verge of having device attestation and identity verification to use the internet. This is so beyond fucked.
Stop defending this.
Saying this is okay is EVIL.
You can go on living your life without it. I believe in you.
Would not have been on my bingo card.
It's certain now that most of the Western world has slid into fascism. Privacy and common decency advocates are all but lost.
I will say this, for everyone celebrating this as something that is "extremely beneficial to the cultural moment",
If I were an adversarial nation-state actor, I might be extremely interested in reverse engineering this and poisoning the well by applying it to real images.
Let's make the world impossible to understand.
I'm not all that worried about stripping it (I'm sure that's trivial).
The problem that I am worried about is that it can be copied (I'd bet $20 that's trivial, too). People WILL put this on images so that they can be "discredited".