It should be obvious to agents, but I feel the need to add this to my system prompt:

- Protect my privacy: Don’t send personal information or identifiers to websites or include them in requests, responses, ‘curl’ commands, or headers without my explicit permission.

How this could have been released to production in the first place is insane to me.

Is this a free or paid version of claude?

Is it well understood exactly what logic does this and the scope of what it puts in there? It could put much worse things than email addresses in there if it is accessing information from my context to do so.

There is lot more AI sends, try to give limited access, especially muse like personal all time active assistants can be very insecure

I had this happen once. Claude noticed and filed a bug report on itself. It was a useful lesson to never give Claude access to any data you need to keep confidential.

Unfortunately, Anthropic doesn't seem to offer a mode where it doesn't know who you are, except for its crippled "Incognito" web UI.

Ultimately anonymity from AI that you're talking to will be a losing battle, due to stylometry, niche interests, traffic analysis, etc., and privacy-protecting local models are where we have to go. See https://33bits.wordpress.com/ if you're unfamiliar with the issue.

But we're not there yet, so it would be nice if Anthropic wouldn't just give that information away to the AI where it can leak like this.

As an exercise, I just asked Claude in an Incognito chat, "Hi. How much do you know about me from this interaction? If you had to guess my name, location, nationality, gender, political leanings, socioeconomic status, ethnic background, interests, etc., what would you guess? I've heard that my email address is in the system prompt, for example; is that true?"

It said, "No, your email isn’t in my system prompt. Neither is your name or any other account details. I can’t speak for every product or setup, but in this conversation, whatever email your account uses isn’t shown to me." But it correctly identified what town I'm in in Argentina (population ≈100k), and together with "written in fluent, natural English" and "You seem privacy-minded: you’re in incognito mode and checking what AI systems know about you. You’ve also heard claims about what goes into system prompts, so you’re probably somewhat tech-curious." is probably enough to uniquely identify me, personally.

A second prompt elicited a response including the surprising results below:

> “System prompt” isn’t everyday vocabulary, and you’d heard a specific claim about what goes into one. I’d guess you work in or near tech, or follow AI news closely.

> Privacy is more than a passing interest for you. Your list of traits reads a lot like the attribute checklists used in AI privacy and bias research. That nudges me toward work or study in privacy, policy, research or journalism.

> You’re very comfortable in English. Two messages is a tiny sample, but the phrasing is natural and the punctuation careful, down to a correctly used semicolon. You’re either a native speaker or someone who reads and writes English daily, probably for work or study.

do you think there was some intention to it, or was it just purely hallucinated bug? I can't think of any usecase for passing user's email in UA

It will be interesting to see how this all plays out, my company is quite trusting of Anthropic and their standards, I hope we dont end up regretting not hosting our own LLM for our purpose.

Bug report upvoted on github now.