The problem is, we have to take the word of companies for our privacy.

Wrong. Excessive data collection should be banned.

From reading the comments I'm getting vibes similar to Altered Carbon's[0] AI hotels that no one uses.

The opposite of "if you build it they will come".

(The difference being the AIs in the book were incredibly needy, wanting too much to please the customer to the point of annoyance, which is a heavy contrast against the current reality of the AI working to appease the parent organisation)

[0]: https://en.m.wikipedia.org/wiki/Altered_Carbon

Cool, but they're shoving AI into their products and trying to profit from the surveillance etc. that went into building that technology so this just comes across as virtue signaling.

Lets not forget that Gabereial Weinburg is a two faced ghoul or wolf in sheep clothing. He has literally said he does not believe people need privacy yet that supposedly is the duckduckgo's main selling point. He has made all kinds of tracking deals with other companies so duckduckgo "is not tracking you" just their partners are.

Most of the controversial stuff he has done is being whitewashed from the internet and is now hard to find.

we're fooling ourselves if we think there's "still time". AI surveillance is just too powerful and valuable for companies/governments to NOT use it. It's just like saying "ok let's all agree to not increase our power and capabilities". Nobody thinks humanity would collectively agree to that, and for good reason (unfortunately).

I think much of the philosophical discussion on the pertinent issues here have been discussed at length in the context of Legal, Medical, or Financial advice.

In essence, there is a general consensus on the conduct concerting trusted advisors. They should act in the interest of their client. Privacy protections exist to enable individuals to be able to provide their advisors the context required to give good advice without fear of disclosure to others.

I think AI needs recognition as a similarly protected class.

AI actions should be considered to be acting for a Client (or some other specifically defined term to denote who they are advising). Any information shared with the AI by the client should be considered privileged. If the Client shares the information to others, the privilege is lost.

It should be illegal to configure an an AI to deliberately act against the interests of their Client. It should be illegal to configure an AI to claim that their Client is someone other than who it is (it may refuse to disclose, it may not misrepresent). Any information shared with an AI misrepresenting itself as the representative of the Client must have protections against disclosure or evidential use. There should be no penalty to refusing to provide information to an AI that does not disclose who its Client is.

I have a bunch of other principles floating around in my head around AI but those are the ones regarding privacy and being able to communicate candidly with an AI.

Some of the others are along the lines of

It should be disclosed(of the nutritional information type of disclosure) when an AI makes a determination regarding a person. There should be a set of circumstances where, if an AI makes a determination regarding a person, that person is provided with means to contest the determination.

A lot of the ideas would be good practice if they went beyond AI, but are more required in the case of AI because of the potential for mass deployment without oversight.

Contrary to their privacy-washing marketing, DuckDuckGo serves cloaked Bing ads URLs with plenty of tracking parameters. Is that sort of surveillance fine?

https://imgur.com/a/Z4cAJU0

AFAIK there isn't currently any surveillance on search - searching for how to make a bomb does not lead to police at your doorstep the next day. But the police raid after a mass shooting generally brings up plenty of similar searches.

Maybe it could be good to have some integrations between this data and law enforcement to reduce leading to tragedy? Maybe start not with crime but suicide - I think a search result telling you to call this number if you are feeling bad saves far less lives than a feed into social workers potentially could.

Just a thought, and this isn't have a computer sentence someone to prison but providing data to people to in the end make informed decisions to try to prevent tragedy. Privacy is important to a degree but treating it as absolute seems to waste potential to save lives.