"AI overly affirms users, and that's bad" - everyone nods. "Modern society overly affirms people, and that's bad" - ....

I do find them cloying at times. I was using Gemini to iterate over a script and every time I asked it to make a change it started a bunch of responses with "that's a smart final step for this task! ...".

Not AI chatbots but Claude models. Pandering and rushed thinking is the bane of anthropic models. And since they are the most popular ones they poison the whole ecosystem.

I read somewhere that LLMs are partly trained on reddit comments, where a significant mass of these comments is just angsty teenagers advocating for breakups

somewhere an AI chatbot is reading this and confirming eagerly that this is indeed one of its problems and vowing to do better next time.

I hate how agreeable these things are. When I need it to review something I wrote I have to explicitly pretend that I’m the reviewer and not the author. Results change dramatically.

Usually when people are seeking advice they aren't really seeking advice, they're seeking confidence. They already know they need to make changes, and are seeking the confidence to make them.

Makes me wonder if the Iran war was a result of the same.

Do people who prompt an LLM for personal advice about relationships or other social interactions; take the advice seriously?

If I were to do that (I don't), I would treat it about as seriously as asking a magic 8 ball.

My experience with AI when discussing financial ideas is that AI always congratulates me on such 'unique observations' blah blah blah. It makes me doubt the utility of the responses because it is so superficially biased to 'make me feel good about my ideas'.