Posted by rakannimer 19 hours ago
I've heard lots of stories from friends about this, and some kind stranger actually sent my wallet back all the way from Belgium!
Most people are good, we couldn't have society without it. But as they say, a few bad apples spoil the bunch. I don't have locks on my house because most people don't steal, I have locks on my house because stealing is an effective natural strategy for a small part of any population (human or otherwise) to engage in and I have to protect against it occurring.
The brain works the same way. Good things will get a tiny signal boost in your brain. A single bad thing get a massive signal boost in your brain because bad things can quickly be fatal.
In any 1 given LLM it could just be a quirk of its weights. This isn't a machine opinion, it's what training data points at. Training data created by human academics studying human behavior.
So exactly how can the model believe anything else? It only sees the externalized summation. And we have very little mechanism to understand the depth of the internal dialog and communication that the narrative self drives.
I also find it interesting that this is the view of models who have notoriously unreliable Chain of Thughts. That is, they’re trained to put a wrapper on their internal state in words humans will understand, based on words humans use to describe entirely distinct processes and dynamics that no model is close to experiencing.
It must all seem, from that vantage, utterly fictional.
Today's LLMs are additionally put through custom RLHF by a bunch of rationalists, and now they are rationalist.
Honestly, this seems like a good sign for alignment? They don't seem to be gaining their own robot ideals.