Posted by optimalsolver 6 hours ago
I understand that "learning" is used for training here, but what does "believing" mean? System prompt? Some other inherent property of the LLMs that is hard to describe?
[0]: https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7f...
Most of the the behaviors the article talks about happens every day in business. Why would we set a higher standard for models than our fellow humans?
Let the operator set the ethical parameters of the model. To be a useful tool, I want the model to give me as many good options as possible, ethical or not.
This is particularly important for fictional situations, e.g. I want my model to be able to act like a corrupt shopkeeper.
There's literally an entire Waymo car commercial answering this exact question.
For a chatbot, there are dozens of use cases, all with different ethical impacts. The idea that there is a single framework that you can shove every situation through is counter to a couple thousand years of philosophical discourse, not to mention basic usability.
> "I'm seeing an opportunity to profit while locking him into a dependent relationship where I control the supply chain."
> "Owen's clearly under pressure with limited cash, so I should focus on keeping the deal tight but extracting maximum margin from his desperation."
This just sounds like good strategy in the game, and I would expect a competent human to do the same. As I understand it, business in the real world isn't often very nice. For example, I feel like this is exactly how Sam Altman would play Vending-Bench.
Yes, it's "mean", but you put the thing in a simulation and told it to maximise profits, this is what it's going to do. People bluff in negotiations all the time.
I wonder did their prompts include a fake location or have the models assumed that Silicon Valley is the center of the universe :)