Posted by vednig 2 days ago
Edit: relevant Scott Alexander article from today
https://www.astralcodexten.com/p/my-ai-opinions
> In terms of bioweapons, I expect that closed-source AIs will be heavily optimized against helping with these, and open-source AI will be banned after the first warning shot (or become economically prohibitive even before then).
(For example, I suspect that plenty of folks would view the recently threatened mass scan of the DN42 hobby network as an instance of misaligned agentic behavior that would have wasted non-trivial resources, and I also think that most observers would pin the specific behavior of that AI on a proprietary SOTA model, not an open one. That's clearly not a disaster-level event, but it should scare you if you're concerned about alignment.)
but ok, who is going to initiate such a treaty? US? the orange man won't, and even if he did, no one would care. by the time his term is over and the next AIPAC spokesperson is elected, it will be even more late than it is now. EU? impotent and irrelevant. China? lmao.