Posted by pella 18 hours ago
Hopefully they will drop it all together and focus on making models that are useful for everyone like their original mission was instead of playing games with politics.
What safety evaluation? What safety hardening? They already evaluated it and found it to be highly capable at exploiting security vulnerabilities. So we know it is not "safe", and they don't seem to plan to do anything against it. What could be more dangerous than hacking? Biological weapons research? I don't think Chinese labs are doing anything against this either.
Data and content related to "biological weapons" already exist on the internet, in books, etc. The real issue is access to facilities and tools. There are models that help researchers, but they are not LLMs, rather they are models trained specifically on biological data (like AlphaFold).
Cybersecurity is basically used like a dog whistle pioneered by Anthropic to achieve regulatory capture. Otherwise, the widespread availability of good tooling for security analysis would eliminate more of these cyber threats, rather than gatekeeping them for a few private companies.
I understand you need to verify the goal is achievable. But if the judge agent has the same goal as the training agent (solve), and both are of the same model, then aren't the judge and the training agent doing the exact same thing? What is the point then? Can someone explain this to me.