Posted by dr_pardee 11 hours ago
For you, yes, prompt kiddie rooting your own device is legal. In fact, it's one of the only things I actually want AI to do, because breaking DRM is a bullshit job[0] and shouldn't exist. AI deals in bullshit, so it's very poetic to use AI to destroy its own bullshit. However, from the point of view of the model provider, there are very specific legal risks to letting someone vibe code their own jailbreaks, especially if a model is already cloud-hosted and heavily regulated. Allowing hacking on your own devices could be construed as trafficking in circumvention tools, so offering that capability to randos opens Anthropic up to another billion-dollar lawsuit.
I could see this being another thing that gets put behind Trusted Access programs. Corellium was able to get away with offering cloud-hosted virtual iOS devices, using an OS they don't own, because DMCA 1201 has an explicit carveout for security research. But "make my device stop doing this thing I don't want" isn't security research, so a lot of prompt kiddie jailbreak uses become legally fraught again.
[0] In the same Graeberian sense that all military officials are staffing bullshit jobs - it is a job that exists solely to undo some other job.
By chance, would you mind sharing your prompts?
It is possible, but is way more involved. You need to get cyber verification for either of them, and it's a little easier to get with OpenAI. Afterwards you can do such work.
I have valid cyber verification with Claude (they approved it super fast, in ~2 hours after applying)
It still blocks and stops pretty much all the time because of rail guards. Specially since the release of Opus 5. I do believe when Anthropic asks during the verification process "what will you use this for", that they somehow use that info during the chat to decide whether to block or not the request.
So you might be able to do one thing in cyber, but not another one. I seem to be able to research and reverse binaries with Claude, most of the time, and if I phrase my questions in certain ways. However, any kind of code developing that could be tangentially related to malware is blocked, for me.
I am seriously considering switching to GLM or another Chinese vendor, even after being cyber verified on Claude. The routine blocks I face on the tasks that I applied to the program (reverse engineering, exploit dev) are enough to make me think its better to move ship.
Better to opt for an open source model that can do most of the work but obviously its not going to be as good.
It is a possibility we are aware of, also given other instances of the fight of totalitarian or perverse or counterdignified drives of all colors against tools.
But the real fundamental risk I see is that of forgetting the principles of ownership, when circumventions become more possible (like in this case). For example, if cars started behaving insanely and unofficial patches will become available, that would soften the need for a principle "my car must behave seriously: my car must not have advertising modules" etc. and "I must not need to patch my car because of the manufacturer's malicious and vile practices".
would be amusing to watch it happen while the second coming of the austrian painter is still in power. the media who fearmongered with sci-fi skynet tropes for the past 3 years will have no choice but to condemn the move.
Seems mistakes cannot be made.
On another note (that I've been consdiering), perhaps, the most efficient way to get from A to B is not always the best route to take...