Top
Best
New

Posted by hazebooth 7 hours ago

Why is this fingerprinting script creating bioweapons?(nullpt.rs)
709 points | 156 commentspage 4
move6729 3 hours ago|
[flagged]
_brucedh 1 hour ago||
[dead]
slipperybeluga 4 hours ago||
[dead]
aussieguy1234 1 hour ago||
tl;dr

Its not creating bioweapons, but has code obfuscation that makes it look to an LLM like it is, to resist LLM reverse engineering of its code by triggering refusals. It even has Tienanmen square references to trip over the chinese models.

xfcgv 4 hours ago||
[flagged]
lol8675309 1 hour ago||
Fuck this sensational title.
void-pointer 4 hours ago|
Outside of the clever semi-benign technical aspects of this: this is really quite sickening to think about.

Sure, We will stop our LLM from decoding obfuscated code by… forcing it to “think” about suicide, self-mutilation, and killing people. This is just philosophically unsettling to me.

I don’t know what software industry I wanted to live in. But it’s certainly not this one.

flexagoon 3 hours ago||
The LLM doesn't "think" about those topics any more than the HN database "thought" about them when you posted your comment
pxmpxm 1 hour ago|||
The copy of excel 2007 that lives on my old windows 7 laptops gets really sad every time it has to do my taxes.
Muromec 14 minutes ago||
You are all dismissing but I specifically bough an LCD display so my thing can display emotions on it as it goes through the stuff
qlte 2 hours ago|||
Those same models have already been subjected to the equivalent of probably millions of hours of RL on simulated user messages containing every possible combination of self-harm, torture, murder plots, genocide, child endangerment, CSAM, etc to test refusals.

"Model welfare" seems like an objectively absurd concept given the inherent training process needed to produce a usable instructions tuned model.

whoanoway 1 hour ago||
> self-harm, torture, murder plots, child endangerment, CSAM

You guys are so cool. I'm, like, missing out by not collaborating with you all :(

Edit: who tf upvoted me? Let me keep my downvotes.

josh-wrale 4 hours ago|||
+1.. Can we not get something more like: https://en.wikipedia.org/wiki/Non-photo_blue
akersten 3 hours ago|||
Well, the very coarse filters that are tripping on this are before the model even loads the text into VRAM, so hopefully it is less sickening to you to know that it's very likely a "if str.contains(`suicide`) { refuseAndSuggestHotline(); }" is thinking about your message on the CPU instead of a chain of `fuzzyWeightAddMulX` commands on the GPU.
aesthesia 2 hours ago|||
If things were implemented that way, you would see vastly more complaints about overeager refusals. For at least some deployments, the safety classifiers are based on internal LLM activations, so the input does get processed by the model.
prynpo 3 hours ago|||
A person still had to think about suicide and self-mutilation to make this anti-LLM script, though.
akersten 3 hours ago||
I'm not sure what the point being made is. No one is forcing anyone to come up with anti-LLM scripts...?
prynpo 2 hours ago|||
The point is that something like this exists, or had to have existed. What lengths a person took to prevent de-obfuscation of code. The commenter above is saying it's sickening that a LLM has to "think" about this in order to not analyze the code, and I think it's equally sickening that, if someone wants to achieve said goal, they must think about stuff like that
comboy 3 hours ago|||
Are you sure?
whoanoway 4 hours ago||
[dead]