Top
Best
New

Posted by quantumgarbage 23 hours ago

Stealing Reasoning Traces from Proprietary LLM APIs(stolen-thoughts.com)
645 points | 288 commentspage 5
throwa356262 20 hours ago|
This is laughable security. People claim security is now "solved" thanks to AI but from where I am standings it looks more like the fun 90ies making a return.

Anyway, can someone explain the part about K3? What are they trying to say?

qrios 20 hours ago||
The interesting part is what they try to not to say: More indications for K3 is based on distillation from Claude and GPT.

From [1]:

> As you might guess, this suggests that distilling reasoning traces may have been possible for a long time without ever breaking the cryptography.

> An anecdote: we find that prefilling Kimi-K3 reasoning with a few tokens of Opus reasoning measurably shifts its response toward Opus’s

> A small memorization analysis showed that specific Claude and GPT reasoning spans are up to ~6 orders of magnitude easier to extract from Kimi-K3 than from the next-closest model.

[1] https://x.com/kotekjedi_ml/status/2087147042888114428?s=42

varenc 10 hours ago|||
The part about K3 is just very strong evidence that K3 is partly a distillation of Opus. Probably even a distillation of Opus's CoT, which means they already had broken CoT encryption themselves awhile ago.
neuroelectron 20 hours ago||
Security is solved, but business needs overrides it
alansaber 22 hours ago||
Neat.
syntaxing 20 hours ago||
Prefilling Kimi K3 with opus is a super interesting idea. That being said, I absolutely hate this website layout
bonoboTP 21 hours ago||
It's not stealing.
retinaros 16 hours ago||
curious seeing how anthropic is agressively fighting this stuff how did you get to experiment on this? did you just try and shown them results or did you need approval first? I am interested mostly because I research on distillation
dylanw2468 2 hours ago||
[flagged]
smeltworks 7 hours ago||
[flagged]
tizerluo 11 hours ago||
[flagged]
lossy_compress 14 hours ago|
[dead]
More comments...