It’s scary the number of security tokens that end up being ingested by these models.

TIL it actually sends the traces. I had assumed this is entirely server side

I really like this website

So to make the APIs stateless (the "ideal" where they don't use server side sessions/etc) we ended up with this. I'm sorry but this is kind of hilarious. Given the salaries paid to the workers at these companies and the hype of the models, I can't believe they all fell to the same flaw.

OpenAI and Anthropic will probably now resort to save this server side, instead of relying on encription to be able to keep state on the client.

I can’t believe they don’t validate a decrypted signature belongs to the user or use a unique encryption key per user/session.

The problem with this kind of excellent work is that the response to it is always to say "Fuck the user".

For example, when there was a paper that came out showing that having model logprobs makes distillation an order of magnitude easier, the closed LLM providers instantly yanked out support for getting the full logprobs at every time step. You get at most top 10 candidates now and I'm sure even that's on the chopping block.

People will use this to argue that a model which has exceeded Opus 4.8 (Kimi K3) somehow got most of its performance through distillation of Opus 4.8.

I still don't buy that distillation was worth more than 3 months of "catch up" time for the chinese labs. Most people who use the word "distillation" to much are revealing their sinophobia.

This is beautiful work, congrats

Can someone tell us how they were able to decrypt the encrypted payload? The article says they inserted the cyphertext into a session with a different model. Ok, but how does that allow you to decrypt it?

The fact that frontier LLM providers pirated all the data that they used for training, then to go on to encrypt all of the reasoning traces that they use to come up with the conclusions it's really disingenuous, and then have the balls to say distillation is some kind of bad behavior. they are the kings of distillation.

The hiding of this data only brings distrust to their frontier models. I think most people want to understand how something comes to a conclusion they don't want have that part left out on purpose...

it's this kind of behavior that forces people move to to open source models in the end, it's the lack of trust. the frontier model providers treat the end user/customer as a threat or adversary. Fable 5 is notorious for this. a lot of the serious questions you ask the model they won't even respond to you because of the woke guardrails. it wasn't only a couple weeks ago that huggingface had to use glm 5.2 to get the right answers about their security incident because Fable 5 didn't want to answer it.