Posted by htrp 18 hours ago
>distillation attacks are the only vector to keep up
It's demonstrably wrong, they invest in architectural improvements as well, for example, DeepSeek's compressed attention. When you lack compute, you need fast training/fast inference, and distillation alone doesn't solve it. From what I understand, that kind of distillation "attack" (28 mln exchanges) only slightly improves instruction tuning/reasoning traces. If the base model is crap, distilling Claude on a few million exchanges alone won't magically make your model as good as Chinese models currently are (or magically make inference faster on the limited hardware they have). And training the base model needs a proper training run. Serving users at scale needs optimized architectures as well, especially with test-time compute and ever growing context lengths. That's where architectural innovations are happening in Chinese labs when it comes to compute.
It would still be extremely difficult to muster any sympathy for an organization whose MO is to go public not to honestly raise capital to fund growth and development, but rather to dishonestly leave someone else holding the bag, in some cases involuntarily as their retirement funds are passively invested.
And even supposing they were honest and didn't have an IPO, it would still be extraordinarily difficult to care about their misfortune, because "consolidating all thought-work into the hands of those few who can afford frontier models and datacenters and power plants" is also a special kind of misanthropy.
And even if that were not the case, they're filthy rich already, so who gives a shit if the Chinese companies prevent them from becoming quadrillionaires? :)
Is reconstructing the compressed knowledge in the model like reconstructing a lossy JPG or MP3 a reasonable analogy?
Claude will also help you with (mostly good advice) if you ask something like “Research and help me make the most effective plan to train a smaller student model to be better from a teacher model”.
I actually was doing an experiment with a GLM->Gemma E4B for fun, and Claude kept on suggesting I should also add Claude Opus as a teacher lol, suggesting techniques I haven’t heard of like thinking inversion (train a small model to deconstruct summarised thinking into detailed native thinking format of the student).
So I can absolutely see and understand the concern around Fable’s frontier LLM development mitigations, but their approach of silently degrading is completely wrong and dangerous.
AI classifiers, like all AI, can make mistakes, and it’d only be a matter of time before it mis-fires and silently sabotaging a university’s HPC cluster for physics simulations or something because the shape looks like DeepSeek or whatnot to a dumb fast classifier.
Back in the day, an "attack" was supposed to mean be someone acquiring our assets without paying for them or without having our consent. But none of this seems to have happened in this case.
We built a product without paying for most of the raw material we have used, and we don't call that as an "attack". Did we change the meaning of "attack"?
> you would NEVER distill a model..