“We have information that Moonshot distilled Fable for the development of K3”

Posted by softwaredoug 11 hours ago

“We have information that Moonshot distilled Fable for the development of K3”(twitter.com)

https://xcancel.com/mkratsios47/status/2079933645888880708

217 points | 562 commentspage 6

tanh 8 hours ago|

For code can't they distill from public GitHub commits? If they could figure out who used Mythos/Fable assitance in the commits.

SwellJoe 8 hours ago|

I think they're distilling "reasoning", not merely code. There's plenty of human generated code. What they're trying to extract is the process by which really large models "think" their way through complicated problems. That's what all the "traces" datasets on HuggingFace are about.

kouteiheika 9 hours ago||

Assuming they did then they surely paid for them, which makes it "not stealing". Am I also "stealing proprietary U.S. technology" by harvesting my Claude chats from my `.claude` directory and training a bunch of models on them?

That said, I doubt the "they distilled Fable" is the reason why K3 is as good as it is, considering the timelines involved, and that Anthropic hides thinking traces, and their overly aggressive "safety" filters.

This constant FUD spread by Anthropic is so tiring.

sosodev 9 hours ago||

Model distillation can't be stealing at all if you rationally apply copyright law to it. Anthropic is not deprived of Fable so there is no theft. At best it would be infringement, but even that might not hold up in the courts given the current position that model outputs can't be subject to copyright.

yencabulator 3 hours ago|||

It's at most a Terms of Service violation escalated into political theater because checks notes China bad.

skeledrew 8 hours ago||

> harvesting my Claude chats from my `.claude` directory

Just reminded me to set a backup on that directory. Just in case someone sees it fit to override my setting to preserve my chats for 10k years.

goldenarm 5 hours ago||

I have information that Anthropic distilled the internet for Fable

sscaryterry 5 hours ago||

How much credible, provable evidence? None really.

seydor 5 hours ago|

the bar for credibility in the US administration is in the subatomic scale.

proof is generally not even needed

jchw 6 hours ago||

Is this person trustworthy? I struggle to believe that in the relatively short time Fable was available it has already been distilled so effectively. If this really is actually true, very impressive work.

xinayder 4 hours ago||

I wonder if this was caught with the malware code Anthropic included that detects if you're in China...

exabrial 5 hours ago||

Oh the irony... LLMs go and read a billion pirated book, but now are crying when their models get distilled from a million queries.

seydor 5 hours ago||

Good pretext for bannign chinese APIs in the USA and their vassal states.

If distillation is so good, why aren't US companies distilling each other?

riknos314 7 hours ago|

If it's true that in under 15 days of access significant improvements were realized in K3, then the moat of closed-weight models is far smaller than previously thought.

Doesn't bode well for the valuations of these labs.

More comments...