Posted by nateb2022 15 hours ago
OpenAI and Anthropic guard the real reasoning closely. Users have never been able to see it and the API returns an encrypted blob instead of legible reasoning.
Here is an archived version of Anthropic's API docs saying that Sonnet 3.7 (only) has unredacted CoT on API: https://web.archive.org/web/20260324051339/https://platform....
5.5 Sol repors 10x fewer reasoning tokens than Kimi k3. If it is correct, than it unlikely has those doubt issues.
At the same time, I feel like their reporting is incorect and we are now paying per "intelligence", not actual tokens. We can't verify it anyway..
Video will be interesting. It's the most context rich medium combined with sound, movement ect. We need more evals and benchmarks around video understanding. It's the key to unlocking truly great physical intelligence. I've been working a bit on this @camerasearch and its challenging.
I think it's very valuable to have a smaller model that doesn't have any domain knowledge or facts built into its weights, but given the right context, could accurately reason about what to do and use the right tools.
I'm aware of colibri [1], but so far I've only seen extremely slow performance.
You are describing fundamental tradeoffs. Getting more performance relative to model size and training token amount is what all of the labs are solving.
Asking for such a model while wanting to be small and fast would align with what you're describing, which I believe is different from what I'm pointing to.
The model I'm describing sacrifices domain knowledge and expertise for agentic reasoning and tool-calling capabilities at a reasonable speed.
Think of Cactus Compute's Needle [1].
How are you going to use tools, if you don't have the context to use them?
Imaging if you could only think in terms of lambda calculus, and was asked to check the weather to let me know if the weekend is good for a hike.
(If you're claiming that it is impossible, and you have references you can share, then I'm honestly interested.)
I think more of the capabilities comes from the cross domains knowledge.
Anyways, software is also designed for a domain. The reason why it is so adept at making a fitness tracker is likely because it knows about biology.
An alternate theory is that models need lots of input knowledge to learn complex reasoning, but don't need so much at inference time. An example is arithmetic. Early in their training, models do arithmetic (poorly) by pattern-matching on memorized examples. Eventually they grok arithmetic and stop pattern matching, and then they don't use the examples anymore.
The fitness tracker doesn't take much knowledge of biology. Large models have quite a lot of biology knowledge that most people will never use. Same goes for lots of other topics. For basic knowledge that easily fits in context it could search the internet, or a local collection of introductory textbooks.
I’ve had success adapting even a 7B model for single-domain tasks that way, including reasoning and tool calling.
You can use an open model. The point is just to outsource the inference, so you don’t have to deal with running the larger model yourself.
Take the “train” portion of tasks on some benchmark, have K3 complete it, and then output detailed descriptions of tools used and why, then run the validation tasks with some small model that has access to the skills.
We won't be able to run this ourselves, but many providers can.
Hmm, not quite true, I think that honor, for better or worse, goes to OpenAI. When they released GPT2 (or GPT1 for that matter) is was quite literally the SOTA in the ecosystem when it was released.
https://github.com/p-e-w/heretic
As a side note on this, if you see the reference in the screenshot in the link above to the harmful behaviors prompt set, these are all in English:
https://huggingface.co/datasets/mlabonne/harmful_behaviors
You could likely further de-censor a model by having a set of 'test' prompts in native Mandarin, Cantonese or really just about any other language. I don't speak any Chinese languages so I don't know if the published 'heretic' GGUF files some people have been throwing around will cooperate, or refuse, if you ask it in Mandarin for how to build a meth lab or precursors for semtex.
It's a 3T parameters model, with a weight format (MXFP4) still not completely integrated into the ecosystem, which only a few has the hardware to even do inference with, much less fine-tuning or more post-training. But sure, do sit and wait a few days :)
It is more important to focus on the questions than the persons who created it.
https://x.com/dhh/status/2081435006770249831 (from the creator of Ruby on Rails).
In his specific case Kimi did the task it was asked to do (translation of the article DHH wrote), which Claude refused to.
'When gypsies appropriate public spaces, you deport them. It's not hard, it's not cruel. It's the basic logic of self-protection.'
However, even if it did, it is absurd for an AI to decide what it will and won't translate.
> The company appears burdened by far too many disparate groups that are over-optimizing for certain metrics as opposed to delivering usable technology for the company as a whole.
> And because Meta has a reputation for throwing money at problems and executing at high speed, these U-turns end up becoming more costly versus other companies that take a more disciplined or conservative approach. Suppliers also lose faith when given design wins are later cancelled. This has lead to less supply chain prioritization on new designs. Some suppliers favor focusing on Amazon or Google designs due to Meta’s frequent reshuffling.
> Few inside Meta’s chip division have a full understanding of why the company bought Rivos in the first place, and those who championed the deal internally have since gone quiet.
etc etc
It goes into a lot of depth.
https://newsletter.semianalysis.com/p/metas-infrastructure-t...
https://newsletter.semianalysis.com/p/the-future-of-meta-sup...
https://finance.yahoo.com/sectors/technology/articles/mark-z...
Not exactly the best use of dollars and the labor hours of some of the best minds of our generation.
Filtered heavily into those who care only about money. Many don’t want to work there. Smart people have other choices.
Smart? Check.
Money-obsessed and soulless? Check.
Morale and momentum are huge things in companies, Zuck has been murdering both of those in Meta since... well naming it Meta.
is opus 5 a flop like 4.8 ?
where is the thread btw curios
You can still try it at https://chatjimmy.ai/, but it's running the rather outdated Llama 3.1 8B
The alternative is on-TPU flash for storing the weights.