Posted by jonotime 2 days ago
I've benchmarked, rigorously, deepseek-v4-flash for programming and personal use, and it is definitely less smart than Qwen3.8-flash-next (which in turn, is not terribly smart).
Local models are also really slow, unless one spends insane amounts of money.
Having said that, Qwen3.8-flash-next is an impressive evolution; it reaches the small versions of the frontier models (like Sonnet) - but again, it's massively slower and not 100% reliable (including: stability).
> if one looks at the CoT, it's evident that it's way way stupider than frontier models
Frontier models don't show the full CoT
> I've benchmarked, rigorously, deepseek-v4-flash for programming and personal use
You've measured something, but I'm not convinced you've measured what matters, because that's a lot harder than people give it credit for.
> You've measured something, but I'm not convinced you've measured what matters, because that's a lot harder than people give it credit for.
"What matters" is what matters to you, right? Who, by the way, don't know what "something" is.
Anyway, if you're so sure that DS performs as good as other frontier models, you're entitled to your opinion. For me that's just having low standards.
That's right, what matters to me is what matters to me, and the something you've measured I don't know - but that's not a point in your favor.
The worst sin a model can commit in my opinion, is to give an excellent dazzling response to a slightly different assignment than the one you gave it. DeepSeek seems really good at NOT doing this.
But if you ask the model what it expects to be asked, of course you won't have that problem. It could of course be that DS commits this sin, but just happens to expect the tasks I give it.
But I rather think that it's Claude which is good at expecting your tasks - because I have seen all your "high standards" models commit this sin.
Can't you just say "shrank to 1/437th the size"? It's not that hard.
P.S. That's not to mean there aren't dangers regarding AI. I just don't trust the people making money from selling AI to manage those risks ethically instead of "protecting" us from those risks like pimps but with suits and good manners.