Top
Best
New

Posted by joshcsimmons 5 hours ago

Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU(github.com)
132 points | 41 commentspage 2
walrus01 2 hours ago|
I very recently spent a lot of time using deepseek v4 flash 0731 before it was superseded by newer more capable things (glm 5.3-flash, v4.1 flash 1017, etc) and I am exceedingly skeptical of anyone that claims to get good QUALITY results out of it at that size. Whatever they're doing to shrink it will make it worse than an unsloth Q2XS quantization of it or equivalent. That size of model needs something like 150GB of RAM to run properly with full unquantized kv cache and not with a ton of training data discarded.

I remain exceedingly skeptical of people who are claiming to run fairly large models in ridiculously tiny sizes. For comparison a 'regular' version of v4 flash 0731 has file sizes more like this: https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF

Even the 97GB Q2XS will show very significant divergence from the full precision thing. Running a lobotomized model just to say you're doing it is really not accomplishing anything special.

spwa4 2 hours ago||
Anyone else finding these AI generated pages really hard to read?
dofm 33 minutes ago|
Everyone else, yes. This one is particularly mindbendingly bad. It is almost not human-readable without LLM summary.
outfinity 3 hours ago||
It is 2 bit? anybody tested this? is any good?
efficax 3 hours ago|
yeah 2 bit quantization, so probably not that great for actual work
junaid0016 50 minutes ago|
[flagged]