Posted by Philpax 18 hours ago
However, the chart comparing the score-by-parameters with existing models is misleading, because the others are all MoE, and this model is not. The article describes it as an architectural decision to optimize against NVIDIA cards(which sounds reasonable, as US labs are relatively easy to gain access to these high-performance chips) but the chart should've been adjusted to reflect the difference, say, by using the heuristic that `effective param size ≈ sqrt(total param size * activiated param size` on MoEs.
I think we will find the market is going to move into omni multi-modal models: text, moe, agentic tasks, voice, images, and video all in one LLM.
I cannot evaluate how valuable this model is based on the performance table alone because the comparison is kind of narrow. Waiting for third-party benchmarks to provide more information.
I do not have the hardware to run it properly without super-heavy quantization (about q2). Besides, the architecture is not optimized for local usage (sliding window attention also requires KV cache). I think I will just move on.
I am looking at the HF model page https://huggingface.co/ricursiveAI/Ripple-65B-Preview and it looks like I need to wait a day for an Apple Silicon version.