Does anyone have a https://tenstorrent.com/hardware/cards to try it on?

All this performance at such small model sizes, why are the API fees so high for the AI monopolists on this side of the world?

Really excited to see what people do with this. 3.7 27B was probably the best compromise between size and intelligence to run on consumer hardware

I wish this can run directly on my RTX 4090, seems like 30B is the sweet spot for dense model to run locally, sadly RTX 5090 is very expensive and I need a new PC and new power supply(and UPS) to run that, adding a second RTX 4090 is another option, but not sure if my PC can do that yet.

Looks like a pretty significant improvement on the DeepSWE benchmark compared to the previous 27B model.

Wow. Speed improved as well. 200t/s on a RTX 5090!

https://x.com/sgl_project/status/2088281320422322413

q4km is about 48 tps on a 4090. my llama.cpp params are --flash-attn on --parallel 1 --load-mode mmap

Need to upgrade to a second 3090! Slowly building up my local models with Krea2, MiniMax H3 (and their new Music3), and now Qwen 3.8