I assume Nativ doesn't support SSD streaming like DwarfStar.

There is a Gemma 4 model with 12B parameters which might be worth trying. e.g. https://huggingface.co/mlx-community/gemma-4-12B-it-qat-4bit

That said, your computer will still get hot!

If you go small enough it should be no problem. For example Gemma 4 E4B in Q6 or Q4 quantization should run well on your laptop. It shouldn't be too taxing, but would still want to eat 7-9 GB of VRAM or so.

Now that model is mostly useful for writing or chatting.

You need more RAM, plus the models take up a lot of space. 32gb min but I’d recommend 48/64gb, you won’t get close to frontier but it’s still fun to play with, images are very good

heating up is normal, that cannot be avoided. it should become laggy, but you just have very little RAM (I assume 16 GB?) So most models are too big with other stuff running.

Tbh, the only reason to run a model locally is when you want to be completely safe.

From productivity point of view, it doesn’t make sense to have any notebook running a local LLM.

We have one life. We should spend it wisely.

That phrase itself is such an astounding performative contradiction [0].

0: https://en.wikipedia.org/wiki/Performative_contradiction

Something nobody needs is pointless hot air like "Everything you need. Nothing you don't."

You’d be surprised how hard this actually is. I spent 3 days iterating on a marketing site, where I had very explicit / “well written” copy, and it would just repeatedly rewrite it back to the most awful slop. Over and over again! Ended up adding various AGENTS rules telling it to leave the copy alone

I don’t think anyone ever looked at result. Website if full of overflow bugs. It’s pure slop.

There is also https://github.com/ml-explore/mlx-swift-lm which is what https://jan.ai uses.

The webpage isn't the point. The software the webpage refers to is the point.

Agreed. Just based on this not being Ollama, so I will give it a try.

my thing is kind of an Ollama competitor (surrogate?) too. more for prose/text planning, not so much for coding, at least the harness, but i'm sure someone could set it up to do that: github.com/0gsd/enough