And those people are buying $10k laptops to run local AI?

If the security of your home possessions is an issue, it should be addressed first before you spend thousands on a very stealable laptop that you need to carry everywhere.

GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.

Can you explain what data is going where?

Exactly. Get yourself a DGX spark and plug it in at home.

I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.

Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.

What part of what I said implied data was going to the cloud?

I have a laptop for work. 95% of the time i'd prefer a tower or at least a mini PC with proper cooling, but that remaining 5% i have to be around with the laptop is enough. I don't like the idea of having to remember to sync the projects every time (especially while i'm in the middle of testing something new so it's not like i can just pull the latest commit or sync the latest good version of something)

To be absolutely crystal clear:

I am not saying don't use a laptop.

I am not saying you have to use cloud inference.

I am saying using a laptop for inference is dumb. If you want private inference:

- use a dgx spark or similar plugged in at home

- set up tailscale so you can connect from anywhere

- use your phone as a hotspot on the go

- work on your laptop, using tailscale to connect to your home inference server if you're on the go

This gets you everything except for the few hours you might be on an airplane.

If you do any travel whatsoever a laptop is usually not optional.

Yeah, that was why I felt that, while Taalas might not be the right thing in general, it could be really appealing in a laptop, that would otherwise not be able to run any big model at all.

I think I misunderstood what the technology could do a little bit, because I imagined these fixed matrix multiplies encoded into hardware as small-ish, as using routing space etc., not just fast.

I think what I really imagined was useful would be more easily achieved by having a big ROM with model weights connected to an ordinary GPU.