I get the impression they want AI marketing points but don’t actually want people to use local AI on their products.

I have no idea why. They could be so successful if they leaned into local AI.

> Apple's unusually timed announcement of new Mac mini and Mac Studio models this week was driven by unexpectedly strong enterprise appetite for AI hardware, according to The Information.

Obviously; no one else can justify the expense.

Maybe it's not anything specific to Apple? There's high demand and short supply elsewhere due to AI, so it doesn't seem all that odd that many companies would try to buy gear from Apple too.

So for people who don't understand, there are two markets for Apple hardware in this space:

1. Running an agent like OpenClaude. The $599 Mac Mini was an insanely good deal for this. I happened to buy a M5 Pro Mac Mini for $999 last year for other reasons. The equivalent is now almost $2000; and

2. Hardware for running inference on local models. This to me is the far more interesting market because Apple has a real opportunity to disrupt NVidia's stranglehold on the market.

With current architecture, the largest model you can reasonbly run is the amount of memory on the GPU and is a function of the quantization (eg int4, int8, fp8, fp16, etc) available and the number of parameters. NVidia aggressively segments the market. The most VRAM on a "consumer" card is 32GB on the 5090, which allows you to run ~31B parameter models.

In comparison, the RTX 6000 Pro has only slightly more CUDA units than a 5090 but has 80GB of VRAM. A few months ago they were $10-11k. Now they're ~$16k.

Macs use a shared memory architecture. Apple has previously sold Mac Studios with up to 512GB of RAM. Almost all of that memory can be used to hold much larger models without taking a penalty for interconnections between different GPUs or machines. Plus Apple interconnects between computers are actually relatively good by chaining TB5. It's still slow but it's about the best non-enterprise option available.

But the previous Mac Studios just didn't have the raw FLOPS and memory bandwidth. The M5 Ultras are up to 1.2TB/s of memory bandwidth. M3 Ultra had ~900GB/s. RTX 5090s and RTX 6000 Pros are 1.8TB/s. The current best HBM3 NVidia DC GPUs are at 3.2TB/s IIRC. But the M5 Ultra has a claimed ~4.5x the FLOPS of the M3 Ultra.

We don't have our hands on these yet but it probably means they are going to be much closer to a 5090. I expect ~50% of a 5090's inference speed. That may sound bad but it's actually really good because a 256/512GB Mac Studio can probably locally run the best Flash models. With NVidia hardware you'll need to spend many tens of thousands for that.

We'll see what the inference speed is but I expect it to be usable. DeepSeek v4 Flash, for example, will be entirely runnable. We're not at DeepSeek v4 Pro local yet.

Sounds like people want those bespoke servers that Apple has been rumored to have developed.

I wish they sold something that could go in a colo - redundant power supplies, lights out management, etc. you know they have them internally...

Classic monopoly move: Control the user base, then control hardware. Any decent always-on local LLM setup with Apple devices will have to compete with these behemoths now. Great.

This sounds like advertisement, disguised as an "article".

I hope Apple does not gain some exclusive enterprise tier for hardware. Part of what I love about them is that everything is available to consumers. A lowly home user can buy the exact same 256 (or 512) gigabytes of memory in a Mac from Apple, as long as they have a couple dozen thousand dollars to spare. I'd be really sad to lose that.