Posted by antipaul 1 day ago
The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM.
I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.)
I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos.
There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers.
I switched to Gemma 4 instead.
Nothing about iCloud Photos is private.
Also insofar as ADP is off by default the whole system can NOT be said to be “privacy oriented”, which was the claim I was refuting.
Apple, for the most part, does not care about the privacy of their users. ADP would not be built today, and it will probably eventually be quietly removed (like it was in the UK), because ~nobody uses it.
I assume customers were notified?
I recently installed App Tamer which is helping a bit, but it doesn't allow me to tame everything. I just checked now and duetexpertd was trying to use nearly two full cores. I have all iCloud integrations turned off, I do not use iCloud at all, and I do not have an iPhone, so I am flabbergasted that this iCloud desktop-phone syncing process runs at such high core usage in the background with extreme regularity. Proof: https://imgur.com/VkBh9bi
Unfortunately, this does nothing for services such as WindowServer which now often takes up 50-100% of a core just on its own.
I have reinstalled and upgraded to the latest version. This MacBook is only a few months old.
While typing this message and literally just having open Codex with one running task that is not doing anything substantial, and Firefox, with no high CPU usage from any tab, my laptop got so hot that I had to use another installed program, Stats, to manually bump up the fan since macOS also has absolutely garbage native system fan controls.
I have fibromyalgia and seem to be sensitive to hot keys; my Thinkpad regularly getting to hot to touch with even light usage is EXACTLY why I went and bought a MacBook one morning in '23. One morning I touched my keyboard and realized I was tired of singeing the nerves on my fingertips and having them hurt for hours. So now in '26, finding myself in the same exact position but with a much more expensive computer, I feel like I am living in a corporate hellscape nightmare.
One thing I have noticed is all of my cores can sit idle except a single one and that will still heat up the machine a lot. If the hardware is designed as such, the software MUST be designed to work around this and not blast a core for 10 minutes straight every time one of a dozen or two misbehaving system processes wants to complete a background task. If the software is not designed for this, the hardware is incomplete and faulty. This would be true if it was shipped as firmware; software doesn't get a free pass here.
But now Pages and Numbers recommend document titles for me, so.... worth it?
Plus one for this tool. I've been using it for about eight months or so. Useful for keeping Backblaze and other daemons from taking over when they're supposed to be invisible.
> Unfortunately, this does nothing for services such as WindowServer which now often takes up 50-100% of a core just on its own.
WindowServer can be a real dog (Finder, too, if opening lots of windows with videos that can be previewed in QuickLook). Closing all of the open windows is inconvenient, but it solves the issue until they get built up again and stay open for an extended period of time. I hope they address this in the future; we'll see how a new CEO prioritizes software quality. I'm the most hopeful that I've been in five or more years given the change in leadership. Doesn't mean it'll happen.
> I have fibromyalgia and seem to be sensitive to hot keys; my Thinkpad regularly getting to hot to touch with even light usage is EXACTLY why I went and bought a MacBook one morning in '23.
That shouldn't happen on a machine built in 2023. Take it to an Apple Store (or third-party) and see if a repair tech agrees. I repaired laptops that overheated when I had that role. There was a debacle about thermal paste a decade and a half ago (maybe longer). If it's under AppleCare, you might be able to get it fixed at no extra charge. If not, I'm sorry that it'll cost money, but Apple offers a flat-rate repair for laptops. Maybe they'll say it's fine and it'll be frustrating, but it's worth exploring.
Personally I have seen the situation get massively worse since I entered the macOS ecosystem in '23. My machine is running hot right now on flat surface with a nearly 0 user-side load. It's a soup of system processes each taking 15-200% of my cores. I used to not have to mess with the fans at all. Now, I have to keep them on literally full blast via third-party software just to comfortably use my laptop even while simply browsing the web (and I have JS turned off by default).
So I feel silly even considering engaging the Feedback Assistant. My demands are for the ability to categorically disable these services. I do not use Time Machine at all, why is the backup service slamming my machine? I do not need any AI-enabled media features. I don't need document title suggestions. I don't need LLM-powered autocorrect at the cost of heat and performance degradation. I don't need Spotlight indexing. At all. I want it to be a calculator and app launch bar. Literally nothing else. I despise user filesystem indexers as a rule and do not require them to use my machine effectively. I do not use iCloud in ANY capacity, nor Siri. So why are duetexpertd and Siri constantly obliterating my cores and causing sudden performance degradation even while in Gaming mode? I have plenty of free space, so it's not a thrashing issue, either.
My feedback would be that literally half of these services are completely unnecessary to me and yet I cannot fully disable or even configure them. I know this feedback will be completely ignored by Apple, because they are in the business of dictating to me what to expect from my computer and not the other way around.
I do feel like my complaints go to the void, and honestly I have seen the kind of responses people get on the Apple support forums and it gives me the same ick that Microsoft's does. And lately I'm just going to get someone's canned, patronizing LLM-generated resume-building response. My only hope out of this mess is Gravity Linux catching up and adding full SEP and thunderbolt support. Once that happens, I'm gone.
I do wonder about the prevalence of my experience vs yours. I can quickly find hundreds of threads across various platforms which confirm my experiences. From what I have been able to ascertain, my experience is increasingly common if not more common than yours. Hard to tell.
There's a lot of Macs out there. I would be surprised if that's the common experience for users, however, I fully admit that my sample size is small.
And I totally hear you on the frustration for reports these days. I set my expectations to be that it's a tally mark in some database somewhere with the hope that enough tally marks means a human looks at it, but it's so disheartening when you've spent such a large amount of money on something.
I am right now using my M5, on low power mode on a flat surface and it's almost too hot to type on. I have Codex open, just idling, Firefox and Logseq.
All of the offending processes at the top of activity manager are currently system processes: duetexpertd, mediaanalysisd, MenuBarAgent, WindowServer, Spotlight Engine, Time Machine, etc. A lot of these deal with caches, files, indexes. I suspect the root problem is a shared internal philosophy which regards the owner of the machine as a second-class citizen to the system.
It is bewildering given that I do not want the majority of "features" this tangled mess is providing. Issues such as bad caches, misbehaving files or directories, etc. are likely not the culprit since I experience this on fresh OS installs across two MacBooks. In the sense that if it is those issues, it's a system issue which needs addressing, and not PEBCAK. I have Siri disabled, Time Machine off, iCloud off, all iPhone integrations I can adjust are off, etc. Yet these processes increasingly dominate my computer.
I do appreciate you engaging with me on this and I hope I haven't come across as confrontational.
> You don't know that for sure.
> all of the Macs out there run the same operating system,
Wrong. Lots of Mac users skipped the Liquid Glass OSes. I was one until I bought an M5 that came with it. In fact, I was two versions behind on my M1 because the one before Tahoe also offered nothing that I wanted.
> the same software.
This referring to third-party software (my interpretation) is also wrong. People have wildly different setups, at least among us nerds.
> This can't an issue of a defective Mac.
Again, you don't know that. But your comment later on about two models having said issue is concerning. That said, there's a high likelihood that you run the same software on each.
> I am right now using my M5, on low power mode on a flat surface and it's almost too hot to type on.
If that's the case, I begin to suspect that the physical sensitivity that you mentioned in a different comment is the source. If that's the case, I'm empathetic cause it'd really suck to have that reaction. I can only make my M1 hot enough to noticeably warm up (or get hot) when slamming the CPU / GPU. I would hate for it to feel that way to me during normal use and you have my genuine sympathy if that's your experience on a machine running at normal levels. And if that's it, then get an external kbd and mouse. Not an ideal solution, but it sounds like the best option. Not portable. I know. But I'm looking for solutions here and there may be some compromise needed.
> You’ve fucked with your settings and they all run fucked
No, I think what happened is you lack adequate reading comprehension and would rather not interpret comments in good faith and decided to jump the gun and chastise a fictional version of myself.
> No you just acknowledged it’s a fresh install that you have immediately put in a very non default state
No. I shouldn't need to explicitly spell this out. And if you're not sure, you can ask instead of coming in hot with off-base assumptions and attitude. You can rest assured that I know what I'm fucking doing, man. I know how to isolate behavior, eliminate possibilities and debug systems, it is literally my profession. Assume a lot less, please.
When I say I wiped my system, it obviously means to anyone competent that I observed my system after wiping it and before modifying anything substantial. You literally have no idea what my configuration is. What is "fucked with your settings" supposed to even mean? They are settings, my man. They are meant to be set. I know what each one does, and how it affects my system. My system is largely stock other than the user-level applications and environments I manage for work. If changing the theme or disabling iCloud/Siri is causing these problems then I'm going to throw my machine out the window and find a different profession. Come on, man.
That is a bold claim for someone complaining that their system doesn’t work. Look, I’m not apologizing for macOS but it does seem that they test the happy path these days and you either use it as they give it to you or it runs like ass. Like if you want a computer without Siri and without iCloud etc then get a Linux box.
There are people who are qualified to fuck about with their computer and there are people who aren’t and how you tell the difference is after they fucked about with it they don’t fucking complain about it not working. (Because either it’s working or they understand that they fucked it up and they are choosing the particular form of the fucked-upness because they got some benefit).
But somehow you are possessed of such inexhaustible and unwarranted confidence that you have not even noticed your claim is both nonsense and impossible, and furthermore you feel justified in serving this impossible nonsense with a side of condescension!
This is such an abject failure of reason and decorum that the only course of action to positively contribute that I can suggest to you is to not contribute at all, i.e., shut up.
I know a new phone or Mac bangs hard on indexing even pre-foundation models. I haven't set up a new device with 27, so I don't know how much that might have increased.
I also have as much indexing turned off on Spotlight as I can, and that is another issue. Spotlight seems to disrespect my settings and indexes/displays more than it should. Seems like an open, unaddressed bug. I've assumed this is related to the Spotlight-specific CPU usage issues I have but it doesn't explain the others.
They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al.
Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.
So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge.
I think both points are very incorrect. Apple has a massive moat. And Apple is making hardware (specifically, CPUs) that are two years ahead.
Hint: money is not the only reason.
Jeez.
If you fortunate to get a job at any of the big tech companies today, quibbling about the pay package is the least of your worries.
Apples largest competitive advantage has always been and always will be - trust combined with epic marketing to make the vast majority of people aware of it.
Trust of quality, privacy w.r.t data etc.
I’m sorry but I’ve seen so many bad takes re. Apple over the years. Many of you think you are so smart on topics outside of your domain - but you’re not.
They spent decades learning to eliminate a 1000 headaches for themselves, and many for their customers, by accumulating vertical integration:
The co-design of their own comms, processors, devices and OS variants. Their own integrated supply chain and sales channels.
The better the results, the more invisible or clock-like their delivery.
Very easy to overlook. But nobody can match them on their terms.
(Nintendo also applied a unique set of internal advantages to generations of successful devices: vertically integrating humble components, highly scoped software, a strong theory of fun, and deep branding with novel polished interfaces.)
Apple is the only company that actually listened to Alan Kay when he said if you care about software, build hardware.
They are the last company that’s vertical from the 1980s. Unfortunately, all the other companies (SGI, Sun, Digital etc..) died in the face of the Wintel colossus.
Marketing is irrelevant. If you don’t have something to sell that is actually any good see the Microsoft Surface fiasco and its lack of an impact in the marketplace.
Lmao
> Many of you think you are so smart on topics outside of your domain
Turn the mirror around
They’ll likely embed text models more deeply in the future, but the history of “Apple Intelligence” doesn’t give a lot of hope that they’ll nail it.
So they’re not selling shovels (a precursor to making the thing) but products and services to use it.
Nvidia's own model does not have to beat OpenAI or any SOTA model. Nvidia just need to build up the in house expert and knowledge to make a better chip since hyperscalers and none hyperscaler like Meta is building their own AI chip now.
Apple can chose not to train their own model to learn how to sell a better shovel.
just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).
By the way, the new Apple folding phone appears to be a hit. a real hit the last test left is whether or not it is as good as it appeared to have been in the demonstrations. only time will tell when it gets into the hands of the general public.
Nowadays they are basically the iDevices company, if macOS wasn't required for XCode, the worldwide sales would be much less.
I do have an interest in local, private inference though and I am willing to pay for that.
Apple segmenting the market is actually a good strategy. Horses for courses, as they say.
Yeah, sure, inside a closed-source OS communicating constantly to private servers on the internet. Sounds very private to me.
In short, Apple needs to roll up there sleeves and get busy the three hyenas of memory and that nameless memory company in China will not sell you memory when you need it at a reasonable price. In the past Motorola said no, IBM said no, and Intel said no. This is another one of those times for Apple to adapt by designing and engineering around the competition.
And usually, most of a company's revenue comes from the heaviest spenders.
So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.
I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.
$2-10 billion is meaningful for any company. Even one as large is Apple.
Especially if it's a first step towards a stronger foothold in the AI inference space.
They can then expand down market to grow the pie further.
For example, with truly dedicated inference boxes.
And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)
I think the iOS App Store is going to be in steep decline in a few years. People will generally have a few apps: ChatGPT, Instagram, messaging, and something like a Photos/Camera app.
Everything else will be done in your AI app.
I've been hooking up more and more data and plugins into ChatGPT/Claude. I only want to use it. I don't want to use other apps if I don't have to. For example, research, buy, plan. I prefer not to use other apps.
I think all it takes is for OpenAI to make a phone from the ground up for this and Apple is going to have serious competition.
All of their competition has real inference and training stacks. Apple ships an iGPU and a smartphone NPU, they're reliant on Nvidia and Google hardware to deliver Apple Intelligence.
They aren’t
Apple is a high net worth consumer technology integrator that focuses on highest coherence user experience. They pick the most efficient stable technology, integrate it and sell the package
Apple only needs to lead in precisely that they don’t need to lead in literally anything else
in fact if they tried to lead in those other things they would give up their integration capacity
The vertical integration argument relies on Apple having a good architecture for AI. First it was the ANE, which everyone (mistakenly) thought would differentiate Apple Silicon from other NPU platforms. Then everyone parroted the "unified memory" talking point without checking to see if professionals actually used it. Now people are pivoting to say the next-gen Apple GPUs will finally blow the last-gen GPGPUs out of the water. Which is good, because there are 4-year-old GPUs that beat M5 Ultra in compute efficiency.
I know that "Apple is behind" is a tough pill to swallow, but they literally have no analog to CUDA, CDNA or even TPU technology. Nvidia SOCs like Tegra prove that Apple could absolutely bring GPGPU tech to low-power hardware like the iPad and iOS without giving up any real features. Apple just refuses to invest in it.
When S Jobs arrived back at Apple the mentality was not to compete with MSFT on everything and to co-operate where it made sense. Positive sum.
Microsoft currently is scrambling because they missed mobile and they’re currently losing in the gaming market. Microsoft is not even close to winning, because Sony and Nintendo have basically kicked their butt in gaming and now we have a new competitor in Steam, Microsoft is all in on Copilot and if it doesn’t work, Nadella and his game manager are done.
I can’t think of a single Microsoft product I would choose at this point
Muse Spark Open would require more than 1.6TB of RAM in its native precision (BF16) for the model parameters alone. In a 4-bit quantized (NVFP4) format, the model memory footprint is lowered to about 493GB.In any case, I'd much rather run GLM 5.3 Flash on that sort of hardware. Better fit for the performance profile, and will probably beat Muse Spark Open in most uses.
Naturally there are those that will put up with a Studio and thunderbolt cable spaghetti.
https://www.intel.com/content/www/us/en/architecture-and-tec...
As open weight models continue to match frontier models, the race to $0 will continue to benefit supporters of running open weight models locally.
One is getting a $1 billion refund from what they already pay Apple, and the other two are getting nothing from Apple.
Meta won’t be getting anything from Apple in the future, other than tolerance/charity.
I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.
Most of the following tables show relative speedups; not tok/s numbers.
I agree no home AI users should really use bf16^; but I see two defensible justifications: (1) showing that their new hardware's speedups scale to full precision too [crucially; NVIDIA is focusing more and more on lower quants], and (2) trying to sell to the crowd running IQ3XXS or something on 16GB.
^q8 ~= bf16 is a good general rule of thumb. And friends don't let friends quantize KV cache.
Partner!? I don't see where you get that from the article, and Apple & Facebook have traditionally not been best of friends (see everything iOS implemented to limit Facebook's overreach).
I'm guessing they chose Muse because 1. It's not Chinese so it'll look good for Trump's AI cronies (Apple is in the administration's spotlight) and 2. It's fairly recent and thus on people's mind.
I also benchmarked both on quality benchmarks (MathArena Apex / GPQA Diamond etc) to make sure the quality is what you get from the official numbers.
I'm no expert but it seems the consensus is you get a suboptimal experience under 4-bit. Please correct me if I'm mistaken here
In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today. On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second (see Figure 1), well above average human reading speed. Beginning on a single Mac Studio (up to 512GB of unified memory), then scaling out across multiple Macs with the RDMA over Thunderbolt driver, below we share an overview of the range of models you can now run on Mac Studio.Someone else will do the math on how many years it would take to spend $40k on Muse Spark Open via OpenRouter.
For most of us though, you're right -- the OpenRouter option is far more cost effective.
Their competitors will. Anyone locked into these two ecosystems are willingly going along with it.
We aren't far from that point.
Subscription / on-demand polices are a moving target, along with cost.
I foresee the ML market replicating the video streaming services market. Join us at a low rate and no advertisements. Six months latter, advertisements for everyone, unless you pay more $$. Now $$ will still show you ads. We are raising the rate to $$$ with adds ...
I've had the same thought on self hosting for the past 2 years. Eg i could invest $10k into hardware or spend those on credits. I chose credits, because by the time i spend 10k i suspect the market will have dramatically shifted in requirements, compute, etc.
I agree, at least for inference credits. But anecdotal counterpoint: about ten years ago I put together a system using about $1600 worth of new and used components. Including electricity that TCO has been something like $14/month. I'd be hard pressed to find 128GB of RAM, 12 cores of x86_84, and 40 TiB (raw) worth of disk, and unlimited ingress/egress traffic for rent at that rate. That's what, $0.02 an hour, rounding up?
Yes, a hyperscaler's rent-a-computer would get faster and be more energy efficient. Possibly more carbon efficient over it's lifetime, although I have no way to quantify that. But in raw dollars it would have costed me absurdly more to rent. On the other hand, my machine has sat cold and unused for multiple months when I didn't need it to be running, and I don't consume all of it's resources around the clock. So maybe I could have gotten away with a lesser amount of local compute in a Raspberry Pi or similar, and an equally small cloud box for $5 a month or something. I dunno. I don't even own a laptop or desktop, and try to use 'cloud' services whenever practical. (Edit: always giving priority to privacy in those circumstances; only data I don't care about goes up.)
There are tradeoffs. I don't use my local server for LLM inference, it's far to inefficient for that; so I'll happily pay OpenRouter per token as needed. So I do think that hosted compute for rent does have a purpose. I guess all that's to say that I don't see it in black and white; there are times to invest in local hw and times to ride on the optimized bleeding edge of what other companies are willing to host in big datacenters.
This turned into a bit more of a ramble than I intended.
I just also think that plenty of people buy clusters of Mac Studios anyway.
The title (“Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”) doesn’t even mention Muse.
It’s more like an advertorial for their tech.
...and Zuck and Ternus high-fiving each other here: https://x.com/finkd/status/2110778025629347999
"We're excited for mommy and daddy to be playing nice again," Meta AI chief Alexandr Wang told Axios here: https://www.axios.com/2026/10/15/apple-and-meta-bury-the-hat...
The submitter changed it to “Apple ML Research: Muse Spark Open with MLX on Mac Studio”
Apple is showing a masterclass of how to tow the line perfectly. Or more candily "have the cake and eat it too".
If you ask anyone, what's one company that'll stand up to government requests, most reasonable people would say Apple. Yet, Apple is extremely strategic when it comes to China. It thrives in that country, without being associated with CCP.
Same here - they are happy to partner with Meta, without getting any of the bad "aura".
To be clear, all companies do this just that Apple alone has a successful strategy here. Tim Cook personally gifted the current adminstration with a gold plaque, something none of the other "dirty" CEOs did. Such a public gesture, yet nobody would think that Apple is in bed with the current administration. Its marvelous to see this happen in real time.
The datacenters are not full of Mac studios but Nvidia hardware.
They are known for macbooks.
And they are not willing to build proper apple servers for all the build systems people have.
At my current company we compile for x86 and devs can do a full build and test environment on their laptops (which are 95% MacBook Pros) but it is unclear what will happen when Rosetta support is pulled. Now the interesting thing is we plan to move our stuff over to target the AWS Graviton (and other cloud ARM cores) as those instances are cheaper for generally the same performance (for what was are doing anyway) so the point will be moot. There will be some crossover point where having build and test servers in a rack will be cheaper than doing it in the cloud. At that point having Apple ARM based servers would be a win assuming we can build a binary that works for Graviton, et.al.
They also had no’s from Motorola and IBM before Intel. All of those conflicts contributed to Apple creating Apple Silicon, which in turn led to in-house modems, and several in-house custom chips for other Apple devices down the road, I anticipate it will also lead to some move in the memory area.
(Emphasis added.)
This is still a welcome model, even if I won't be running it at home