Top
Best
New

Posted by speckx 1 day ago

Apple is fighting for TSMC capacity as Nvidia takes center stage(www.culpium.com)
724 points | 439 commentspage 2
mitjam 23 hours ago|
As a heavy user of OpenAI, Anthropic, and Google AI APIs, I’m increasingly tempted to buy a Mac Studio (M3 Ultra or M4 Pro) as a contingency in case the economics of hosted inference change significantly.
utopiah 21 hours ago||
Don't buy anything physical, benchmark the models you could run on your potential hardware on (neo) cloud provider like HuggingFace. Only if you believe the quality is up to your expectation then do it. The test itself should take you $100 and few hours top.
mitjam 13 hours ago||
This is certainly the best approach.
pram 23 hours ago|||
FWIW the M5 appears to be an actual large leap for LLM inference with the new GPU and Neural Accelerator. So id wait for the Pro/Max before jumping on M3 Ultra.
mitjam 23 hours ago||
Thanks, that helps me keep things in perspective.
mohsen1 23 hours ago|||
the thing is GLM 4.7 is easily doing the work Opus was doing for me but to run it fully you'll need a much bigger hardware than a Mac Studio. $10k buys you a lot of API calls from z.ai or Anthropic. It's just not economically viable to run a good model at home.
zozbot234 23 hours ago|||
You can cluster Mac Studios using Thunderbolt connections and enable RDMA for distributed inference. This will be slower than a single node but is still the best bang-for-the-buck wrt. doing inference on very-large-sized models.
mitjam 23 hours ago|||
True — I think local inference is still far more expensive for my use case due to batching effects and my relatively sporadic, hourly usage. That said, I also didn’t expect hardware prices (RTX 5090, RAM) to rise this quickly.
boredatoms 23 hours ago|||
If theres a market crash, there could be a load cheap H100s hitting ebay
wmf 23 hours ago||
You can't run those at home.
SoKamil 23 hours ago||
Why?
wmf 22 hours ago||
Because they are extremely loud, consume 8-10 kW, and probably cost $20K used.
varenc 15 hours ago|||
Plenty of home runs all electric heating systems. Running inference on a H100 could be dual-purpose and also heat your home! (albeit less efficient than heat pumps, but identically as efficient as resistive heating)
kccqzy 22 hours ago|||
The 8-10kW isn’t a big deal anymore given the prevalence of electric vehicles and charging them at home. A decade ago very few homes have this kind of hookup. Now it’s reasonably common, and if not, electricians wouldn’t bat an eye on installing it.
MandieD 21 hours ago|||
In the winter in northern Europe or the colder parts of North America, as part of a radiator system? Kind of works!

Any other time and place? The power to run it, plus the power to cool it.

alt227 21 hours ago|||
But the cost of running them is.
mifreewil 23 hours ago|||
You'd want to get something like a RTX Pro 6000 (~ $8,500 - $10,000) or at least a RTX 5090 (~$3,000). That's the easiest thing to do or cluster of some lower-end GPUs. Or a DGX Spark (there are some better options by other manufacturers than just Nvidia) (~$3000).
mitjam 23 hours ago||
Yes, I also considered the RTX 6000 Pro Max-Q, but it’s quite expensive and probably only makes sense if I can use it for other workloads as well. Interestingly, its price hasn’t gone up since last summer, here in Germany.
storus 22 hours ago||
I have MacStudio with 512GB RAM, 2x DGX Spark and RTX 6000 Pro WS (planing to buy a few of those in Max-Q version next). I am wondering if we ever see local inference so "cheap" as we see it right now given RAM/SSD price trends.
clusterhacks 21 hours ago|||
Good grief. I'm here cautiously telling my workplace to buy a couple of dgx sparks for dev/prototyping and you have better hardware in hand than my entire org.

What kind of experiments are you doing? Did you try out exo with a dgx doing prefill and the mac doing decode?

I'm also totally interested in hearing what you have learned working with all this gear. Did you buy all this stuff out of pocket to work with?

storus 20 hours ago||
Yeah, Exo was one of the first things to do - MacStudio has a decent throughput at the level of 3080, great for token generation and Sparks have decent compute, either for prefill or for running non-LLM models that need compute (segment anything, stable diffusion etc). RTX 6000 Pro just crushes them all (it's essentially like having 4x3090 in a single GPU). I bought 2 sparks to also play with Nvidia's networking stack and learn their ecosystem though they are a bit of a mixed bag as they don't expose some Blackwell-specific features that make a difference. I bought it all to be able to run local agents (I write AI agents for living) and develop my own ideas fully. Also I was wrapping up grad studies at Stanford so they came handy for some projects there. I bought it all out of pocket but can amortize them in taxes.
mitjam 13 hours ago|||
Building AI agents for a living is what I hope to become able to do, too, I consider myself still in learning phase. I have talked with some potential customers (small orgs, freelancers) and learned that local inference would unlock opportunities that have otherwise hard to tackle compliance barriers.
clusterhacks 18 hours ago|||
Very cool - thanks for the info.

That you are writing AI agents for a living is fascinating to hear. We aren't even really looking at how to use agents internally yet. I think local agents are incredibly off the radar at my org despite some really good additions as supplement resources for internal apps.

What's deployment look like for your agents? You're clearly exploring a lot of different approaches . . .

mitjam 13 hours ago|||
That‘s exactly my fear.
storus 22 hours ago|||
M3 Ultra with DGX Spark is right now what M5 Ultra will be in who knows when. You can just buy those two, connect them together using Exo and have M5 Ultra performance/memory right away. Who knows what M5 Ultra will cost given RAM/SSD price explosion?
mitjam 5 hours ago||
I have researched a bit more and think your recommendations are spot on. The 256 GB M3 Ultra is probably the best value right now even though it's 2k EUR more expensive than the 96 GB version.
PlatoIsADisease 23 hours ago||
There is a reason no one uses Apple for local models. Be careful not to fall for marketing and fanboyism.

Just look at what people are actually using. Don't rely on a few people who tested a few short prompts with short completions.

mitjam 22 hours ago||
yes, I'm using smaller models on a Mac M2 Ultra 32GB and they work well, but larger models and coding use might be not a good fit for the architecture, after all.
01100011 1 day ago||
That's great! Apple has the resources to incentivize and invest in alternate production capacity(Intel, Samsung, or others). Sure, it will take years, but a thousand mile journey begins with one step...
SecretDreams 1 day ago|
Apple is actually a big reason why TSMC is the king of fabs today. They were a reliable cash source for years before TSMC was even ahead of Intel.

Apple can and should do it again!

nonethewiser 20 hours ago||
Fabs are in kind of a catch 22. They need big business to improve and to get lots of business they need to be competitive. Im mostly familiar with that narrative in terms of Intel's current uphill battle - was it really the same for TSMC? I guess maybe there was a similar dynamic except the playing field was more even at that time, so it was a bit less of a catch 22.
SecretDreams 15 hours ago||
Yes, it was. Intel was well ahead of TSMC for quite some time. But TSMC had a diversified and hungry list of clients, with Apple at the forefront. Apple got the taste for wanting their own chips which pushed TSMC to be hungrier. Meanwhile, Intel got fat and complacent. It also helped that phone chips were considerably smaller, so managing yields was easier.
captain_coffee 1 day ago||
Legit question - what is the current status of the construction of chip production factories in the US?

I know about the existence of the initiative but I don't know how it is progressing / what is actually going on on that front.

jobs_throwaway 1 day ago||
TMSC's Arizona fab is up and running producing 4nm chips

There's ~a dozen in the works or under construction

TMSC plans to have 2-3nm fabs operational in the next 2-3 years

So we're 2-3 years behind the standard (currently 2nm), and further behind on the bleeding edge sub-2nm fabs

techgnosis 1 day ago|||
Don't forget Intel. They are producing chips on 18A right now, with 14A up next.
alt227 21 hours ago|||
> TMSC's Arizona fab is up and running producing 4nm chips

Are the majority of the staff still shipped in from Asia?

jpk2f2 21 hours ago||
No. It was originally 50%, unclear what the current numbers are (it was supposed to decrease over time as they train local replacements).
FuriouslyAdrift 1 day ago|||
TSMC is already producing at their first one in Arizona (N4 process), second one comes online for N3 in 2028, and third one (N2) broke ground in April 2025 (online date 2029-30)

https://www.tsmc.com/static/abouttsmcaz/index.htm

tim-tday 1 day ago||
The projects seem to go well and then union bosses threaten to shut the whole thing down.

Then the essential skilled personnel can’t come train people because the visa process was created by and is operated by the equivalent of four year olds with learning disabilities. Sometimes companies say fuck it we’re doing it anyway and then ice raids their facility and shuts it down.

I’d post the news articles about th above, but your googling thumbs work as well as mine.

mekpro 12 hours ago||
I think the opposite. Having NVIDIA investing in TSMC's bleeding-edge process node should benefit Apple rather than disadvantage.

It means that Apple doesn't have to be sole investor in latest node development which is more harder to justify, especially in the year where smartphone upgrade cycle is slowdown. Having NVIDIA (and AI boom) in the picture should help Apple reduce CAPEX for their semi-conductor investment.

alexpham14 7 hours ago||
Ironic, everything will eventually end in some kind of compromise that benefits everyone. That’s how the giant techs have always played.
jlarocco 19 hours ago||
This isn't really news. Apple has to pay market price like everybody else.

NVidia gets the capacity because they're willing to pay more. If Apple wants to, they can pay more to get it back.

radium3d 16 hours ago||
This all is just spotlighting the weakness of NVIDIA, AMD, Apple, Microsoft, etc. They all avoided manufacturing in-house for so long and now they're fighting for fab time. Intel on the other hand is interesting...
philipallstar 2 hours ago||
It's a trade-off. If it's worked for this long, it was probably a good idea.
bigyabai 13 hours ago||
Intel still hasn't proven that they've got the whole EUVL thing figured out. The best Intel chips you can buy right now use TSMC chiplets on the die.
signatoremo 11 hours ago||
Incorrect. The best Intel chip you can get is Panther Lake which is made on Intel 18A node, available globally at the end of this month. Intel has already used EUV machines in Intel 7 and Intel 3 nodes, for the last few years.

https://newsroom.intel.com/client-computing/ces-2026-intel-c...

bogdan 8 hours ago||
> available globally at the end of this month

So it's not available yet then?

JanSolo 1 day ago||
I'm surprised that Apple is not considering opening up its own fabs. Tim Cook is all about vertical-integration and they have a mountain of cash that they could use to fund the initial startup capex.
bob1029 1 day ago||
Semiconductor manufacturing is not an incremental step for Apple. It's an entirely new kind of vertical. They do not have the resources to do this. If they could they would have by now.
boredatoms 23 hours ago|||
They could buy global foundaries and pour in a pile of cash, 5 years later they’d have something useful

Or they could buy out Intel and sell off their cpu design division

bgnn 21 hours ago|||
In that case they would have just burnt cash for 5 years and didn't have anything to show for it.
alt227 21 hours ago|||
If it was that simple, they would have done it.
zvqcMMV6Zcr 1 day ago||||
Designing CPUs also wasn't their core business and they did it anyway. Apple probably won't care that much about price hikes but if they ever feel TSMC can't guarantee steady supply then all bets are off.

I wonder what will happen in future when we get closer to the physical "wall". Will it allow other fabs to catch up or the opposite will happen, and even small improvements will be values by customers?

JKCalhoun 21 hours ago|||
How do they not have the resources? Certainly they have the cash resources.

At this point it would be corporate suicide if they were not outlining a strategy to own their own fab(s).

bob1029 20 hours ago||
> Taiwan Semiconductor Manufacturing Co. plans to spend a record of up to $56 billion this year to feed the world’s insatiable appetite for chips, as it grapples with pressure to build more factories outside Taiwan, especially in the U.S. [0]

Apple has less cash available than TSMC plans to burn this year. TSMC is not spending 50 billion dollars just because it's fun to do so. This is how much it takes just to keep the wheels on the already existing bus. Starting from zero is a non-starter. It just cannot happen anymore. So, no one in their right mind would sell Apple their leading edge foundry at a discount either.

There was a time when companies like Apple could have done this. That time was 15+ years ago. It's way too late now.

[0]: https://www.wsj.com/business/earnings/tsmc-ends-2025-with-a-...

cmgbhm 1 day ago|||
Apple has very much been wanted absolute flexibility to adopt major technology changes so much they’ve tried hard to not be the sole customer of a supplier and deal with political ramifications (source: Apple in China/Patrick McGee)
xnx 1 day ago||
$20 billion for a new fab is a lot of money, even to Apple.
LunaSea 6 hours ago|||
Apple has around $55B in cash, much more in stocks.
DetroitThrow 1 day ago||||
Closer to $40b for a new fab for an established company to do it all correctly. It's a much more major investment to open a fab without ever doing it before, then continually use the brain power/institutional knowledge you've built up to stay near the forefront of fab tech, and then basically have weird incentives to build a foundry for only your products rather than the world at large.

You're setting yourself up for making a huge part of your future revenue stream being set aside for ongoing chipfab capex and research engineering. And that's a huge gamble, since getting this all setup is not guaranteed to succeed.

JKCalhoun 21 hours ago||
Is that true? I guess what I mean is, is it $40B if you are trying to replicate the scale of a TSMC fab? Or could you do it for considerably less if the fab is initially designed to the needs of single customer (Apple)?
DetroitThrow 17 hours ago||
Closer to $40B for some of the latest fabs from TSMC you're seeing, yes. While there could be huge simplification in SoC and packaging processes if it was focused on a single product, Apple's needs will likely still be about having cutting edge processors, so it would still be pretty high even if they were to just buy TSMC.
HardCodedBias 1 day ago|||
If it were only 20B then Apple would jump at the chance.

As would almost innumerable others.

JKCalhoun 21 hours ago||
Well, if the future of your company depends on a fab, twice $20B is cheap.
chao- 21 hours ago||
I look forward to Intel announcing that Apple is the major customer they hinted at having for their 14A process.
wewewedxfgdf 20 hours ago|
I thought this got sorted out with giant piles of cash several years ago, didn't it?
hu3 19 hours ago|
someone has a larger pile of cash now
More comments...