Top
Best
New

Posted by intelkishan 4 hours ago

Memory has grown to nearly two-thirds of AI chip component costs(epoch.ai)
196 points | 209 commentspage 3
deadbabe 3 hours ago|
Here’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab.

And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solidify subscriptions forever.

aurareturn 3 hours ago||
Keeping prices at this level is precisely how one or more competitor will rise up. Making memory isn’t super hard. That’s why it is a commodity. The problem with the memory market is that up and down cycles have bankrupted the vast majority of players in the past. Now we only have 3 players left except for a few smaller ones in China.

The reason memory prices can stay high for years in this mega cycle is because the 3 players will be very cautious on overbuilding. They’d rather under build, make great profit (not maximum) and reduce the risk of going bust if this suddenly ends.

Same for TSMC in chips.

Great opportunity for Chinese companies though. This shortage is exactly what Chinese companies need to scale.

petra 1 hour ago|||
//Making memory isn’t super hard. That’s why it is a commodity.

These two aren't related.

Dram is a commodity because the you can replace a chip from hynix with a chip from micron, the have the same behaviour.

And a price competitive Dram isn't easy manufacture, or China would have made it already.

dymk 3 hours ago||||
> Making memory isn’t super hard.

Then why do only 3 companies make it?

aurareturn 3 hours ago|||
Bankruptcy risks.

When Samsung had to sell memory at a loss after COVID, no one came to save them. They buffered their memory division using profits from their other businesses. That’s how Samsung survives memory downturns.

According to some stories, this is how Samsung convinced TSMC to not enter the memory business - that you need a nation or other lines of business to prevent bankruptcies.

The market has stabilized to 3 players.

dymk 3 hours ago||
...And why do they go bankrupt?

Because it's an incredibly capital intensive process, involving billions of dollars of investment into manufacturing infrastructure.

That is to say, making memory is quite hard.

aurareturn 3 hours ago|||
The technical process of making memory is relatively easy. Hence, it is a commodity.

I didn’t say owning a memory business is easy.

kortilla 2 hours ago|||
You’re confusing two independent things. There are simple processes that are extremely capital intensive with long lead times and then there are complex processes that require lots of R&D and industry secrets. Memory is the former in the chip world.

Other examples from outside of tech of easy but capital intensive processes are power generation and railroads. Very easy to do, but easy to end up broken by overbuilding for demand that fails to materialize or stay stable for the duration of your financing.

DoctorOetker 3 hours ago|||
Making the memory can be much easier than predicting future demand.

Placing the bet isn't as hard as making an accurate prediction.

jazzyjackson 3 hours ago|||
> up and down cycles have bankrupted the vast majority of players in the past

Exactly, so what’s the incentive for anyone to sink half a billy into building out more capacity.

The existing players get to rest on their laurels and succeed whether or not the AI bubble busts.

aurareturn 3 hours ago|||
The incentive is that your 2 competitors will build more than you and gain market share on you if you are too conservative.

Samsung, SK Hynix, and Micron all have to balance between capex spending, making as much profit as possible, and risk of bankruptcy.

deadbabe 3 hours ago||
So are the new competitors currently in progress of starting up? Because it takes at least several years.
aurareturn 3 hours ago||
Only Chinese companies have a chance. Problem is that China can’t buy EUV machines and the most advanced memory chips now use EUV.

Heck, the US is now pressuring ASML to not sell even DUV machines to China, period.

jtbayly 3 hours ago|||
When costs are high enough, you can recoup that, if you have an appetite for risking the downturn.
YetAnotherNick 3 hours ago|||
If the collude to say make the price $1000 for a component that costs them $100(including opportunity costs), then either a new company or a greedy company in the collusion can make their price secretly $900 and get massively more profit.

Right now their opportunity cost is too high.

> risky it is to spin up a new fab

You don't need a new fab. You can build memory in 20 years old fab.

stavros 3 hours ago|||
Then that's a cartel and hopefully regulators will act.
deadbabe 3 hours ago||
They won’t.
aurareturn 3 hours ago||
They will. DOJ prosecuted memory makers in the late 90s and 2000s for collusion.

This boom is magnitudes higher than before. The attention will be endless.

deadbabe 3 hours ago|||
Current DOJ is corrupt as fuck, it will not happen. Get back to reality.
aurareturn 3 hours ago|||
They will respond when people are loud enough. If memory stays at $1200 for 128GB for years and investigative journalists say it could be colluding, enough people will make enough noise.

I’m sure Nvidia, Elon, Tim Cook, OpenAI, Anthropic are already whispering in Trump’s ears to do something.

BigTTYGothGF 2 hours ago|||
> I’m sure Nvidia, Elon, Tim Cook, OpenAI, Anthropic are already whispering in Trump’s ears to do something

You can't expect me to believe that any of those would want any kind of antitrust action against anybody.

aurareturn 2 hours ago||
Sure they do. They all have money interests in this. They all want lower memory prices.

Memory prices and shortages directly impact all of their profit margins and revenue.

wahnfrieden 3 hours ago|||
Once the masses are disenfranchised network state serfs according to plan, loudness won't matter
kortilla 2 hours ago|||
Corrupt doesn’t mean “acts without incentives”. If you assume a corrupt system, then the inputs are going to be who has influence over the DOJ. If there is more money to be made by breaking a cartel, then they would absolutely do it.
CamperBob2 3 hours ago|||
That was a very different DOJ. They no longer work for us. They act as Trump's personal law firm.
shaky-carrousel 3 hours ago|||
Then China will come and eat their lunch. I for one will only buy Chinese RAM from now on, no matter the prices.
granzymes 3 hours ago||
>I for one will only buy Chinese RAM from now on, no matter the prices.

Memory is a commodity, so I think you will be very lonely in your quest.

sieabahlpark 3 hours ago||
[dead]
brcmthrowaway 3 hours ago||
Anyone invested in Micron stock?
lostlogin 2 hours ago|
Up 700% in a year.

WallstreeetBets has been disturbingly accurate in its predictions - basically anything related to AI.

positron26 3 hours ago||
The algorithm advances are going to crash this so hard.
Legend2440 3 hours ago||
Or will more efficient algorithms just mean we run even more AI models, increasing the demand for AI chips even more?
helterskelter 29 minutes ago|||
Better algos = more demand

Memory squeeze will get worse before it gets better.

Npovview 1 hour ago|||
I heard Greg Brockman on a podcast saying they are limited by computer and memory. They have line of sight in solving many different kinds of problems. But they also have to survive in the meantime. Hence the focus on enterprise recently. They could just ask Government to fund them doing other research areas
Coffeewine 3 hours ago||
I mean, god willing, but it'll be just as likely that we'll blissfully consume 100 million token contexts in that case.
iamtheworstdev 3 hours ago||
isn't there a law for that? as things become cheaper you consume more?
sobellian 3 hours ago|||
You're probably thinking about jevons paradox. But you slightly mis-stated. It is the phenomenon that increasing the efficiency of resource consumption can end up increasing total consumption.

As you stated it, it would merely be a property of (nearly) all demand curves. Jevons paradox only happens sometimes. It isn't a law.

dangus 3 hours ago||
An example of where it stopped happening is with gasoline in developed countries. Cars having better fuel efficiency doesn’t make me drive further to the grocery store or work.

Generally when someone replaces their vehicle the new one is more fuel efficient than the old one even if I bought the same car.

simonw 3 hours ago||||
Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox
loloquwowndueo 3 hours ago||||
Jevons paradox.

https://en.wikipedia.org/wiki/Jevons_paradox

sidhantdhar 3 hours ago|||
jevons paradox
ck2 2 hours ago||
if we survive the bubble bursting and there isn't a "too big to fail" bailout with public money manipulation by bought politicians

we are going to have amazing cheap used hardware for a decade

jalospinoso 1 hour ago||
[flagged]
thinklanceai 2 hours ago||
[flagged]
Lapsa 2 hours ago||
[dead]
alasdairnicol 2 hours ago|
C