Top
Best
New

Posted by teoruiz 14 hours ago

Tokens too cheap to meter(jyn.dev)
218 points | 175 commentspage 4
Salparvezml 4 hours ago|
[dead]
marbleotter115 11 hours ago||
[flagged]
sick_of_slop 6 hours ago||
[dead]
automatic6131 11 hours ago||
>We are likely to see LLMs running locally at current frontier-quality on commodity hardware in the next 3-6 years

Yeah okay bud, anyone checked in with the state of consumer hardware recently? Not the author, evidently.

>oh in 3-6 years this will all be over

Yeah I'm sure Samsung, Nvidia and sk hynix will all be very calm with lower volumes and lower margins.

sanderjd 9 hours ago||
Seems like 3-6 years is a pretty conservative timeline to me. Memory and inference chip production could ramp up massively in that amount of time.
samuelknight 10 hours ago|||
The article observes that the cost of frontier intelligence from 2025 has fallen 100x in the last year. It also notes that the energy to run models is also collapsing. Consumer hardware is borked right now because these new algorithms are revolutionizing the utility of a computer. Computing is technology who's cost has been collapsing for 90 years, and its a safe prediction that it will decrease again.
api 11 hours ago||
> Yeah okay bud, anyone checked in with the state of consumer hardware recently? Not the author, evidently.

RAM prices will crash when demand drops even a little. They'll probably crash to a lower (inflation adjusted) level than before. This has happened before.

Industrial scaling in general often looks like a sawtooth: price spike, capacity investment, crash, repeat.

Part of what's keeping prices high a little longer is that everyone knows this and is a little reluctant to plow resources into chip fabs for fear of having the bottom fall out before they recoup or sell that to someone else to hold that bag.

Graph the average compute and RAM in a mid-high end laptop at an inflation adjusted price point for the past 40 years. It's very exponential and hasn't slowed down much.

yxhuvud 10 hours ago|||
No. Prices will crash when supply side expands to meet the increased demand. Because demand won't go down to pre-bubble times any time soon. Unfortunately the supply side has been very slow in increasing production, partly because most steps of the production chain are all maxed out.

On a long enough scale you are right that prices will likely normalize to a better level, but before 2030? That would mean the factories are built quickly once they begin.

bryanlarsen 10 hours ago||||
> everyone knows this and is a little reluctant to plow resources into chip fabs

except cxmt who is plowing resources in like crazy

csoups14 10 hours ago||
That has more to do with geopolitics than it does the current price of memory.
glub 7 hours ago|||
The entire reason why hardware prices are so absurd right now is because manufacturers across the board are doing everything to prevent that crash.

They all collectively chose NOT to increase supply with increased demand. So if the bubble pops, they just go back to previous prices without oversupply driving the prices to rock bottom.

129348681 10 hours ago|
So this week's coordinated shock and awe campaign is about "open models" and "prices are falling rapidly". Also on the front page:

https://news.ycombinator.com/item?id=49815526