Posted by jonotime 2 days ago
My OpenCode Go monthly window was scheduled to reset this morning. It was sitting at 22% used despite me using DeepSeek V4.1 Flash heavily as my implementation agent the past couple weeks (I use gpt-6.1-sol high for planning/orchestration).
I had 1.5 hours left so I fired up first 10, then 20, and finally 50 concurrent subagents all working on reverse engineering C code from an old PC game. They found over 100 new functions.
This is the first workload I've found that could make a dent in my sub. It got my 5 hour window to 85% used, but sadly my monthly was still only at about 35% when it reset. So that cost maybe $2.
Currently have auto compaction turned off. When the orchestrator's context is getting close to full, I have it write a handoff markdown file and point a fresh agent at it.
I do feel like I'm getting close to the point where I might be ready for something more sophisticated, especially wrt to subagents communicating with the orchestrator.
In a good harness that should be how auto compaction works anyway
Check: https://agentmgmt.dev/ and find the one that works for you.
I quite like Paseo (been maining it for a week), but Orca also looks good.
so even if the week reset with some %usage left, its not actually lost if its not the end of the month.
1. a model that works for one person/task may not work for another;
2. there are many models (DeepSeek, Qwen, GPT, Claude, Gemini, etc.) that are released every 6 months or so;
3. it takes time to use, test, and evaluate the suitability of a new model and not everyone has an automated evaluation process for their use cases.
Thus, if you find a model that works for you then you are not going to spend more time evaluating a model that may not work, or may only do so when time permits.
I don't think so.
That said, it's my best understanding that these american companies aren't profitable and will eventually raise rates (the old uber trick) so I'm keeping myself ready to switch when that day comes.
VRAM & Memory Requirements by Precision
• FP16 (Full Precision): Requires ~1,664 GB of VRAM (e.g., an 8x B300 288GB cluster).
• INT8 Quantization: Requires ~832 GB of VRAM (e.g., 8x H200 141GB).
• INT4 Quantization: Requires ~416 GB of VRAM (e.g., 8x A100 80GB)
VRAM aint cheap, Sam Altman ruined the cost of memory, Nvidia doesnt make enough consumer GPUs letting the market go insane over them, I still have friends on 1070s or 1070 TIs because GPUs have been severely overpriced for too long. I remember when a gaming PC was only $1000.
Even so why would anyone not sleep on a model they cannot run?
Memory companies have price fixed multiple times. They've paid hundreds of millions in fines. wikipedia even has a page on it. https://en.wikipedia.org/wiki/DRAM_industry_price_fixing.
Look at the financials of these companies, they're all making obscene margins and do they plan to increase production? No. Micron is doing a stock buy back to pump the price of their share.
The Micron CEO just recently said this is the exact plan https://www.theregister.com/systems/2026/10/01/ram-supply-se...
There's sanctions, tarrifs, and a DOJ who doesn't give a shit. Until we can fix that the insanity will continue. Phones will be unaffordable. Laptops will be obscene. Gaming consoles will be thousands of dollars. Desktops will be dead.
If you're waiting for some David Ricardo equation to happen, tough cookies, it's not coming.
The market is legally locked down and we're in hostage pricing mode.
And what's the story? You can't afford electronics because we're using it to build robots to take your job? I mean ...
Nobody is coming to save us. That's our job.
Micron has 3 brand new fabs currently under construction, 2 Boise, 1 in New York as the first of 4 planned for a campus.
Plus expanding other existing facilities.
These things take ~3-5 years from breaking ground to full production. You'd have had to anticipate the current demand years before it happened in order to be bringing production on-line before 2030 or so.
Samsung and HK Hynix also have fabs under construction and planned.
CXMT started 11 years ago and only now is reaching any real volume. If they decided a year ago to react to the current demand cycle they'd be 6-7 years out.
Not much you can really do to wish for more fabrication to exist on any timeline not measured in fractional decades.
Could they do more and react quicker? Probably, but everything I've read on the subject seems to point to 3 years is absolute bare minimum if you happen to have a shovel ready project with the land bought, local permitting completed, infrastructure extended to the site, and a skilled workforce already in place. They could suspend buy-backs/dividends today and dump it all into building production and there would be no material impact until around 2030.
> The Micron CEO just recently said this is the exact plan
CEO simply stated the demand pressure will not go away through 2027, and supply will not increase until around 2028 when currently under construction fabs start shipping volume. The article does not support your statement.
Costs did go nuts, but there are signs of easing in the market of late. CXMT is starting to have an impact and priced will probably fall in 2027.
It's taken them this long to catch up to the DDR5 standard. They've only recently been through qualifications to be a DDR5 supplier for the big boys.
> Every Major Motherboard Maker Now Validates CXMT DDR5
https://www.techtimes.com/articles/321572/20260725/every-maj...
After their recent IPO, they have more than enough cash to ramp up in a major way.
It's just a matter of time.
Some of my family is pretty happy, though, with the job security as they are pretty convinced these projects are all going to take much longer than what's being stated publicly. Micron is saying the first chip from the new fab will be in 2027... though they also predicted it'd be 2026. The date seems pretty slippy.
Especially given CXMT has been able to scale up much faster than what most people expected, only reason their isn't a bigger impact is modern HBM is hard to CXMT even today.
We are likely to see supply double in the next 3 years, but demand even out with optimizations, cooling of data center demand, and most importantly moving some of the dram to flash demand instead which is much easier to produce and scale.
You need to keep the market healthy, not some insane Bitcoin style HODL pump - that's how you get wrecked.
I mean I'm not a neoclassicalist but I've read all of them. I'm in consensus with them here. There's a bunch of theories on what a healthy market is but what we're currently seeing matches none of them.
It's short term profitable but long term disastrous, especially in a world where new mathematics and techniques could literally collapse the demand overnight.
Imagine if some paper hits arxiv and the 256 GB requirement for some model now becomes 64. Woops!
Some clever trick about how attention heads and context Windows work could potentially slash a bunch of requirements by giant margins and all they're doing is firing the starting gun at that global race with every obscenely priced unit they sell.
But if prices were reasonable, this wouldn't be an apocalypse. It'd be fine. Consumers wouldn't rush to 64GB, they'd say " Cool I can multitask now at 256" or " great I can do horizontal scalability' or something else.
But no they created the market conditions so now what would happen is the consumer will immediately flip the 192GB they don't need on eBay, hoping to snatch a profit before the prices tank and the second hand market will be flooded the rug will be pulled out from the luxury pricing and everyone will get screwed.
This has happened in electronics markets before. Many times.
When Engels talked about the grave diggers of capitalism they were looking at it through a 19th century labor/manufacturing lens but arguably this same dynamic is at play here.
This is a tiny percentage of the population.
Samsung is cutting phone production because of RAM prices.[1] The consumer market is badly affected: budget phones, laptops, general electronics.
The budget segment of sub $100 devices in India has been almost wiped out. Manufacturers cannot afford to spend 50% BOM on RAM+storage. Unless employees are getting a 15-20% wage rise this year, I expect a similar situation in most places.
Between the engineered conflict in the ME triggering O&G price rises, and stratospheric RAM pricing, the situation is pretty bad.
[1] "There is no profit even if we sell"…Samsung to cut smartphone production by 30% (https://www.mt.co.kr/en/tech/2026/10/08/2026100709554237233)
Let's say ram used to cost $100 and now that same unit costs $1000. You paid say $500x1,000 for that unit during the price increase or some price where you can currently flip for profit.
You have a very expensive data center and you're in debt financed on the premise that you have these special computers.
Now a new technique comes out and it turns out you only need 1 memory unit for something that used to require 8 or 4 or some meaningful multiplier.
This stuff happens all the time. It's why we don't use BMP files on websites or serve giant MOV files on YouTube. It's why postgres queries are faster now than they were 10 and 20 years ago.
You rent out your machines. You need to service your debt.. Demand may 8x overnight to accommodate but you have a monthly bill to pay and that's unlikely. It's likely going to drop.
Think about it. Your customers are paying maybe $10,000 a month and serving their customers. Now they can drop that to $1,250.
On market if you were to sell some of that ram you have 100% profit right now but not for long.
Jevons paradox assumes unlimited capitalization, zero debt servicing, infinite time horizons...
We live in the real world so what do you do?
Historically the answer has been "sell that shit"
There's an aphorism for this "stairs on the way up elevator on the way down"
If we had a healthy market with sane prices where you can't flip the thing you bought for 100% profit the answer would be "create more value."
Or they could stay at $10,000 per month since they are willing to pay that much already.m, so they just use AI more and in more places.
Why not? Unlike many other workloads, LLM inference actually seems pretty suitable for decentralization (effectively stateless means no availability concerns; bandwidth and latency are relatively forgiving too).
People who say they want local runs really mean it: they want local runs on hardware in their room, not on some decentralized system which, if it existed, would almost certainly just be a worse, less-reliable version of cloud hosting. I'm not saying nobody would use it, but it sounds a lot like things like IPFS, which have also completely failed to displace either cloud storage or buying a bunch of disks for your own private use.
Decentralized storage is much harder, since there reliability matters a lot more as it's inherently stateful. You have to assume data loss, so you have to replicate everything; with inference, you only have to spend extra resources at failover time. Also storage can't be time-shared in the same way as compute; if it's full, it's full even when not actively accessed.
If you were a DRAM manufacturer, isn't this exactly the kind of thing that would make you think twice about investing years and $billions in new fab construction?
How many people, outside of tech geeks and megacorps care about RAM prices? And how gullible would you be to BELIEVE the politician they could actually make it happen, and even if they did, that it would extend to the average person, and not JUST megacorps/megadonors?
I think a lot of people care about the downstream effects of memory prices, but I agree with you that they may not realize that they happen because of memory prices.
That might be true at micro level, but at the macro level more memory just means developers get more lazy with their optimizations, causing apps to get more bloated, eating up any gains in extra memory. There's no reason why slack needs 1+GB to run, yet people are perfectly happy to put up with it.
They don’t care about RAM prices, but they do care about the price of things that have RAM in them (or even NAND), and all of them are increasing way faster than inflation.
In this case I think investment in more production is the only option, and it needs to happen even if it is expensive and slow.
wonder what voting would be like?
gamer vote ++
datacenter hater vote --
datacenter lobby ++
micron lobby --
I spit out my coffee laughing when I read this
Priorities
How? Increase production? The time needed to scale up the production is longer than one election cycle.
I know it's just a figure of speech, but damn. I laughed out aloud in public just reading this.
I guarantee Micron & friends are not intentionally orchestrating their business such that they would suffer a massively reduced chance of yielding on a per-die basis. Unless someone is actually buying HBM devices, they are not going to be making them. These are not a commodity that can be speculatively manufactured in any economically rational way.
Or abolished VAT (the meaning of VAT is that you pay a "rent" for all the infrastructure used to produce the thing) and import taxes (protect your market) on stuff we don't produce in our markets anyway.
https://web.archive.org/web/20250612003557/https://diskprice...
After 70 years of decreasing computer prices all of a sudden it's gone 3x, 5x, 10x up in 1 year, we are in total clown world and saying "dur AI" is lazy and doesn't map to reality.
It's Argentina style inflation - as if Honda said "we're only making $500,000 luxury cars now. Everything under $50k we've stopped." and then those cars shoot up to $125k.
It's destroys the market, destroys the consumer, destroys the company, dismantles everything, and they do it for the short term payday.
Is it really so hard to believe that RAM prices are up because demand is simply exceeding supply, especially in a market where additional supply takes years and billions of dollars to come online? There's no need to posit cartel behavior and a fair amount of evidence that there is none.
I got a 4090 in 2023 for 1600, a 5090 in 2025 for 2000 with 256 DDR5 for about $1,000 ... and then, after some protectionist legislation passed, these prices quickly shot to the moon.
Connect the dots.
You can't say "connect the dots" at the end of a raving, mostly-incorrect post and act like you've made an ironclad argument.
It was supposed to be over by now and then they said 2027, then it's 2028, and now I hear "oh it's going to continue to rise the rest of the decade".
I'm likely going to be flying into Shenzhen to put my next computer together. The one I put together in 2025 would have cost me about $25,000 right now. I paid under $5,000.
I can round trip to China for $750. So once they ramp up production that's the strategy.
Other countries already do this. Apple and Google aren't in every country and those people buy new electronics when they travel.
The USA is soon to be on that list
You aren't engaging in good faith and there's no reason to continue with you.
Then good portion of those weights are n-grams (~200GB) that don't need to be in VRAM.
Then KV cache of that model is super lightweight at ~1GB per 1M tokens. If HBF succeeds, then accelerator with 16GB of VRAM and 1TB HBF/NAND is probably all you need (?).
Edit: I went and checked for you. The LM backbone is 307.2 GB (286.1 GiB), straight from DeepSeek's upload. The n-gram table is 203.1 GB (189.1 GiB), which goes in host RAM. Note the embeddings are higher precision than the expert tensors, so it's a larger fraction of the bytes than it is of the parameters.
So,
> Call me crazy but:
You're crazy. :-)
Hardware update cycles are 2-3 years even on the high end, so it's still a ways away before "good enough" and "local" belong in the same sentence for the average person.
And by then, DeepSeek V6 Flash will be too cheap to meter, 5x faster, and 10x better, so... You'd still need to go out of your way.
Most people are spending most of their time on their phones anyway. ..
I know my comment is a little nit picky because it's still pretty expensive to run, but it's not quite as bad as this comment makes it out to be. Really, if you're VRAM constrained, take a look at GLM 5.3 Flash or Qwen 3.8 Flash Next before you worry about this model as all three models perform pretty similarly.
1070ti launch MSRP was $450 ish. 5070 could be had in the last year for 5xx-6xx range easily.
All things considered - (inflation being about 30%~ (guess)) between these two timelines. You are looking at 300% performance difference at a cost dollar for dollar that is cheaper then when they purchased their cards.
Might be a bit of a stretch blaming it on "severely overpriced for too long..."
Or you can just use any of the neoclouds' shared hosting. The thing for them to be freaked out is that these models are getting good enough very quickly, and all the shared hosting providers can run them for a tiny fraction of what the frontier model companies charge.
It has a set of n-gram tables which you can stream from system RAM or even NVMe
That said it’s still quite big! I can’t fit it on my DGX Spark, though I believe you can if you have two?
I’m quite spoiled with how good Qwen 3.8 Flash Next is on a single spark though: shocking how good local models are getting on attainable-ish hardware
https://blog.jonathanpage.com/
GLM 5.3 Flash runs fine on two Sparks and Qwen 3.8 Flash Next on one is indeed incredible! I made this 3D game with it in two days using Qwen Code as agent:
https://www.storagereview.com/review/dgx-station-gb300-clust...
Allow a question from someone who’s only got a very vague idea of how this kind of stuff works behind the scenes: say I rent usage of this model through one of the many LLM hosting providers out there, and let‘s assume I use it extensively through something like Pi or OpenCode and vibe code away all the time, keeping the hosted model occupied as much as I can, happily burning my credits.
Does that mean that there is a hardware cluster as described by you above that is crunching away just for me?
So at FP16, I alone keep a 1,664 GiB system occupied all the time?
The "expensive part" of generating the next token is streaming in the model weights from memory. The computations are relatively simple, which is called a "low arithmetic intensity" in industry jargon.
So what they do is batch multiple chats together and compute the neuron activations for all of them together.
This is vaguely similar to how some database engines work, where if multiple users need to run a "whole table scan" query, the additional users "join" the streaming workload of the first query mid-way, then loop back around to complete the first part that they missed. The AI accelerators don't do this looping, but the concept is the same: amortize the expensive I/O over multiple computations running in parallel.
The "turbo mode" token rate thing is almost certainly your query getting sent to slower or faster hardware, like B200 vs newer B300 kit.
As background: For the most part VRAM oversubscription/paging/swapping isn't a thing in the same way that RAM for a VM often is. There are some approaches to it, but (to my knowledge) not at that sort of scale.
There are some systemic reasons for this, but very broadly speaking the GPU vendors are building toward the highest bandwidth and lowest latency possible, and the overhead/complexity of something like protected memory modes serves neither of those priorities.
Because it's an open model so providers compete on price.
I reimplemented most of the features of the Deepseek v4.1 flash paper (apart from quantization aware training which doesn't make sense because my implementation uses float32 precision anyways)
I'm currently learning how to distill reasoning traces (check my other github repositories) but I think that a locally selfhostable deepseek is possible with my mixture of experts sharding mechanism. I decided to optimize everything for CPU parallelization, with the idea that the KV cache and meta model have to run from CPU RAM anyways, so the experts can also be loaded/unloaded at runtime if needbe, to save more RAM.
My assumption is that the KV cache optimizations in combination with the CED and compressed attention features are the reason why v4.1 flash has so few hallucination problems and such a strong self-lookup/thinking behavior. But that's more a gut feeling, need to evaluate and test this more thoroughly.
Anyways, would love to see someone train this on their own datasets. Currently my pipeline is kinda optimized for parquet and zim files.
it rips with just 64 ram and a 9070xt
No, you won't get frontier-level intelligence on a 1070Ti. Yes, it should be illegal to do what Altman did. Since we clearly don't live in the best of all possible worlds, we need to settle, and DS4.1 Flash is a good place to do that.
For tasks that don't require vision I personally like the NVFP4 quant of GLM 5.3 from Local Inference Lab better than DS4.1F, but they are both well beyond awesome.
1660 ti, 4790k, 16gb ddr3
He gave demand signal so many times years ago and was mocked for it and now we have the consequences of industry not taking him seriously.