Posted by ihsw 5 days ago
https://www.andysarcade.net/store2/all-other-stuff/vintage-c...
TIL there are 2x 2.5GbE PCI-E HAT adapters for Pi 5.
How to attach RAM to the new NVLink/UALink fiber buses?
Weights need to be loaded into the accelerator's processor fast, which means they need to be physically adjacent to it, but there is limited physical space for that - not enough to fit the all the weights of a 1T+ param model, so weights get loaded into VRAM dynamically according to what part of the model is being run.
ROM (I guess we're talking Flash memory) can be dense, since it is built vertically - many hundreds of layers, but this comes at the cost of poor performance, so even if you could fit enough ROM next to the processor it would not be fast enough.
GPU transistors are smaller due to the more advanced process node (cost per transistor metrics aren't really clear, if they improve on advanced node or not, but I'd say they get cheaper as they get smaller, as technology costs are amortized).
I'm sure both RAM and logic use a process that is quite similar in both inputs and manufacturing steps. So while RAM is a commodity product, this insane price difference didn't make any sense.
So I guess when those fundamental inputs become a constraint, it would make sense for $/transistor move closer for both, which is a massive hike for RAM.
Is it built in different silicon, is it physical steps that's incompatible (ie its actually incompatible), is it different physical preparations that needs to be made (making it economically infeasible to combine)
I cannot help but wonder, even if the answer doesn't change anything in my life.
At a very abstract level, when you're manufacturing DRAM you need to manufacture a lot of circuit elements that have HIGH capacitance, since a DRAM cell is basically a capacitor and the higher its capacitance the less frequently it needs to be refreshed.
On the other hand, when manufacturing logic (CPU/GPU/ASIC) you want to minimize the capacitance of almost all circuit elements, since capacitance introduces delay and switching energy cost.
Nearly everything about the manufacturing processes for DRAM and logic is optimized around this fundamentally incompatible figure of merit.
I worked on the development of Intel's eDRAM process, which was used to integrate DRAM into the CPU/GPU die for Iris Pro embedded graphics from 2013-23. https://ieeexplore.ieee.org/document/6576667/
A Reddit user explains a bit here.
This in turn affects the electrical properties: parasitic resistance/capacitance, gate dielectric properties and so on. The dielectric in particular is critically different between DRAM and regular CMOS, because DRAM needs to minimise leakage (as that determines how long the memory lasts between refresh cycles).
Regular factories will retool somewhat between jobs. Because it is quite difficult to finetune a silicon process node, it is more common that a fab will set up for a particular node and then switch to "do not touch or change anything under any circumstances", as doing so may wreck yields.
("different substrate entirely" does exist: that's GaN, for power transistors in phone chargers, and SiC, for even higher power transistors.)
What's more, the configuration and flow of the machines used for each step are quite sensitive: you cannot in general just stand up another fab with the same machines, apply the same settings, and hit go on a new chip design and expect any yield: you need to dial in each step, certainly for each process, and likely for each design. This makes switching things around more difficult as well.
So, while in general a fab will have certain common features: spin coaters, photolithography machines, vapor deposition chambers, ovens, etc, the number and specification of each one will vary based on the process, and a production fab will generally not want to change their process drastically, or even to swap between different designs too often.
If you want to know more, the Asianometry youtube channel has some fairly good deep dives, such as [2] going through a decent bunch of the 45nm production process, or [3] doing the same for (early) DRAM.
[0]: https://www.youtube.com/watch?v=Bln-v9LmZ3E
[1]: https://i1.wp.com/semiengineering.com/wp-content/uploads/201...
You can't find an explanation why they're different for the same reason you can't find an explanation why writing poetry and riding a unicycle isn't the same process.
and cost per transistor stopped decreasing at ~20-30nm, now small nodes are targetting energy efficiency (and thus performance, since heat is the main limiter)
Granted the machines that make them become more expensive, but that's capital expenditure, which gets amortized as time goes on.
So there are two forces here working against each other.
Supply and demand coupled with the fact that a RAM fab can't (trivially) output compute chips, and vice versa, a compute fab can't output RAM. It's two completely different supply chains.
Also you calculate in the machine cost and R&D.
RAM hiked because the demand spiked and these companies are now in power. Before apple and other companies told them the prices and had hardly any money for investment.
absolutely no idea how useful any of that would be and what kind of latency degradation going through whatever adapter would cause
China is desperate to sell anything to... everyone. If there's a market, they'll eventually be there to fill it.
It took them decades for cars, but now they did it.
For RAM, CXMT went from 20 000 wafers per month to... 240 000 wafers per month in something like two years. And they're extending capacity massively now. It's a company only 10 years old.
The market is there and China shall flood it: that's how they operate with everything.
At some point they'll probably even come with GPUs that shall do 80% of the job for 20% of the price.
Just like you can buy chinese server motherboards at 1/5th the price of a SuperMicro one today.
So I'm not sure hardware is going to be only for big companies: China is going to put pressure on the OpenAI and Anthropic of this world locking all the RAM / SSDs / chips of this world.
I've recently gotten into fountain pens. Sure, a $7 Jinhao or $15 Hongdian pen isn't going to write quite as nice as a $200+ pen, but they're about 80% of the way there, and you can buy tons of them for the cost of a single more expensive pen. Plus, some models will accept Western nibs just fine which means you're buying a cheap barrel and assembling a much higher quality product for almost pennies on the dollar.
One would do well not to underestimate their ability to fill markets. It may take years, but it will happen.
Anyway, the point is that in the future you cannot own things. Whether that is because of the small market or because of other reasons, that does not matter.
I'm sure a chinese EV group could key in on the same pure-value market if there isn't a group already doing that. 'Golf carts for the street.'
