Posted by smartmic 3 hours ago
Cooling is usually what kills you with server gear at home. I don't have a cold aisle, or a case that flows that much air. Can I water cool it enough?
No video outputs would probably be my biggest annoyance. It should be fine at headlessly rendering games for streaming though, as well as AI.
If there's a will, there's usually a way.
Yes they use pcie electrically, but its not the same slot shape and there is usually carrier boards
You wouldn’t repurpose these for home gaming, it would still be for llms
You can just mod them to run on consumer fans that are designed to spin much slower but have higher diameter
So, sort of specialized? but not really.
> If there's a will, there's usually a way.
That's the spirit!
This whole subthread is just sad. "Oh no, it's not the same form factor. It is useless".
Where's the "Hacker" in "Hacker News" there.
It's a shame that we need to defer to "China" as a shorthand for "actual engineers" just because our own engineers lack imagination and drive that would validate being given that title in the first place.
There are a lot of reasons other than skill why China is the way it is. Culture and locality are big ones. I don't know a whole lot but I disagree that any "actual engineer" could easily come up with a sustainable business for this stuff in the US -- for example, harvesting and processing used parts for their components is a hugely laborious process, and China has a huge culture of recycling and reworking that simply does not exist in the US (and would largely be socially unacceptable to introduce!). Take for example the pre-modded 4090s, or the BGA desktop motherboards I mentioned, or any number of other products that are based not merely on sheer engineering. It's not exactly engineering on its own that causes these things to happen, but more the fact that nearly anybody can just throw together an operation like that and have tons of workers available to make it sustainable.
Please don't misunderstand me as attacking you. I was attacking that other guy further up the thread with the empty GitHub account and the equally empty ambitions, trying to drag all of us crabs back into this bucket we find ourselves in.
Oh, plenty of people do not choose not to -- there are tons of blog posts and YouTube videos out there of people doing stuff exactly like this! It's just the making it sustainable part that's very difficult outside of somewhere like China, because -- unless you have special VC privilege -- you can't really offer reasonable prices to the masses, which makes your purchase volume go down, which further drives up your prices even more, until it converges on some insane number.
I'd much rather be in the recovery/asset remarketing market rather than the primary/reseller one when this thing pops, for sure.
Is this our chance to 'get rich' by buying a negative valued asset? Like that time barrels were trading at 0$ or negatives?
PETER SULLIVAN: We have to hold these assets on our books longer than we might ideally like to.
JOHN TULD: Yes.
JARED: Sell it all. Today.
JOHN: Is that even possible, Sam?
SAM: Yes. But at what cost?
Is it a data center with nothing but GPUs, that does nothing but inference all day and all night?
Is it a data center custom-built for only AI companies to use?
Do these “AI Data Centers” have data storage? VPS? …
Or, as I suspect, is “AI Data Center” simply a fake media buzzword that really means “Normal Data Centers” but with political baggage and stigma built-in.
No you're absolutely right. The only difference that I can tell is power requirements are legitimately higher because GPUs consume more electricity. More cooling is also needed because there's more heat generated. Aside from those two things, it's a plain old datacenter that companies have been building at break neck speeds since like 1996. Now that I think about it, how come there's no panic about additional Internet backbone capacity needed to support these "AI datacenters"?
It's a different build-out to a data centre that can host video/YouTube content (fast CPU, no GPU, large number/size of hard drives), or that can run web servers (fast CPUs, low to average hard drive capacity).
For my part, I do work on machine learning projects at home. But I'm nowhere near operating at a scale where datacenter GPU features like being optimized for 4- and 8-bit calculations would be a net benefit. Nor do I have any particular desire to have one of those things making my home office sound like I live next to a steel mill.
It's not too far off from why I don't have a box truck parked in my garage even though it's technically more capable than my Kia. And on the occasions when that is what I need, the local hardware store will gladly rent one to me for $20 per day.