Posted by acossta 9 hours ago
Basically SETI for AI workloads? At least, this is the impression the landingpage left me with.
I am a bit lost, tbh.
There is no distributed execution of a single LLM across multiple GPUs using tensor parallelism here. Instead, it relies on task-level parallelism-breaking down a large workload into independent subtasks that run across separate machines. For example, generating embeddings for a million documents.
it's a good idea but you often end up with huge tail latencies, slight incompatibilities due to fleet heterogeneity, and a database/scheduler management problem.
https://developer.nvidia.com/gpudirect
And this is because I can't be bothered to dive into older papers, so recent NVIDIA stuff will do.
Yes, we can distribute tasks to idle machines in homes and on office desktops, but it is often much more expensive than sending it to a data center where power is cheaper and the hardware is more efficient.
(Interestingly enough, I think PRC and ROC are combined in public financials.)
Also they have the ability to put in sufficiently large amount purchase orders to buy up whatever capacity they want.
Even at ludicrous markup prices from TSMC, ifs going to be orders of magnitude cheaper than in house a fab.
(As much as conglomerates are fun…)
Swift's main reason to go open source and have good Linux support was naturally for app developers to be able to use it on server side, which naturally wasn't going to be macOS.
I macOS wasn't required for developing iDevices apps, they would surely sell even less desktops.
Maybe their chip team is already on the work, or already made, a server chip for AI purposes used internally, and doesn't have to talk about it yet.
I would definitely opt out.
Is Apple going to compensate the owners?