Posted by vednig 2 days ago
I've been training a teeny specialised model to run in a browser on a phone to detect harmonium notes played in a song (harmonium turns out is a pita, another story for another day), getting good labelled data is _all_ of the hard work.
That being said, maybe for cheap inference, using a big model to train something ultra-suited for the task at hand might be how we could handle local inference; thinking language specific models.
Hear me out, economies of scale can only be met when there is a large enough liquidity for it.
The amount of people willing to purchase multiple hardware releases year after year just to run LLM is already tiny and businesses already do use their own hardware and there is no desire for manufacturer to reduce their own margins.
Who's gonna pay to power an open source AI? Will it perform well enough to make Chat-GPT and Claude obsolete?
Biden's GPU controls should give you an idea. Thank you, China. Open source AI must win.
Famously, the PowerMac G4 was briefly subject to export controls. Apple turned it into a marketing campaign.
Go ask Claude to criticize Anthropic and see how long your account stays active.