I feel with current government decision to block Fable, this is not a mere opensource issue, considering how US government restrict frontier models, what we need is sovereignty for every country. If not they will release every model with a kill switch in future like F35.

In the US -- once our nation finishes attacking our own education system -- this is definitely something a group of academic institutions could get together and accomplish. I assume the same is true in other countries. Companies like Nvidia and AMD might even support that effort, as they make money on the hardware and would probably be more than happy for there to be more reasons to use it. There may have not been a compelling enough motivation to achieve this before, but "models" didn't have this level of strategic relevance until relatively recently. Nvidia has been fairly good about releasing open weight models in the last few months.

Not today, may after the next 3-4 breakthroughs. One thing that people don't realize is that the AI they use today is highly highly subsidized bc of the capex that has gone into it. Even if people collaborated together - will not be able to raise billions of dollars that are needed.

These are still very very (and very) early days of the modern AI and there are so many changes that are gonna happen. It's possible that all the frontier labs of today won't exist in a few years.

I think it's also important and heavily overlooked to develop and maintain open source "pro" level models. Those that are able to think for 80 minutes and yield heavy solutions.

I'm not an expert in LLMs so it's hard to understand how much are we lacking, is it just the compute and thinking strategies / parallel chains, or something specific architecturally. But I feel there's value there and I haven't seen anything like it available so far.

Did open source phones win? No, iPhone is pretty dominant.

Did open source operating systems win? No, MacOS/Windows are pretty dominant.

Does open source... cloud hosting, social media, ride sharing apps, you name it win? Not in my experience?

While it is not at all practical to train an LLM with tens or hundreds of billions of parameters on hobbyists hardware, what if there are other architectures that perform just as well but are easier to train by 1000 volunteers?

I always wondered if 1000 1M parameter models fine-tuned to specific tasks with a small router could perform as well as 100B models.

And I know this is roughly how MoE works, but current MoE models still require training the model as a whole, and big players don’t have an incentive to change that.

But OpenSource community does…

At d5s.tech we are recreating the layers built on top of models, working on dogfooding our own product to run a large chunk of the company.

I feel extremely strongly that a future in which most companies depend on one or two large AI-megacorps is going to lead to excessive rent seeking sooner or later.

I remain positive that the long term steady state will consist of proprietary models, -but- with open source AI models statistically close.

If compute keeps growing the relative cost of training current frontier models will decrease. An open source Fable/Mythos model simply seems inevitable.

I think models will be a commodity sooner rather than later. This whole race doesnt matter. First mover advantage is real, but over enough time it wont matter.

Everybody who understands the technical problems is proposing a government fix. There is another option, foundations / NGOs could do this. Of course, openai has shown how quickly that can pivot into something completely different.