Posted by louiereederson 11 hours ago
https://x.com/JensenHuang/status/2080643682408321103, https://xcancel.com/JensenHuang/status/2080643682408321103
https://www.wired.com/story/silicon-valley-is-completely-div..., https://archive.ph/LAkVI
And besides, it seems to me like the ethical choice. Given how these models are, in some very real sense, mechanical plagiators, built on the generosity of creators past and present, some of them now in danger of being replaced by the machine. I think the least the labs can do is open these models up. These and other such considerations were the reason OpenAI started with that name. Of course, it was questionable that those ideals would survive the encounter with generational wealth. Just look up what the founders of Google were saying about advertising when they were two students tinkering at an as of yet unproven tech. Same thing for OpenAI, self-interest speaks that much louder when there's real money on the table.
It just boggles the mind that people now make excuses for their all-too-predictable about-turn.
Perhaps worth not calling them "labs". Are they not (for-profit) companies?
There's nothing "open" about China. Google, meta, openai, etc all blocked. Go visit and see how it goes when you try to access your gmail or open facebook. Try to use chatgpt. Try to get citizenship and see how that goes. China blocks many western companies with their great firewall and force internal similar products. This is smart, China wants to prioritize their own.
To me, "open these models up. " must mean provide all the data and supporting documentation required to reproduce the model. That would be "open". Postgres is open because you can download all the data and supporting documentation and reproduce the binary yourself. However, being able to only download a postgres binary would make it no longer open.
Additional training on top of an open-weight model sounds analogous to writing mods for minecraft. You may change some behavior but that doesn't make minecraft "open".
1. You have the weights, so you can run the model yourself on your own hardware. 2. You have the weights, so you can do post-training and shift those weights for your own purposes. It's not the same as training the model, but for many people its fine as what we want is a quantization, or a fine tune, or to create hybrid models.
What they don't give you are the training data, and reproduction instructions but... the toolchain to create the software has never been a part of 'Open Source'.
Even though it feels like a huge loophole, it's technically open source if you deliver the source code, without having a compiler that's available so you force them to recreate the toolchain from scratch.
To me, though, a better analogy is to research science where you'll be happy when they give you the full result set they compiled even if you don't get the raw data which may have IP or privacy concerns, or their often poorly documented lab notes so you can actually reproduce.
What you want to do is run your own experiment, and get your own results... not duplicate theirs directly. Even if reproduction is your aim in science, being unable to reproduce without copious notes sometimes points out that the original experimental process must have been flawed.
LLMs have the same issue. The creation process isn't entirely well documented, and the raw data can't be released since although the company have the right to use certain sources, they can't transfer those rights to others.
Having a standardized training set is valuable, though.
Like, that's just a logical thing to call it. I don't believe anyone is making a judgement on the intelligence of the reader to call it "open weight" when it refers to weights that are openly available.
"Open source" would be a more appropriate term to describe a model which also includes the training source.
Instead a bunch of tech companies are gathering to try to stop OpenAI and Anthropic fear-bouncing the White House and the Republican Congress into giving them regulatory capture and repeating the mistakes they are making around RISC-V.
Those mistakes won't just entrench two companies, they will entrench the bigger-better-faster-more model (closed companies making ever bigger cloud-bound models) when it is abundantly clear that enormous progress can still be made on smaller, even desktop-bound models (where, due to distribution, open weights are essentially inevitable).
Regulatory capture that stops open weights work will also have impacts on local and on-device AI work, as well as on academic research.
Note: Just pointing out the comment intent and nothing else
nVidia did and they released good models – same with Meta, Microsoft, IBM, Mistral – all are signatories.
It would seem as if the community either isn't doing that or is relying on the Chinese to do that.
> Distillation ... reflects a long tradition of learning from, building upon, and improving existing technologies, a tradition that has helped drive innovation since the rise of the open-source software movement. By contrast, unlawful efforts to extract value from closed models raise legitimate concerns. Those concerns should be addressed through targeted legal and commercial frameworks rather than sweeping restrictions on techniques that play an important role in AI innovation.
Sounds like they are saying "please protect our IP theft" that created closed weight frontier models in case we arbitrarily decide to close our models. But don't get rid of distillations in general so that we can all also keep benefiting from open models. We don't want to lose the ability to benefit from the work of others as we launder IP into closed models.
Surprised Linux Foundation kept their name on it with that.
i've said this in other comments but the fact that these companies are trying to align with open-source when there's no "source" included with their models is pretty damning. I think it lays bare the absence of any kind of noble or righteous motive with respect to distillation.
Microsoft, NVIDIA, Meta, Palantir, IBM...They have all been actively hostile to open source for decades, and have a history of embracing it only when convenient and profitable.
Microsoft benefited from its close partnership with OpenAI for years right up until it went sour. Where was this enthusiasm for open weights then?
Meta was developing open models and then abandoned that effort in search for profits. Muse Spark is now fully closed.
All these companies have the resouces to train and release frontier open weights models today, but choose not to. So spare me the marketing and virtue signaling.