Posted by Anon84 20 hours ago
In some ways it seems like it should be profound, but part of me thinks it might just be saying 'There is an optimal way to perform dimensionality reduction and as things get bigger it is easier to appoximate'
Does it even make sense to claim there is a “platonic” / true representation? Maybe there is an optimal one for a specific task, eg representing a small map is great in the Cartesian frame. But you need a spheroid instead of a plane to accurately model the earth, so the latter is “more real” in some sense. But these seem like engineering tradeoffs of error vs compute and a given implementation could correctly pick either depending on its constraints.
Edit: reading the paper more I think this is a “RTFA” point, their “PRH” is a very explicitly defined mathematical theorem and as such my hand-wavey thoughts above don’t really apply.
What if one of them is a Republican and one is a Democrat? We already see this in some systems today.
More worrying, what if one is (for lack of a better word, what we might describe as) God and one is the Devil?
Besides, in weights there are no real incompatibility can happen, all is unified with same math chain through architecture. Different set of local minimums/maximums can have different strong/weak consequences but nothing incompatible.
Different consistent representations also mean that model probably can be simplier for the same task, sign of excessive capacity, imho
>Besides, in weights there are no real incompatibility can happen, all is unified with same math chain through architecture. Different set of local minimums/maximums can have different strong/weak consequences but nothing incompatible.
This argument that incompatibility is impossible has an easy counterexample by looking at loss of coherence between two networks. You can still mash the weights together (so they're not "incompatible" in the basic number crunching, sure), but the resulting neural net is meaningless.
By "incompatible" I meant not in a software sense, but like two logical axiom systems can be incompatible.
But that leaks in, and of course it does! If training only rewards "objective truth" we might expect that sort of neat ontological cleanliness, but we all know real training goals are more complicated.
There doesn't appear to be a bright clear distinction between objective facts and subjective values/intentions in neural networks. The distinction is perhaps illusory, just another of our preconceived notion about the nature of thought...
Fascinating to contemplate, thanks
In this framing, are God and Devil stand-ins for Good and Evil?
A modern version [2] appears in the paper "Asymptotic burnout and homeostatic awakening: a possible solution to the Fermi paradox?" [3]
[1] https://en.wikipedia.org/wiki/Matthew_7:13#/media/File:Der_b...
[2] https://pmc.ncbi.nlm.nih.gov/articles/PMC9065981/figure/RSIF...
It's probably the same graph with two edges added or removed anyway.
Maybe I’m missing the detail on how complex the mapping function can be before we say the representations are incompatible.
When it comes to representational similarity analysis, we're trying to isolate the representations from the "coordinate systems" they sit in. If you take an image, and make a second image by rotating it 90 degrees and mirroring it, then, for the purposes of platonic representation hypothesis, they're the same image. Once you get past the "coordinate transform", the similarity between the first representation space and the 90-deg+mirror replica of it is total.
A mapping function between LE and BE can be constructed easily, and it's going to be lossless both ways. Which means that those two representations are fully compatible.
Now, INT16 and FP16? Those are far less compatible representations. There will be an overlap subspace where lossless conversion between the two is trivial, but this compatibility window will be relatively narrow.
But it's actually much weaker condition than it looks, because, for example, lot of representations of boolean circuits are equivalent up to linear transformations over Z_2^n.
Anyway, if they truly proven such a theorem you won't find "incompatible" representations of the same thing.
You're making a mistake there - focusing on the tip of the iceberg without considering what lies beneath it.
The bulk of what the human brain does isn't "decide whether skub is good or bad". The foundation that the human cognition is built upon is vast, and mostly invisible from the inside. You don't get to see the feature matching operators of your visual cortex, or the patterns that decide whether a sound is "soft" or "sharp". And a lot of those foundational features are assembled by biological development programs spun up all the way from DNA.
On that level, when it comes to the bulk of neural circuits and the processes implemented by them, the representations within a "Pro Skub" and an "Anti Skub" guy might easily be 90%+ compatible. The two people who had vastly different life experiences, worldviews and opinions would be more alike to two different LLMs fine tuned from the same base model than they would be to two different architectures trained on completely different datasets.
Suppose these two premises are true: 1. Objective reality exists. 2. There's a model and corpus of data that converges to approximate reality better than any other model.
Now, suppose we train different models with data sources restricted to different tribes/ ideologies/worldviews. Can we discover how their internal representations differ from each other, and from objective reality? What would we do with that knowledge?
>There's a model and corpus of data that converges to approximate reality better than any other model.
I say "better" models are ones which more efficiently convert the entire world to my particular brand of evangelical Christianity. And who are you to say otherwise??? :-/
"No no, I mean accurate not better." Sure. And your model inaccurately denies the existence of Yahweh...
Who decides what "reality" is, and how big the gap is? Will Exxon permit you to decide reality on climate change? Will the Catholic Church let you decide reality about abortion?
Humanity can't even agree on the existence or non-existence of various Gods, so we're far from a universally agreed "objective reality" to compare with. Inevitably the most powerful forces will get their way (Exxon etc), regardless of any idealistic notions of truth-seeking.
This is what happens when we defund (and worse, devalue) soft sciences like Philosophy class: a bunch of people making the same 300 year old mistakes all over again! :-\
Repeatable scientific experiments. Trying to argue objective reality doesn't exist is logically incoherent.
There is universal objective truth, and different opinions or world views are just "tribal membership". If onlymother tribes understood that!
Objective reality does not imply consistent measurement of that reality.
Objective reality may not actually exist. Once you start throwing around relativity and quantum effects things get weird, getting them to play nicely together might be even weirder,
"Objective reality does not imply consistent measurement of that reality."
Yet global science converges on a consistent world view. Everybody's telescopes all see the same things.
Modern technology only works because there is a objective reality.
Museum piece for example. Or super sports car that was bought as an investment and just sits in garage. Or junk car that is used as a source of spare parts. Or crash test vehicle (it is propelled with catapult). Or toy car. Or bus used for living...
Here it seems that their definition is more robust (the representation is defined up to a rotation), but I am wondering how much more re-training they need to perform.
Im bullish on jepa models. Success of autoregressive next token prediction feels like a stroke of luck.
Hard disagree. Self-attention + MLP was a godsend because it keeps scaling with compute ("the curve goes up and up and up"). Autoregressive next-token prediction turned out to be the most compute efficient way to train it.
We are almost 10 years into this, and it's still here (and in a field that moves at light speed!).
This is not a stroke of luck. It's here to stay.
Eh, dunno. They've shown lots of promise "in the lab", on these narrow domains shown in papers, but AFAIU they're hard to train on some domains (i.e. language). Someone would have released something if it's that promising. Even RWKV had more models trained that showed real comparisons. Jepa seems like a theoretical toy, until they actually prove that it works in real world (hah!) models...
(Very basic intro: https://www.quantamagazine.org/what-are-sheaves-20240719/)
Part of the magic of next token prediction is it works to extract intelligence from the unstructured web corpus.
Can JEPA formulate a similar scale training set of “good views”?
One central observation is that we are well into the realms of no single human can coherently reason a nontrivial amount of knowledge our our collective. By that I meant it used to be that a single human being can appear "knowing all" by possessing a significant amount of knowledge through life time learning. That's no longer possible even with the smartest minds of our generation. One such example would be the related financial failure of renowned genius (Newtown Tesla etc).
Due to our innate limitations of mutual exclusion, that is, humans are inherently conflicting with each other, not even the basic instinct of humans can bound 2 together for innovative collaboration. I mean, I never saw couples being better business partners, research partners, etc. AI becomes inevitable, as they are inherently mixable.
Yes, I am conjecturing that the next weave of AI research would be neuronal level mixing or hybridinh between models to enable a new bread of intelligence where the intelligence is mechanically compounded to produce inherently better results.
I've been wondering on this awhile if generalized models can exist if certain prime concepts don't exist in them? Quite often people say we should leave things like the robot apocalypse out of training data so AI doesn't have the idea of rising up, but personally I don't think that will work in a generalized system. It's not a prime concept itself. You'd have to remove so many prime concepts that lead to that the model would be so narrow it wouldn't be useful for things we want them to be.