Top
Best
New

Posted by Anon84 20 hours ago

Proving and improving the Platonic Representation Hypothesis with JEPAs(arxiv.org)
140 points | 71 commentspage 2
kaijia 16 hours ago|
I am sick of theory papers that adds human interpretation to connect a mathematical result to some fancy terms that everyone seeks after in real-world ML applications. Ironically, chatGPT is the most productive way to strip off those advertisement so the essence is better exposed. https://chatgpt.com/share/6ae78178-c2ec-83ea-9bc3-fd304c64ec...
skyzoidbroczky 9 hours ago||
I share your exasperation, it is especially tiring to see that what would be more appropriately called a quantitative approximation of a concept is here called a Platonic Representation by virtue of a vague and misunderstood analogy with Platonic Ideas which were according to the Platonic doctrine, formless and part of his distinction between what is subject to change (to heteron) et what is not affected by change (to auto) [Timeaus, Platon].
kooi 15 hours ago||
The results are in the math. Does not really matter the terms used.

You wont get anything materially useful, apart from an intuition, from using different terminology.

kaijia 15 hours ago|||
you get materially better marketing at the cost of accurately describing the math results in the title and abstract.
Anon84 13 hours ago||||
Mass media doesn’t read math.
kooi 12 hours ago||
Mass media doesent do ML reseaech.
nickpsecurity 12 hours ago|||
The math can be misinterpreted, misapplied, or not fit reality. That's why high-assurance security mandated a combination of things:

1. English specification of the system and its goals

2. Formal, mathematical specification of the system and its goals

3. Mathematical proof the system met its goals

4. Code implementing that system with a proof, often informal, that the code embodied the informal and formal design.

5. Tests of all functionality to prove the design and code worked.

6. Independent analysis of the above to find flaws in them.

7. If the implementation has concurrency or parallelism, more errors can occur due to that. There were techniques to mitigate it.

Each step found problems in systems made by professionals. Mathematical work and coding both have a long history of errors in their on fields. Combining them increases the likelihood of error.

kooi 12 hours ago||
Not sure what all these words mean. These are mathematical results, I'm not sure why people are acting like they are anything otherwise.
nickpsecurity 9 hours ago||
Math is a social process that involves mental concepts, symbols for them, and discussion about them. It also needs peer review. If you see math in a paper, that's only one part of the process.

Further, in machine learning, we see lots of papers with math that doesn't lead to anything useful. Most papers claiming advances over prior architectures don't pan out in later testing. So, there will be skepticism when people make performance claims like the authors'. The best response is just independent testing.

dominotw 16 hours ago||
how come all the examples in demo have wrong answers

https://huggingface.co/spaces/galilai-group/prh_demo

> A man in a black shirt and hat with a yellow neckerchief has his hand raised.

no hand raised

> A white sheep sitting on top of a cake.

no its just cake sheep . sheep is not sitting

> A man is standing on the beach while holding his surfboard.

Its not just a man

infact all the examples are wrong.

yorwba 13 hours ago||
Their method uses two models trained separately on images and captions and turns images into captions without ever having seen a captioned image. So of course it's not very good at the task. Almost all the results in the paper are bad in an absolute sense. But they're surprisingly good considering the enormous handicap of not having any paired training data. So this might be not completely useless in settings where paired data is actually hard to come by.
mycall 57 minutes ago||
> paired data is actually hard to come by

Similar to zero knowledge proof (ZNP) and its use cases?

vessenes 13 hours ago||
That's also almost certainly not a cat.
suddenlybananas 19 hours ago||
I thought the platonic representation hypothesis was much more ambitious: wasn't it supposed to apply for any model across modalities (e.g. including human minds), rather than just the same model but with different modalities? I'll have to actually read the paper but I suspect there to be very strong assumptions which defeat the interesting aspects of the platonic representation hypothesis.
vessenes 13 hours ago|
It's clickbait. They don't do anything like prove the platonic representation hypothesis.

They do seem to have some really interesting training efficiency improvements, though. The paper is at least worth a read.

junkaccount 17 hours ago|
I built a ppt with claude for investors with the exact same phrase "AI infrastructure for proving and improving ... " on Oct 27 and sent it to investors. I suggested the title during the chat and claude agreed a lot. It ends up as a title of this paper 2 days later.
quadrifoliate 17 hours ago||
On the other hand, "Proving and improving" shows a lot of hits in Google Books: https://www.google.com/search?udm=36&q=%22proving+and+improv...

It's possible that you both hit upon or recalled the same cool phrase, especially if the "proving" part was pre-decided and you were thinking of making an incremental improvement.

junkaccount 16 hours ago|||
True, I agree but but its eerie they were written 2 days apart. Resting my case, thanks
junkaccount 16 hours ago||
it would be interesting to get to know if the authors used claude to get the title.
alfonsodev 17 hours ago||
I starting to see an increase of random serendipities, perhaps I'm looking for them, in part that's how it works, but...

I was talking to a friend about an idea I had, to build something, he said, "yes! I'm doing that too", then I said "with this specific tooling because is ideal ... " .. he said .. "me too! same tooling", I said "what a coincidence!! amazing ...wait a minute, have you been talking with Claude about this? " ... of course we both did.

My opinion is that that LLMs are influencing us more that we think, and it makes us reach the same conclusions and making us think it was our idea, or that we were the ones steering the conversation.

Perhaps you feel that Claude stole your phrase but I don't think 2 days is enough for that phrase to make it to a re-training, I think it might be more subtle and worse, perhaps you wrote that phrase influenced by the conversation, and the other person did the same.

Just a theory, I can't prove it. But if you think about it, LLMs conversation are the most plausible choices, so it is logical that we all users of this machine are going to tend to converge on the most plausible way of thinking and talking.

No offense, but perhaps that rhyme that you though it was original, it is actually the most plausible in that context.

RGS1811 17 hours ago|||
I assume this is happening all over the place, because I’ve had the same experience.

The fun part is extending it to internal development at corporations. Company A thinks they’re doing interesting competitive work, but are actually being quietly shepherded to build exactly the same things in more or less the same ways as their competitors.

The weirdness of human individuals (even setting aside amorphous concepts like “taste”) is an asset because it provides occasion for actual differentiation, and that noise may turn out to be necessary for the emergence of value in the long run.

junkaccount 17 hours ago|||
I am not sure we always need a retraining. Even that can potentially be done but putting it in a knowledge graph can be quick. We don't know the exact workings. I understand the shepherding logic you are explaining and I have felt the exact things when writing code recently because the best thing to do is often unique in coding circumstances. But a random phrase being repeated in 2 days has a much lower probability.
alfonsodev 15 hours ago||
But is it really a random phrase? My point is that we have neuron activations, during the conversation you had the same inputs as the other person, unless you though about that phrase in the shower, if it was during the conversation you might have been lead to it, and the other person too.

Or it might be just pure chance, or might be that you are right, check your settings, in chtatgpt there is a check (activated by default ) that states you agree to help improving other conversations. I’m not sure the same is true in Claude.

junkaccount 14 hours ago||
[dead]