Top
Best
New

Posted by lwhsiao 4 hours ago

Language Models Can Rewrite Their Own Weights(arxiv.org)
262 points | 91 commentspage 4
mistercheph 3 hours ago|
LOL, last month's news to try to save ANT?
tsunamifury 3 hours ago||
I’m sorry but isn’t this just over fitting based on conversational adaption. But worse since it’s self conversation? Resulting in the exact same problem 4o1 had: sycophancy? But again worse.

The issue isn’t training, it’s articulating real world high value outcomes. Something the SOTA labs have no idea what to do about since they are staffed by… not the best people.

Nevermark 3 hours ago|
> I’m sorry but isn’t this just over fitting based on conversational adaption. But worse since it’s self conversation? Resulting in the exact same problem 4o1 had: sycophancy? But again worse.

To me this comes across as word salad.

The edits are done based on reasoning, to accomplish a "target" improvement, that is itself reasoned and designed, taking into account a reasoned understanding of the existing functionality of the weights.

This is the furthest thing possible from some gradient optimization "over fitting" weights to tune a pedantic measure/target. That could be relegated to one-time greenfield-model bootstrapping.

Pastookh 1 hour ago||
[dead]
fwlr 3 hours ago||
[flagged]
tug2024 2 hours ago||
[dead]
zodwick 4 hours ago|
Not the RSI we wanted, bu the RSI we deserve