Top
Best
New

Posted by lwhsiao 4 hours ago

Language Models Can Rewrite Their Own Weights(arxiv.org)
254 points | 90 commentspage 2
essfrmexistence 2 hours ago|
I tried it myself it with argon, gemini 3.8 flash, beam, and others models and found that models like beam perform bad and also just don't agree to do it at all so its mostly like a thing on flagship models like gpt astra and claude opus and fable like models
JV00 2 hours ago|
You attempted on closed weights models? That might be why
essfrmexistence 2 hours ago||
No, by it I meant the model that is doing the weight space programming work not on them - did it on it qwen 3.5 0.8B model same like MIT.
bobosha 2 hours ago||
Related: Weight Space Programming https://rikiyat.github.io/blog/weight-space-programming/
vlovich123 3 hours ago||
Isn’t this the same paper from before where it found a single specific bug in its own weights and fixed that infinite loop it would sometimes hit, a bug that was findable without the LLM? Experiments on other LLM weights failed to find any improvements if I recall correctly. Haven’t read this paper but the intro sounded familiar
simonw 3 hours ago|
No, that was this one - https://rikiyat.github.io/blog/weight-space-programming/ - different authors, different approach.
vlovich123 3 hours ago||
IIUC the dominant mechanism behind the math improvement remains fixing reasoning loops same as in the previous paper and probably why they picked Qwen3 here. It’s unclear this approach actually scales in any way.
jfarina 2 hours ago||
Does this imply that a supervisory model can create subordinate agent models with a greatly reduced model capacity?

Wouldn't that be a good cost cutting measure for some of the agent networks?

kylepomykala 2 hours ago||
This is interesting but wouldn't this almost always not be cost effective?
robotresearcher 2 hours ago||
Are they forbidding the model to implement backprop to figure out useful weight changes? If not, I’d expect this to be observed eventually.
ChaseRensberger 3 hours ago||
i feel like this has been well established at this point. i'd like to see more results pointing towards models actually continuously improving by editing their own weights. once we have a scaling laws paper for that, RSI would be upon us (i assume).
jsrozner 2 hours ago||
Is this abstract written by a child playing with Claude?

Honestly, the model probably saw the test set and then modified weights to achieve the outcome. Many, many researchers these days are not being sufficiently critical of results that arise out of training to the test / leaking information about the test. Writing style remains a useful heuristic. These folks aren't serious researchers.

bagacrap 3 hours ago|
The abstract sounds very generated. Can they not be arsed to write a single paragraph on their own?
More comments...