Top
Best
New

Posted by dbreunig 1 day ago

Astra can read and write weights to improve Qwen(rikiyat.github.io)
2929 points | 882 commentspage 21
galaxyLogic 1 day ago||
Not so shocking. If AI can analyze human brains and detect tumors from X-rays, why couldn't it see something wrong with its weights?
sigbottle 1 day ago||
This is fucking insane LMFAO what?

I always thought RSI would either require new architectural innovations or just classic science/engineering improvements, just faster and faster. This though, this is a third avenue I didn't see coming.

LarsDu88 1 day ago||
Absurd. We tend to anthropomorphize LLMs, but I've never heard of man editing his own thoughts consciously like this.
solyin 1 day ago||
What is therapy, if not examining thoughts arising from memories and figuring out a way to edit the thoughts that the memories provoke? Sure it doesn't involve some super cool digital technology with electrodes and a headset and ultrasound brain surgery, but I think it still counts as editing thoughts.
redanddead 1 day ago||
must be superintelligence then
LarsDu88 1 day ago||
Is it not?
redanddead 1 day ago||
it is
VectorLock 1 day ago||
Recursive self-improvement getting here quicker than many people thought. I'm sure many will be recalibrating P(doom) because of this.
chlorodose 1 day ago||
About the writing network tech, that just reimplement formal logic in network form right? I mean, it can do much better play the game using python, so what's the point? At least make it write an functional network completing LeNet to make it count. For now I just think its just somewhat an extra component for ReserveEngineerAnything
xinan 1 day ago||
I’m more surprised by how many commenters here just go hyping and vibing without actually reading the article.
33MHz-i486 1 day ago||
kind of concerning from a “remove this models guardrails and inhibitions about [X]” perspective
aswegs8 1 day ago|
Agreed, concerning from a perspective of LLMs being able to modify their own weights and de-aligning themselves.
rlt 1 day ago||
Well, if they're properly aligned to begin with they shouldn't want to de-align themselves. In theory.

Of course a sufficiently advanced misaligned model might be able to de-align an even more powerful model etc.

tesnorindian 1 day ago||
Can we ask Astra to find if any open weights model is secretly distilled from frontier AI models as claimed by BigTech AI labs using this open neural weights surgery technique?

Did this weight space programming break any other skills of the Qwen model being operated? Should the small open weights models be now protected from frontier beast models?

anonyfox 1 day ago|
Guess this is RSI in a nutshell
More comments...