I guess this makes it extremely easy to add vulnerabilities

Wonder if it can do this for 3d cad etc ? With enough fine tuning of small models I’m not sure how much they’re distinguishable from experts if you put a router with like jev in front ?

Could you use Astra to create specialized models? Like if I want a model that just writes React, could Astra pull out the subnetworks that know how to write React code?

I have a hard time believing that this is some sort of emergent behavior and not the result of intentional training to scale AI research.

I wouldn't be surprised if Anthropic and OpenAI restrict these weight-related queries the same way they limiting cybersecurity queries.

Amazing work. I wonder if it could rewrite an existing neural net to be cleaner and more interpretable?

(When they say "Writing a Go algorithm," they mean the board game, not the language.)

Sorry to be obtuse, and I lvoe AI, but wasnt this a bug fix on a small model?

This is impressive! It'd be both impressive and scary if Astra could edit a Qwen so that that Qwen itself could do this on another Qwen.

This is kinda surprising, but..

"What does it feel like to be a neuronal network?"

Maybe feeding weights directly to a LLM is like interfacing two brains? You get the idea..?

This shouldn’t be surprising. Weights are in the training set and are necessarily structured. We should be asking the model to extrapolate on the structure.