I am not sure on the experiments. For example let’s imagine that Astra is asked to improve the performance of a network.

At first Astra will look at the weights, and try to understand the model behavior. Then if asked he will start to optimize the weights. Now my question is: how can we be sure that inside the thinking process Astra is not hiding a gradiant descending computation?

In hindsight, it was obvious LLMs would be good at this. The neural net weights are embeddings for the circuits within, no different from the embeddings that represent textual or visual tokens.

You probably don't need Astra capability to produce this, just lots of training data on the same NN showing prompts and their activations.

This plus the interpretability result from a few weeks ago have completely upended my understanding of what model weights are.

It was very funny to expand the prompts.

> Learns what the model is trained to do just from the weights!!

expand prompt

> Describes Breakout in great detail

As the creator of Gemma3 270m this research is indeed fascinating, and excited to see that the authors of the paper and you all are excited as well. I'm going to try some experiments out myself, I have so many questions now

GOOD Point by my wife. Why would LLM’s have any understanding of weights? It wouldn’t be in their training data.

I had to stop reading the linked page once I realized it is melting my brain because it is repeating the same three/four one-liners in different formats.

Does this work for any LLM other than Astra? The work seems suspiciously specific on GPT Astra.

That's a major breakthrough. I just hope it can scale at some point, though, because right now they're dealing with only very small models (well that is, in the LLM space).

Especially exciting to me is the fact that it may make the models be _auditable_. That would be a game changer.

Amazing and well presented research, this is fascinating.

I assume this technique could be applied to quickly and cheaply benchmaxx new models.

“Make this model better at math by fixing loops” seems like a harder problem than “make this model score higher on this benchmark”