However, I'm rather curious: Does this improvement come from the enhancement of the model's reasoning ability itself, or is it mainly due to the fixed issue of the model repeatedly reasoning and being unable to conclude normally?

"i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models."

It output an ASCII white circle with a 75% confidence, as opposed to the model tested getting a higher confidence but drawing a circle with a filled in inside.

I'm not sure what I make of these results but does that make me less impressed with Astra? Or just that the prompt was quite good? I just can't tell personally at the moment (it was a quick experiment and I haven't thought much on it)

I will say that it's pretty wild to see all the different hypotheses tested.

[1] https://github.com/RikiyaT/lm_understand-logs/blob/main/exp8...

https://www.lesswrong.com/posts/5F6ncBfjh2Bxnm6CJ/base64benc...

[1]: https://arxiv.org/abs/2106.00958

Unless Astra was trained on other networks, how is it able to reason about the weights?

Does this mean that OpenAI has been including networks in its training sets, maybe in an attempt to RSI? I wonder which other networks are able to ingest other networks and reason about their weights?