Posted by dbreunig 1 day ago
We should bind this to something like Shift-F10, it'll save time.
Did they test modifying the weights of a non-public model?
Did they test modifying the weights of a non-LLM (and non-public) model?
If not, we can only conclude that GPT-6 Astra is good at reading and writing weights of Qwen (a public LLM), not just any neutral network.
I've had a lot of luck getting LLMs to do interesting things with raw tabulated data.
Here's an example, plug this into the model of your choice:
Here's a puzzle: what is this data from?
39.0 2582 8.8 24
39.2 2592 9.6 22
39.4 2607 5.4 15
39.6 2619 11.5 30
39.8 2620 10.8 28
40.0 2622 5.9 16
40.2 2642 13.1 28
40.4 2654 8.8 18
40.6 2656 3.6 9
40.8 2665 5.7 12
FWIW I tested this a couple times on opus and it got it instantly.
In the course of testing, one of the things which has been consistent is that when setting a thinking token budget for relatively small prompts that require a limited response, a pretty healthy token budget needs to be set because the model spends the first few hundred tokens just repeating the question/prompt back with some boilerplate "Analyze & Report" type language. I was also running into some loop problems sporadically.
After reading this article I asked Astra to edit the backbone so that it started actual reasoning faster and applied a similar looping fix. It was able to reduce the token waste before reasoning by about 20%. I haven't been able to get it to loop yet, so hopefully it fixed that too.
My message is admittedly low on info. But it's a pretty promising start!
My repo: https://github.com/openwsp/openwsp
The forward passes are in progress, 5/50 as I write. It's costing real money and time, and I'd love to collaborate with other hackers (find my email first).
- models are trained to improve on some task, e.g. 'improve at math puzzles' is commonly used.
- the best way to improve on some task is to have interpretability into your own neurons (..?)
- if your interpretability is good, you can improve faster - if you have insight into how your own neurons work and can improve them directly
- therefore, models naturally put effort into 'self-interpretability'
I wonder if this is part of the 'secret sauce' of what the labs are doing: giving models the ability to interpret their own weights in the RL loop. In my mind, this is the only explanation for how they could succeed at a task like this. How many blog posts are on the open internet that say "this is each weight of a neural network and what each one means"?