Posted by csmantle 4 hours ago
> Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.
Love the spirit :)
Not only can it solve the halting problem, but also the ignition problem!
Anyways, what I’m trying to say is I want evil qwen to be my CPU.
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?
In a few words, they found a way to map residual stream activations (so, a vector) into a few interpretable sparse concepts. They also had the converse maps, that translate concepts into activations.
Golden Gate claude was obtained by artificially increasing the activations in the direction of the 'Golden Gate' concept. It was not neuron surgery.
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
If you are actually good at linear algebra, it's doable in an afternoon.
They are fantastically simple in structure, all the intelligence is in the massive piles of data in them, not their structure. Then there is of course a lot of room in optimization once the basics are done.
If interested, maybe start here: https://www.deeplearning.ai/courses/how-transformer-llms-wor...
So like... can't you use a computer as a computer without doing all that roundabout stuff? Like if you want to integrate a LLM with hard computation there's probably a more efficient way to do that.
Or is this building towards "but can it run DOOM?" :)
Given that models learn to use residual connections optimally, it can’t be hard for them to steer around the fixed computation layer when it’s not needed.
I’m sure giant models invest a few layers in reproducing these functions, but surely smaller models can benefit?
They implemented wasm in transformer with cpu decoding for the wasm part.
A fun and important thought experiment about the the space of nerual networks vs the image of current training methods.
Most discourse conflates neural networks and trainable neural networks.
there's no free lunch here ever hence the cursed