Posted by theanonymousone 5 hours ago
So i asked Qwen 3.8 27B two things:
1. What it is this network all about? I only gave it the ann.c code that implemented a simple artificial neural network library in C and the setup code that specified the network's shape but not its purpose.
Qwen 3.8 27B tried a bunch of stuff, taking the layers one by one, proding numbers through it, etc and finally figured out it is some sort of 5x5 to 2x2 filter, most likely working with pixels (i never mentioned pixels or graphics), though it wasn't certain about the exact purpose. It did rule out a bunch of uses like being a downscaler or classifier, from its behavior.
2. After i revealed what it was, i asked it if it can improve it but keep it 100% compatible with the existing program that used the weights file. And sure enough, it did improve it. I gave it the original training C code and dataset. It made a few experiments (and wrote its own training C code because it found mine too slow :-P) and ended up improving the weights. The difference[1] is small, but it basically eliminated some small noise that was always there and made some transitions less "halo-y". Of course for the purpose i wanted it (use it to upscale games in realtime) it isn't that great (too heavy) but it was learning material anyway. Though one bonus thing i noticed is that it upscales my random doodles much better than any Krita filter[2] :-P.
The above isn't Qwen 3.8 27B observing the weights directly and making sense out of them (ok actually it did seem to make some sense of the weights in the "#1" but i didn't dump the entire network to it - though thinking about it, it is only around 16384 floats so it might work). It just used a bunch of scripts it wrote in Python (and later C) but i still found it interesting to watch it go at it.
[0] https://mastodon.gamedev.place/@badsector/113765818432603610
[1] http://runtimeterror.com/pages/iv/images/a3d94ea369ff95ecf05...
[2] http://runtimeterror.com/pages/iv/images/4e29d027a8aa6e89db5...
> the 2x2 pixel grid that the input pixel should be replaced for a (width2)x(height2) upscale.
Out of curiosity i did apply it twice to the drawing i posted in my previous comment with a 4x bilinear upscale made in Krita[1]. TBH i think a dedicated 4x network would do a better job though even then, there is only so much a network can hallucinate to fill in details :-P.
[0] https://github.com/badsectoracula/gamescope/tree/neural-netw...
[1] http://runtimeterror.com/pages/iv/images/20b9558400bcf26511d...
The age of source code is definitely not going to end anytime soon if ever. But the future will involve incomprehensible machines editing other models in a "fairly certain" way, but we are going to need a lot of tooling to keep up with the process and not just take the model updates as "trust me bro".
It's a blog post which received feedback from collaborators.
It hasn't gone through the academic paper lifecycle ... the authors themselves refer to it as a post and as preliminary.
I feel like I'm taking crazy pills, but can we be precise when talking about work of this form? It started with referring to arXiv pre-prints as papers, and now we're at blog posts with an academic template as papers.
A paper is a fucking paper: something which has been vetted by the scientific community, with at least some form (often, a few rounds) of anonymous review from people outside your circle of collaborators
This is not there yet, and clearly so due to the various feedback from people (including the two pushback posts on HN) critiquing the lack of discussion about the existence of these techniques in mech interp, etc?
I called it a paper here because it's by people with an academic affiliation and is of comparable quality (if not better than) many of the other "papers" we see.
I also like it when a paper isn't a PDF. PDFs are a bad way of communicating information.
(I will note that the authors themselves didn't call it a paper, their own BibTeX reference calls it a "blog post":)
@misc{takehi2026weightspace,
title = {{Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}},
author = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},
year = {2026},
month = nov,
howpublished = {\url{https://rikiyat.github.io/blog/weight-space-programming/}},
note = {Blog post}
}They have the strength of one PDF being frozen & (mostly) treated online as immutable - compared to a webpage which might change, or a blog post which might be edited. If a PDF needs to be replaced there'll be a notice.
And the single-file nature of PDFs makes them more likely to sit in an archive not managed by the author (vs a blog) so they tend to live as long as the archive rather than the author's blog.
For something that needs to Be referenced by others those are useful properties - I'd argue they outweigh how good PDF is at communicating information.
But for (1, 2), just to point out, neither of your criteria would pass muster in the scientific communities who are ultimately responsible for vetting this work, right?
The ideal is: we don't accept papers at NeurIPS (or wherever) because of affiliation, and the badge of "quality" is bestowed by peer review, from other scientists.
> is of comparable quality (if not better than)
But the posts that you refer to in your own post point out several ways in which the quality is not good ...
But, in any case, I find it a bizarre choice to call it a paper when the authors themselves do not refer to it as such. We can agree to disagree, but just offering feedback.
I admit that I don't think formal "papers" deserve their elevated status in today's information economy.
I think they add a whole lot of ceremony that usually results in a format that's harder to understand.
So me calling this a "paper" was a little bit of a dig!
You, being of sound mind, tried the technique and drew reasonable conclusions … but just look at the original post for this blog post: there are a large number of responses by people who clearly don’t understand the technique, missed the experimental weaknesses, couldn’t place the post into the lineage of mech interp results that preceded it … and will leave the discussion with the wrong impression.
In your mind, whose fault is that? Is that that the unavoidable cost of a field, moving too rapidly to engage in peer review?
In its ideal form, the paper lifecycle is not a gate, but a sharpening stone.
I'm not convinced the alternative - delaying publication by 6 months, keeping the new information available only to a much smaller group until then - would have been better.
All of that said, I did just update my own post to include "The paper (OK, the authors classify is as a “blog post”, but it’s paper-shaped) found that frontier models..." inspired by this conversation.
From memory, yolo and roberta and ppo were initially rejected. After a quick google search, pagerank was initially rejected, word2vec got one weak and one strong reject in reviews, LSTM got rejected (90s but still), dropout, knowledge distillation, and many more such cases. All of these are extremely important, foundational and field defining papers that ended up on arxiv or other preprint places.
If that happened to those papers, there's 0 chance today that it'd be a good idea. The field is simply moving too fast, there's too much being published, and the "top venues" are simply swamped. Blog / preprint / carrier pigeon is ok atm, as long as someone can look at it, try it, and hopefully reproduce it.