Posted by theanonymousone 3 hours ago
GLENDOWE: I can call spirits from the vasty deep.
HOTSPUR: Why, so can I, or so can any man, but will they come when you do call for them?
I can interpret weights too, but will that interpretation be correct?
Chances are much higher than a model can figure out what the patterns are, decipher them, and retain that knowledge in a meaningful way it can reason about. Me and you might take years or months to do all of that on our own, with a "map" of the weight we might be able to discuss it, but not as effortlessly as an LLM.
Honestly, I had told a few different colleagues that reading a model is probably not that exclusive to frontier models. It's probably yet another reason OpenAI / Anthropic don't release any big open weight models, but thankfully Google and Microsoft release reasonably sized models. Imagine a new GPT open weight release, it would get dissected drastically.
Isn't that what this whole technology paradigm is about? How is it surprising?
Granted, it is cool. But I see so much of "look at this cool thing I did with LLMs" and honestly... we need more structured text, more explicit documentation, more ergonomic and intuitive CRUD applications.
The amount of "look at cool things" is growing way faster than before and they are all sort of low effort now because the "agentic" LLM-VM looping apparatus can do a lot of heavy lifting and run with our prompts.
And I mean, if one has jargon, knowledge, and intention, they can also leverage that into great structured text, ergonomic and intuitive CRUD apps etc, all of which add great value to our mundane pursuits of applied research, software products and services and so on.
Honestly, I begin to think the problem is people dazzling themselves instead of doing the housekeeping that is the bulk of life. Technology choice is entirely secondary.
No, not at all. I recommend you learn some of the basic linear algebra about how these things work before engaging in wild and completely nonsensical speculation.
Are you sure you have read and understood his point?
It seem quite possible that the flattened-model-weights ingested into a model have a much greater fidelity of representation in a similar model's KV cache than they do in a human's working memory, and that being able to loop and digest and chunk makes it unsurprising that models can excel at this type of task.
I think that's the point the person is making. Do you disagree? If so, why?
There is nothing obvious or expected about weights being easily explainable or being enough to explain, because the weights are static: you need to look at the inputs, the activations, and the weights all together, you can never understand a model with the weights alone, unless it is a trivial model. The person isn't making any coherent points at all.
There are also deep reasons to expect hard limits in intrepretability in general: https://www.seangoedecke.com/alignment-is-impossible/#why-me....
EDIT:
> It seem quite possible that the flattened-model-weights ingested into a model have a much greater fidelity of representation in a similar model's KV cache than they do in a human's working memory,
Bringing up the KV cache is a sign you don't understand what you are talking about here. The KV cache is a computational efficiency thing and has no impact on how these models actually work (and isn't needed at all). The vague analogy to human working memory also just takes the cake here.
> and that being able to loop and digest and chunk makes it unsurprising that models can excel at this type of task.
This is how it does... all tasks.
In 1 Samuel, King Saul visits a medium in Endor in order to speak to the (dead) prophet Samuel. Human dead go to Shoel (in familiar terms, it is similar to the Greek underworld) in the Old Testament. Later on in the New Testament, the idea of a really really bad place is introduced in the form of Tartarus, which is specifically where the rebelling angels were sent. But that is not the same place human dead go.
The idea that everything "not Heaven" must be Hell is an invention, caused mostly by dogma that rose from bad translations of the Greek and Hebrew in the middle ages (for example, King James - itself mostly correcting minor errors of the older Bishop's Bible - refers to all of it as Hell even though the original text most inarguably refers to different places different beings go).
In TFA case, given that they were able to guess what the weights did (which the researchers knew in advance and could cross-check) and to even tweak and improve them, the answer is yes.
Why do this without testing? Why would you assume that?
This thing can accomplish the same results as the frontier models do, if you are willing to play a role of an higher level architect or decision maker. Guide it a bit here and there like a tech lead and you get everything done.
The money you pay for frontier models is exactly for outsourcing that high level direction work to agents.
Qwen3.8-Flash_Next should be in the running for that title as well. It's astonishingly good. I use both of them. If I need more memory on the machine for other things I run 27B, If I can get by with much less RAM I run QFN (takes about 102GB on my Strix Halo box with 262,144 context). QFN runs quite a bit faster on my hardware - about 50t/s. 27B with MTP starts out around 22t/s but degrades over time as the context fills up. Both are great models, but I'd give QFN a bit of an edge (at least for the coding projects I've worked on). But both feel on par with Claude Opus 4.6 - 4.8.
HF is telling me that IQ4-XS is the largest one likely to fit without much contortion.
Unfortunately it still hallucinates, which limits it's usefulness for real world tasks. :(
Curious, why/how does that help? Or do you have some niche needs?
As such, if you want to replicate big model behaviour, you need to basically do context augumentation. How you do that is up to you for your own needs.
Why? Most existing harnesses let you point them at whatever API you want. I'm currently have a great time with Juggler.
How do you validate and verify that outcome?
Or does Qwen-3.8-27B do the validation and verification for you as well? If so, how do you ensure that's correct?
Personally Qwen3.8-27b has not proven itself to me for the projects I’ve tried to use it for. It doesn’t seem to have as much encoded knowledge about the native Mac development stack.
I think over time we'll realize this is the way forward, you want the smartest model with the least amount of embedded knowledge and you provide the knowledge externally or merge it into the weights for the domain you're interested in. Then you don't have to train a ginormous model every 6 months to pull in new information.
A RAG solution would make time to first token awful.
What frontier models are you using? I'm literally sitting with Qwen 3.8 27B locally and trying my damnest to get it to work at least 80% of the quality Astra et al have, and can't get close to it. It keeps being so verbose, regardless of inference parameters, system prompts, reasoning effort, and keeps going on tangents. I'm running the BF16 weights via vLLM, can't imagine how much worse the experience must be with the quantized weights people run with the average consumer GPU.
Even with tight guiding, as I guess you could say I do more "AI pair-programming" than vibecoding, it just loses the plot to often and waste time, enough that it's clearly not an alternative to today's frontier models.
The only reason Qwen is more popular is because it can run decently fast on MLX, while Gemma needs a discrete GPU with 24gb of ram to run at usable speeds.
Also, theoretically speaking, any small model can be as good as a big model if all they have to do is summarize text. Back before agents were a thing, we had prototype coders running of local inference models from Mistral for ai dev. The workflow was to generate llm summaries of the codebase, and a summary of changes to be made. Then we would grab search results from the web, feed it to an llm to summarize, and then finally combine all of that context into a final inference run which would generate code, then the codebase would be run automatically, any errors would be llm analyzed, and an llm would generate the next change to be made in the above cycle.
Nowdays, we can do that with pi coding agent and Gemma4:31b. But we still have plenty of custom harnesses for things.
I don't understand why megakernels haven't become the default choice. The speed advantage is crazy. I made a few changes to the openjet megakernel to run in a dual gpu (older gen, not blackwell) box and doubled my token generation speed on qwen3.8-27B (4-bit though).
I keep debating adding a third matching gpu to see if I can push the speed a bit more and move back to the 8-bit quant, but I'm stuck on the idea of just building a Blackwell gpu system despite the cost factor. Plus a third gpu is going to going to be weird for the tensor split and P2P data syncs and my workstation is old with 3rd gen PCIe etc etc.
There are a lot of known neurons in the older Qwen models that are long known to induce behaviour changes (see evil Qwen by changing one weight).
My question boils down to how much is trained vs being able to actually understand the weights. Presumably a same gen model isn’t trained on any sources dissecting that generation. And how much do longer known conceits like steering vectors, J spaces, LoRAs play into this?
This doesn’t explain the game training cases, which are very impressive to me.
The future is going to be wild.
That sounds a bit like biology; maybe AI will become like that.
Some poor-ass professor -- fighting tooth and nail for tenure, or at least some form of stability from all-powerful admin staff -- can't keep themselves from trying to figure out who's "Dominant" or "Alpha" looking at a bunch of apes or bugs. When in reality the network isn't directed, but a mesh of arbitrary weights, vectors, and dimensionality.
Perhaps if ancient China had beat us to Enlightenment and Industrial Revolution we would be seeing scientists talking about how respectful mammals are of their ancestors.
Samuel Butler was a character.
[1] - https://nitter.netbub.com/AlexKrentsel/status/21173766837246...
I hope you at least switched off all network access, in case the model cheats and reads HN from yesterday.
I think your use case of incorporating novel data probably calls for training, but I'm sure there are use cases where you can tweak the weights to get what you want (say I wanted to tweak the model to be less sycophantic, or found a way to shrink it by removing weights associated with a language that I don't ever use)
You can optimize a model by randomly changing its weights and validating results on your task. A monkey can type a book given enough attempts.
For training you would need to hook it up to a LoRA training harness, identify the looping weight direction, and adjust the weights accordingly. I've not (yet) looked into how to do this.
