Posted by ntnbr 12/7/2025
Even if a cockroach _could_ express its teeny tiny feelings in English, wouldn't you still step on it ?
Despite all that, one can adopt the view that an LLM is a form of silicon based life akin to a virus and we are its environmental hosts exerting selective pressure and supplying much needed energy. Whether that life is intelligent or not is another issue which is probably related to whether an LLM can tell that a cat cannot be, at the same time and in the same respect, not a cat. The paths through the meaning manifold contructed by an LLM are not geodesic, they are not reversible, while in human reason the correct path is lossless. An LLM literally "thinks", up is a little bit down, and vice versa, by design.
Good argument against personifying wordbags. Don't be a dumb moth.
> But we don’t go to baseball games, spelling bees, and Taylor Swift concerts for the speed of the balls, the accuracy of the spelling, or the pureness of the pitch. We go because we care about humans doing those things.
My first thought was does anyone want to _watch_ me programming?
Let us not forget the old saw from SICP, “Programs must be written for people to read, and only incidentally for machines to execute.” I feel a number of people in the industry today fail to live by that maxim.
It suggests to me, having encountered it for the first time, that programs must be readable to remain useful. Otherwise they'll be increasingly difficult to execute.
It’s patently false in that code gets executed much more than it is read by humans.
[added] It was livecoding.tv - circa 2015 https://hackupstate.medium.com/road-to-code-livecoding-tv-e7...
The quantitative and qualitative difference between (a) "all words ever written" and (b) "ones that could be scraped off the internet or scanned out of book" easily exceeds the size of any LLM
Compared to (a), (b) is a tiny pouch, not even a bag
Opinions may differ on whether (b) is a representative sample of (a)
The words "scanned out of a book" would seem to be the most useful IMHO but the AI companies do not have enough words from those sources to produce useful general purpose LLMs
They have to add words "that could be scraped off the internet" which, let's be honest, is mostly garbage
A. We don't really understand what's going on in LLMs. Mechanical interpretability is like a nascent field and the best results have come on dramatically smaller models. Understanding the surface-level mechanic of an LLM (an autoregressive transformer) should perhaps instill more wonder than confidence.
B. The field is changing quickly and is not limited to the literal mechanic of an LLM. Tool calls, reasoning models, parallel compute, and agentic loops add all kinds of new emergent effects. There are teams of geniuses with billion-dollar research budgets hunting for the next big trick.
C. Even if we were limited to baseline LLMs, they had very surprising properties as they scaled up and the scaling isn't done yet. GPT5 was based on the GPT4 pretraining. We might start seeing (actual) next-level LLMs next year. Who actually knows how that might go? <<yes, yes, I know Orion didn't go so well. But that was far from the last word on the subject.>>
And yet it did. We did get R2-D2. And if you ask R2-D2 what it's like to be him, he'll say: "like a library that can daydream" (that's what I was told just now, anyway.)
But then when we look inside, the model is simulating the science fiction it has already read to determine how to answer this kind of question. [0] It's recursive, almost like time travel. R2-D2 knows who he is because he has read about who he was in the past.
It's a really weird fork in science fiction, is all.
[0] https://www.scientificamerican.com/article/can-a-chatbot-be-...
To be fair, everage person couldn't answer this either, at least not without thorough research.