Oddly enough, I've read that the clay was originally re-usable, and the ones that are preserved were baked in accidental fires.

Kind of a weird opposite meaning of book-burning.

Even if you consider something like "build a bridge", that classic comparison of software and traditional engineer, all the components of said bridge have been created by someone else with particular expertise. The civil engineer who designed where the iron beams will go played no part in the metallurgy of those beams.

Milton Friedman put it best, in "I, Pencil". Everything around you is the result of hundreds to thousands of economic actors, loosely but robustly coordinated in a way to get you what you actually want at as low of a price as you can get away with. The complexity of the supply chain is the price of admission to the game.

https://youtu.be/67tHtpac5ws?si=eZk_5K32gL4PxDgv

Yes and no.

The physical world is bound by rules that are unchanging (more-or-less). Above this layer we’ve also devised and agreed upon standards that remain unchanging, though it’s regional: Voltage, screw/bolt sizes, tolerance levels, materials, material measurements, etc.

At this layer, we’ve commoditized and standardized because it’s useful: It makes the components cost-effective and predictable.

In software and computing, I can only think of the low-level networking standard that remain stable. And even that has to be reinvented somewhat for each OS or each nee language.

Everything else seems to be reinvented or rewritten, and then versioned.

Imagine having to upgrade your nuts and bolts in your car to v3.01 or lose support?

I'm not arguing against component-based architectures. I'm saying we're over-engineering, and it shows. Even toasters are less maintainable than they used to be.

Ingredients in the cookies? Yes. 100? No.

(Not GP, but:)

LLMs' initial training is specifically for token-prediction.

However, this doesn't mean that what they end up doing is specifically token-prediction (except in the sense that anything that generates textual output can be described as doing token-prediction). Nor does it mean that the only things they can do are tasks most naturally described in terms of token-prediction.

For instance, suppose you successfully train something to predict the next token given input of the form "[lengthy number] x [lengthy number] = ", where "successfully" means that the system ends up able to predict correctly almost all the time even when the numbers are ones it hasn't seen before. How could it do that? Only by, in some sense, "learning to multiply". (I haven't checked but my hazy recollection is that somewhere around GPT-3.5 or GPT-4 LLMs went from not being able to do this at all to being able to do it fairly well on moderate-sized numbers.)

Or suppose you successfully train something to complete things of the form "The SHA256 hash of [lengthy string] is "; again, a system that could do that correctly would have to have, in some sense, "learned to implement SHA256". (I am pretty sure that today's LLMs cannot do this, though of course they might have learned to call out to a tool that can.)

If you successfully train something to complete things of the form "One grammatical English sentence whose SHA256 hash is [value] is " then that thing has to have "learned to break SHA256". (I am very sure that today's LLMs cannot do this and I think it enormously unlikely that any ever will be able to.)

If you successfully train something to complete things of the form "The complete source code for a program written in idiomatic Rust that does [difficult task] is " then that thing has to have "learned to write code in Rust". (Today's LLMs can kinda do some tasks like this, and there are a lot of people yelling at one another about just how much they can do.)

That is: some token-prediction tasks can only be accomplished by doing things that we would not normally think of as being about token prediction. This is essentially the point of the "Turing test".

For the avoidance of doubt, I am making no particular claims (beyond the illustrative ones explicitly made above) about what if anything today's LLMs, or plausible near-future LLMs, or other further-future AI systems, are able to do that goes beyond what we would normally think of as token prediction. The point is that whether or not today's LLMs are "just stochastic parrots" in some useful sense, it doesn't follow from the fact that they are trained on token-prediction that that's all they are.