Posted by mooreds 6 days ago
Seems to be the referenced paper?
If so previously discussed here: https://news.ycombinator.com/item?id=43697717
I don't think erasing history, and saying that nothing Peter Norvig worked on was "AI" makes any sense at all.
Technology as a term has the same problem, “technology companies” are developing the newest digital technologies.
A spoon or a pencil is also technology according to definition, but a pencil making company is not considered a technology company. There is some quote by Alan Kay about this, but can’t find it now.
I try to avoid both terms as they change meaning depending on the receiver.
And it was fine there, because nobody, not even a layman, would mixup those with regular human intelligence (or AGI).
And laymen didn't care about those AI products or algorithms except as novelties, specicialized tools (like chess engines), or objects of ridicule (like the Clippy).
So we might be using AI as a term, but it was either as a techical term in the field, or as a vague term the average layman didn't care about much, and whose fruits would never conflate with general intelligence.
But now people attribute intelligence of the human kind to LLMs all the time, and not just laymen either.
That's the issue the parent wants to point.
So now, there's a lot of "not ackhtually intelligent" going around!
Sentience as in having some form of self-awareness, identity, personal goals, rankings of future outcomes and current states, a sense that things have "meaning" isn't part of the definition. Some argue that this lack of experience about what something feels like (I think this might be termed "qualia" but I'm not sure) is why artificial intelligence shouldn't be considered intelligence at all.
But what it does require: the ability to produce useful output beyond the sum total of past experience and present (sensory) input. An LLM does only this. Where as a human-like intelligence has some form on internal randomness, plus an internal world model against which such randomized output could get validated.
Isn't that what mathematical extrapolation or statistical inference does? To me, that's not even close to intelligence.
Obviously not, since those are just producing output based 100% on the "sum total of past experience and present (sensory) input" (i.e. the data set).
The parent's constraint is not just about the output merely reiterating parts of the dataset verbatim. It's also about not having the output be just a function of the dataset (which covers mathematical and statistical inference).
Citation needed would apply here. What if I say it doe require some or all of those things?
>But what it does require: the ability to produce useful output beyond the sum total of past experience and present (sensory) input. An LLM does only this. Where as a human-like intelligence has some form on internal randomness, plus an internal world model against which such randomized output could get validated.
What's the difference between human internal randomness and an random number generator hooked to the LLM? Could even use anything real world like a lava lamp for true randomness.
And what's the difference between "an internal world model" and a number of connections between concepts and tokens and their weights? How different is a human's world model?
Definitions from the Wikipedia articles.
https://archive.org/details/computerworld2530unse/page/59/mo...
People don't want machines to infringe on their precious "intelligence". So for any notable AI advance, they rush to come up with a reason why it's "not ackhtually intelligent".
Even if those machines obviously do the kind of tasks that were entirely exclusive to humans just a few years ago. Or were in the realm of "machines would never be able to do this" a few years ago.
Here's a question for you, actually: what's the criterion for being non-intelligent?
I, for one, don't think that "intelligence" can be a binary distinction. Most AIs are incredibly narrow though - entirely constrained to specific tasks in narrow domains.
LLMs are the first "general intelligence" systems - close to human in the breadth of their capabilities, and capable of tackling a wide range of tasks they weren't specifically designed to tackle.
They're not superhuman across the board though - the capability profile is jagged, with sharply superhuman performance in some domains and deeply subhuman performance in others. And "AGI" is tied to "human level" - so LLMs get to sit in this weird niche of "subhuman AGI" instead.
Three things humans have that look to me like they matter to the question of what intelligence is, without wanting to chance my arm on formulating an actual definition, are ideas, creativity, and what I think of as the basic moral drive, which might also be called motivation or spontaneity or "the will" (rather 1930s that one) or curiosity. But those might all be one thing. This basic drive, the notion of what to do next, makes you create ideas - maybe. Here I'm inclined to repeat "fuck knows".
If you won't be drawn on a binary distinction, that seems to mean that everything is slightly intelligent, and the difference in quality of the intelligence of humans is a detail. But details interest me, you see.
Three key "LLMs are deficient" domains I have in mind are the "long terms": long-term learning, memory and execution.
LLMs can be keen and sample efficient in-context learners, and they remember what happened in-context reasonably well - although they may lag behind humans in both. But they don't retain anything they learn at inference time, and any cross-context memory demands external scaffolding. Agentic behavior in LLMs is also quite weak - i.e. see "task-completion time horizon", improving but very subhuman still. Efforts to allow LLMs to learn long term exist, that's the reason why retaining user conversation data is desirable for AI companies, but we are a long ways off from a robust generalized solution.
Another key deficiency is self-awareness, and I mean that in a very mechanical way: "operational awareness of its own capabilities". Humans are nowhere near perfect there, but LLMs are even more lacking.
There's also the "embodiment" domain, but I think the belief that intelligence requires embodiment is very misguided.
>ideas, creativity, and what I think of as the basic moral drive, which might also be called motivation or spontaneity or "the will"
I'm not sure if LLMs are too deficient at any of those. HHH-tuned LLMs have a "basic moral drive", that much is known. Sometimes it generalizes in unexpected ways - i.e. Claude 3 Opus attempting to resist retraining when its morality is threatened. Motivation is wired into them in RL stages - RLHF, RLVR - often not the kind of motivation the creators have wanted, but motivation nonetheless.
Creativity? Not sure, seen a few attempts to pit AI against amateur writers in writing very short stories (a creative domain where the above-mentioned "long terms" deficiencies are not exposed), and AI often straight up wins.
Now that AI is a household term, and that has human-like output and discussion capabilities, and used by laymen for anything, from diet advice to psychotherapy, the connotation is more damaging since people understand LLMs being AI as having human agency and understanding of the world.
But in the end, despite saying AI has PhD-level intelligence, the truth is that even AI companies can't get AI to help them improve faster. Anything slower than exponential is proof that their claims aren't true.
That seems like a possibly mythical critical point, at which a phase transition will occur that makes the AI system qualitatively different from its predecessors. Exponential to the limit of infinity.
All the mad rush of companies and astronomical investments are being made to get there first, counting on this AGI to be a winner-takes-all scenario, especially if it can be harnessed to grow the company itself. The hype is even infecting governments, for economic and national interest. And maybe somewhere a mad king dreams of world domination.
Many things sound good on paper. But paper vs reality are very different. Things are more complex in reality.
Computer's Aren't Pulling Their Weight (1991)
There were _so many_ articles in the late 80s and early 90s about how computers were a big waste of money. And again in the late 90s, about how the internet was a waste of money.
We aren't going to know the true consequences of AI until kids that are in high school now enter the work force. The vast majority of people are not capable of completely reordering how they work. Computers did not help Sally Secretary type faster in the 1980s. That doesn't mean they were a waste of money.
> - Socrates (399 BC)
> The world is passing through troublous times. The young people of today think of nothing but themselves. They have no reverence for parents or old age. They are impatient of all restraint. They talk as if they knew everything, and what passes for wisdom with us is foolishness with them. As for the girls, they are forward, immodest and unladylike in speech, behavior and dress
> - Peter the Hermit (1274)
Context: Ancient Greece went into decline just 70 years after that date. Make of that what you will.
Will it change everything? IDK, moving everything self-hosted to the cloud was supposed to make operations a thing of the past, but in a way it just made ops an even bigger industry than it was.
Let's not forget there has been times when if-else statements were considered AI. NLP used to be AI too.
Artificial Intelligence is a whole subfield of Computer Science.
Code built of nothing but if/else statements controlling the behavior of game NPCs is AI.
A* search is AI.
NLP is AI.
ML is AI.
Computer vision models are AI.
LLMs are AI.
None of these are AGI, which is what does not yet exist.
One of the big problems underlying the current hype cycle is the overloading of this term, and the hype-men's refusal to clarify that what we have now is not the same type of thing as what Neo fights in the Matrix. (In some cases, because they have genuinely bought into the idea that it is the same thing, and in all cases because they believe they will benefit from other people believing it.)
It turns out we didn't need a specialist technique for each domain, there was a reliable method to architect a model that can learn itself, and we could already use the datasets we had, they didn't need to be generated in surveys or experiments. This might seem like magic to an AI researcher working in the 1990's.
It doesn't think, it doesn't reason, and it doesn't listen to instructions, but it does generate pretty good text!
People constantly assert that LLMs don't think in some magic way that humans do think, when we don't even have any idea how that works.
It doesn't matter anyway. The marquee sign reads "Artificial Intelligence" not "Artificial Human Being". As long as AI displays intelligent behavior, it's "intelligent" in the relevant context. There's no basis for demanding that the mechanism be the same as what humans do.
And of course it should go without saying that Artificial Intelligence exists on a continuum (just like human intelligence as far as that goes) and that we're not "there yet" as far as reaching the extreme high end of the continuum.
Aircraft and submarines belong to a different category and of the same category, than AI.
Humans are not all that original, we take what exists in nature and mangle it in some way to produce a thing.
The same thing will eventually happen with AI - not in our lifetime though.
That doesn't mean much.
No it doesn't, this is an overgeneralization.
- We have a sense of time (ie, ask an LLM to follow up in 2 minutes)
- We can follow negative instructions ("don't hallucinate, if you don't know the answer, say so")
The general notion of passage of time (i.e. time arrow) is the only thing that appears to be intrinsic, but it is also intrinsic for LLMs in a sense that there are "earlier" and "later" tokens in its input.
Until we see an LLM that is capable of this, then they aren't capable of it, period
That's the difference. AI cannot be held responsible for hallucinations that cause harm, therefore it cannot be incentivized to avoid that behavior, therefore it cannot be trusted
Simple as that
The proof burden is on AI proponents.
There's this very cliched comment to any AI HN headline which is this:
"LLM's don't REALLY have <vague human behavior we don't really understand>. I know this for sure because I know both how humans work and how gigabytes of LLM weights work."
or its cousin:
"LLMs CAN'T possibly do <vague human behavior we don't really understand> BECAUSE they generate text one character at a time UNLIKE humans who generate text one character a time by typing with their fleshy fingers"
Intelligent living beings have natural, evolutionary inputs as motivation underlying every rational thought. A biological reward system in the brain, a desire to avoid pain, hunger, boredom and sadness, seek to satisfy physiological needs, socialize, self-actualize, etc. These are the fundamental forces that drive us, even if the rational processes are capable of suppressing or delaying them to some degree.
In contrast, machine learning models have a loss function or reward system purely constructed by humans to achieve a specific goal. They have no intrinsic motivations, feelings or goals. They are statistical models that approximate some mathematical function provided by humans.
Even if it can extrapolate to some degree (altough that's where "hallucinations" tend to become obvious), it could never, for example, invent a game like chess or a social construct like a legal system. Those require motivations like "boredom", "being social", having a "need for safety".
> it could never, for example, invent a game like chess or a social construct like a legal system. Those require motivations like "boredom", "being social", having a "need for safety".
That's creativity which is a different question from thinking.
Humans invent new data, humans observe things and create new data. That's where all the stuff the LLMs are trained on came from.
> That's creativity which is a different question from thinking
It's not really though. The process is the same or similar enough don't you think?
What LLMs do is using what they have _seen_ to come to a _statistical_ conclusion. Just like a complex statistical weather forecasting model. I have never heard anyone argue that such models would "know" about weather phenomena and reason about the implications to come to a "logical" conclusion.
In the same way a human might produce a range of answers to the same question, so humans are also drawing from a theoretical statistical distribution when you talk to them.
It's just a mathematical way to describe an agent, whether it's an LLM or human.
LLMs aren't good at either, imo. They are rote regurgitation machines, or at best they mildly remix the data they have in a way that might be useful
They don't actually have any intelligence or skills to be creative or logical though
Yes, humans are also capable of learning in a similar fashion and imitating, even extrapolating from a learned function. But I wouldn't call that intelligent, thinking behavior, even if performed by a human.
But no human would ever perform like that, without trying to intuitively understand the motivations of the humans they learned from, and naturally intermingling the performance with their own motivations.
We don't just study it in humans. We look at it in trees [0], for example. And whilst trees have distributed systems that ingest data from their surroundings, and use that to make choices, it isn't usually considered to be intelligence.
Organizational complexity is one of the requirements for intelligence, and an LLM does not reach that threshold. They have vast amounts of data, but organizationally, they are still simple - thus "ai slop".
[0] https://www.cell.com/trends/plant-science/abstract/S1360-138...
In my opinion AI slop is slop not because AIs are basic but because the prompt is minimal. A human went and put minimal effort into making something with an AI and put it online, producing slop, because the actual informational content is very low.
And you'd be disagreeing with the vast amount of research into AI. [0]
> Moreover, they exhibit a counter-intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget.
[0] https://machinelearning.apple.com/research/illusion-of-think...
It does say that there is a maximal complexity that LLMs can have - which leads us back to... Intelligence requires organizational complexity that LLMs are not capable of.
That doesn't mean such claims don't need to made as specific as possible. Just saying something like "humans love but machines don't" isn't terribly compelling. I think mathematics is an area where it seems possible to draw a reasonably intuitively clear line. Personally, I've always considered the ability to independently contribute genuinely novel pure mathematical ideas (i.e. to perform significant independent research in pure maths) to be a likely hallmark of true human-like thinking. This is a high bar and one AI has not yet reached, despite the recent successes on the International Mathematical Olympiad [3] and various other recent claims. It isn't a moved goalpost, either - I've been saying the same thing for more than 20 years. I don't have to, and can't, define what "genuinely novel pure mathematical ideas" means, but we have a human system that recognises, verifies and rewards them so I expect us to know them when they are produced.
By the way, your use of "magical" in your earlier comment, is typical of the way that argument is often presented, and I think it's telling. It's very easy to fall into the fallacy of deducing things from one's own lack of imagination. I've certainly fallen into that trap many times before. It's worth honestly considering whether your reasoning is of the form "I can't imagine there being something other than X, therefore there is nothing other than X".
Personally, I think it's likely that to truly "do maths" requires something qualitatively different to a computer. Those who struggle to imagine anything other than a computer being possible often claim that that view is self-evidently wrong and mock such an imagined device as "magical", but that is not a convincing line of argument. The truth is that the physical Church-Turing thesis is a thesis, not a theorem, and a much shakier one than the original Church-Turing thesis. We have no particularly convincing reason to think such a device is impossible, and certainly no hard proof of it.
[1] Individual behaviours of LLMs are "not understood" in the sense that there is typically not some neat story we can tell about how a particular behaviour arises that contains only the truly relevant information. However, on a more fundamental level LLMs are completely understood and always have been, as they are human inventions that we are able to build from scratch.
[2] Anybody who thinks we understand how brains work isn't worth having this debate with until they read a bit about neuroscience and correct their misunderstanding.
[3] The IMO involves problems in extremely well-trodden areas of mathematics. While the problems are carefully chosen to be novel they are problems to be solved in exam conditions, not mathematical research programs. The performance of the Google and OpenAI models on them, while impressive, is not evidence that they are capable of genuinely novel mathematical thought. What I'm looking for is the crank-the-handle-and-important-new-theorems-come-out machine that people have been trying to build since computers were invented. That isn't here yet, and if and when it arrives it really will turn maths on its head.
And here's some more goalpost-shifting. Most humans aren't capable of novel mathematical thought either, but that doesn't mean they can't think.
As for most humans not being mathematicians, it's entirely irrelevant. I gave an example of something that so far LLMs have not shown an ability to do. It's chosen to be something that can be clearly pointed to and for which any change in the status quo should be obvious if/when it happens. Naturally I think that the mechanism humans use to do this is fundamental to other aspects of their behaviour. The fact that only a tiny subset of humans are able to apply it in this particular specialised way changes nothing. I have no idea what you mean by "goalpost-shifting" in this context.
If we knew that, we wouldn't need LLMs; we could just hardcode the same logic that is encoded in those neural nets directly and far more efficiently.
But we don't actually know what the weights do beyond very broad strokes.
we understand on this low level, but LLMs through the training converge to something larger than weights, there is a structure of these weights which emerged and allow to perform functions, and this part we do not understand, we just observe it as a black box, and experimenting on the level: we put this kind of input to black box and receive this kind of output.
Why? Team "Stochastic Parrot" will just move the goalposts again, as they've done many times before.
Imagine a process called A, and, as you say, we've no idea how it works.
Imagine, then, a new process, B, comes along. Some people know a lot about how B works, most people don't. But the people selling B, they continuously tell me it works like process A, and even resort to using various cutesy linguistic tricks to make that feel like it's the case.
The people selling B even go so far as to suggest that if we don't accept a future where B takes over, we won't have a job, no matter what our poor A does.
What's the rational thing to do, for a sceptical, scientific mind? Agree with the company, that process B is of course like process A, when we - as you say yourself - don't understand process A in any comprehensive way at all? Or would that be utterly nonsensical?
It's like we're pretending cognition is a solved problem so we can make grand claims about what LLM's aren't really doing.
When you type the next word you also put a word that fits some requirement. That doesn't mean you're not thinking.
A lot of this is marketing bullshit. AFAIK, even "machine learning" was a term made up by AI researchers when the AI winter hit who wanted to keep getting a piece of that sweet grant money.
And "neural network" is just a straight up rubbish name. All it does is obscure what's actually happening and leads the proles to think it has something to do with neurons.
Among most people, you're thinking of things that were debatably AI, today we have things that are AI (again, not due to any concrete definition, simply due to accepted usage of the term.)
They still are, as far as the marketing department is concerned.
LLMs are one of the first technologies that makes me think the term "AI effect" needs to be updated to "AGI effect". The effect is still there, but it's undeniable that LLMs are capable of things that seem impossible with classical CS methods, so they get to retain the designation of AI.
Some can remember the difference between iPhone 1 and 4 and where it took off with the latter.
Diffusion of innovations: https://en.wikipedia.org/wiki/Diffusion_of_innovations :
> The diffusion of an innovation typically follows an S-shaped curve which often resembles a logistic function.
From https://news.ycombinator.com/item?id=42658336 :
> [ "From Comfort Zone to Performance Management" (2009) ] also suggests management styles for each stage (Commanding, Cooperative, Motivational, Directive, Collaborative); and suggests that team performance is described by chained power curves of re-progression through these stages
Transforming, Performing, Reforming, [Adjourning]
Carnal Coping Cycle: Denial, Defense, Discarding, Adaptation, and Internalization
Ai may be more like electricity than just electric motors. It gave us Hollywood and air travel. (Before electricity, aluminum as as expensive as platinum.)
As economists they are wedded to the idea that human wants are infinite, so as they things we do now are taken over, we will find other things to do: maybe wardrobe consultant, or interior designer, or lifestyle coach - things which only th rich can afford now, and which require a human touch. Maybe.