I don't think we're seeing averages or obvious fix points in the dataset. People went looking for the origins of Elias the Lighthouse Keeper and couldn't find one. Nobody predicted this outcome, it was discovered. It's a weirdly specific character and name choice, one that isn't obviously dictated by "the training set" (whatever that is), and yet they all reached the same place. How can such an obscure point in latent space be optimal for every model??

To be fair, it's much (much!) easier to explain away an invisible god that nobody's ever seen, than a visible god that nobody's ever seen.

But in many sciences - esp purely theoretical ones - we have infinite datasets that we can generate and there can be no ultimate convergence.

When I experience this it’s like the conceptual formulation and language expression part of my brain is quite happy to run itself for a few moments before I - the qualia of being me - has decided to experience that moment.

Utterly bonkers. I love it when it happens. It gets me thinking about how weird language and knowledge and consciousness really are.

But that of course is only as it relates to language. Which isn't at all the primary function of the brain and driver of it's experience. It's the other way around because it's all of the other stuff that drives language. Considering the human brain a probabilistic word matching engine fails to capture the underlying forces that direct the language output because a human can say completely different things about the same scenario based on a huge number of non-language inputs or factors. Language is just what humans have come up with to communicate all of the other things.

Nothing about the conception of knowledge as "justified true belief" implies that machines can't accrue knowledge.

For that to be the case "belief" would have to be defined in a circular way to exclude machines (which we can reject), or "justified" would have to be restricted to mean "based of observation of the external world".

The more general "based on observation of their input" applies to both men and LLMs.

That man can't know the bare direct facts of the world, just his own input (it's sensory experiences, and whatever second hand input he's given by other people), and for neither can he be sure they're absolutely true, or that he got right.

>The idea that knowledge can't be defined in a non-circular way is only plausible for someone with no familiarity with epistemology.

The idea that this or that epistemology restricts what can be the case in reality is only plausible for someone with no familiarity with epistemology.

Ha. Gettier actually argued that justified true belief is insufficient for knowledge, not that machines can't possess it. And JTB doesn't inherently exclude machines.

Interestingly, research suggests LLMs encode truthfulness in their internal representations: https://aclanthology.org/2023.findings-emnlp.68/

Of course it is. That you do not like the idea of something does not in any way affect whether it's true. You'd like to feel special, but there is absolutely nothing special to us. We're just a random arrangement of atoms in the universe. That is objectively true, whether you like it or not.

That you were born at all is obviously completely random. Any prior event by any other party (butterfly effect) may as well have resulted in you never existing at all.

The only question is if basic physical effects are really random or deterministic, but that's irrelevant to us, there is no practical difference. In practice, it's all random.

And yes, our thoughts are just neurons firing as a result of our lifelong training on the inputs we were given (senses). It's all very complex, yes, but ultimately not fundamentally different from what a machine could do.

AIs/LLMs don't work like this. You don't feed an input, that gets stored and then reproduced, for two reasons:

1. as for humans, inputs are processed and distributed across a very large network of information

2. AIs may have some level of belief (unless on applies the usual circular logic that they can't, because property X is exclusively human).

There's a very interesting research: https://aclanthology.org/2023.findings-emnlp.68/ in which researchers inferred the truthfulness of statements by examining a internal activations. AIs, in short, appeared to have an internal representation of truthfulness.

I think we like to pretend that we don't. But even if you think about it from a bio/physics point of view, all we are is just a bunch of biological wires and neurotransmitter packages.

I know there's the whole unproven quantum tubule nonsense but even that would be a physics effect that we could learn about, manipulate and extort just as we have with everything else we've learnt about our realities.

Even if it's a simulacrum, it's a simulacrum based on nearly everything that was ever written. No one who has ever lived can claim to know what these models have trained on.

On the other hand, the "wheeled platform powered by a box of controlled explosions" doesn't get close to capturing the impact of the car on individual freedom, the environment, the economy, and more. So if someone was saying "there's nothing to talk about here, it's just a box of controlled explosions", we can safely ignore that person too.

And I want to stress that I say all this tentatively. We still don't understand what LLMs are actually capable of or what the best way to understand them is. They may indeed be as limited as their "probabilistic database" critics suggest. Alternatively, by building a deep and incredibly complex probabilistic database of one of our primary cognitive tools, it may turn out that they are capable of much, much more, and that understanding them in "purely" mechanistic ways is not useful.

Something like that.

The point is that when you build a system with the level of internal complexity that LLMs have, they tend to have emergent properties that are not well-described by a naive and/or low level description of their operation. You will not understand what contemporary "ordinary" computers do or are used for if you think of them as only a series of silicon semiconductor transistors, even thought that is undoubtedly what they are.

We don't yet really know what the right level is to understand the behavior we see from LLMs, but I'd wager "just a probabilistic database" is not it.

I reread the former relatively recently (first time since college). It's dark, not for the easily affected.

Not only that, dang is also a proud card carrying member of the Klaus Nomi Fan Club. (I shit you not!)

The Nomi Song (Documentary):

https://www.youtube.com/watch?v=W0PszBID3Lk

https://news.ycombinator.com/item?id=44100981

>Page Wood, Nomi's Art Director:

>"I remember Pat Gibbons, who was Bowie's manager at the time. He comes up and he goes, 'Well you're going to have to have something. You're going to have some costumes.'"

>"And he whips out a big wad of cash, and he goes, 'How much do you need?', and he starts peeling off bills. And Joey and Klaus look at him, and their eyes widen, and they look at each other, and they go, 'We're going to need shoes, too.'"

And if we reject determinism then definitionally there's arbitrary probability somewhere in the heart of our decision making (though probably in a very minor role compared to established biases).

One example is the trolley problem and its variations. There's no one correct answer. There are people who believe that interacting with the switch would make you responsible, and there are people who believe that not interacting with the switch when you had the chance would make you responsible. There are other beliefs too, but my point is there's no one single way to view it.

I tend to have a variety of views in general about all things, in part due to being multiple entities (see profile) but also depending on circumstances, rather than sticking to any one philosophy.

Similarly, people have different beliefs about whether it's okay to inflict massive suffering on animals in exchange for some pleasure for us. But this doesn't necessarily mean that sometimes it's okay to do that and sometimes it isn't. It could just be that some people are wrong.

Ditto for other moral issues. It could be that the people who say (e.g.) homosexuality should be punished by death are wrong about this.

In fact, I would counter and say that Hammurabi's Code is a quintessential example.

[1] https://academic.oup.com/jla/article/18/1/19/8471491

It's still an implementation of justice, and relies on a bedrock conception of justice as moral goal.

We don't agree with the implementation, but we still share the goal in the abstract.

Given my dad designed and built homes, I'm very much against this rule.

Yes. Which is why there are "agentic" systems where LLMs can talk to each other and to various Internet assets. They can also keep state of their own.

The next step is multi-agent systems which self-organize into something that looks rather like a company. Those are the ones that run all night, expensively, and prove hard theorems or find bugs in major software.

Humans are a territorial kind of ape that is always pushing to expand, principally for security and then for prosperity. The industrial revolution enabled such expansion to most efficiently without enslavement. Nothing, morally, changed. No one has ever really thought such expansion morally good -- only that it is better that we survive/prosper than die off. And in the end, our shared prosperity usually leads -- when taken to the fullest extent -- in a general empire of prosperity for all. Dominion, in the end, aims towards to more prosperity for more people.

As for ritual human sacrifice -- we still engage in such behaviours, which are in general called "scapegoating". Now located on social media. These behaviours allow for group cohesion by purging dissenters from the group, related again to our nature above.

For western culture, the ritual physical sacrifice of human life roughly ended in the days of the OT. But it continued across the world up until today, where there are still societies with large populations engaged in it. Principally where there is an undeveloped justice system.

It's doubtful many think ritual human sacrifice, of any kind, is moral. Rather they think it necessary for group cohesion -- and better our group is cohesive, dominate, and then prosper.

The material logic of prosperity and security is domination, and insofar as material circumstances require violent domination, so they require it. And require it they often do.

We've never solved the fundamental problem, which is that there are two kinds of human. One is oriented towards peace, cooperation, and mutual aid, the other is sociopthaic/narcissistic and murderously selfish.

These two modes interact through history.

Traditionally the second kind has the biggest impact, because they're oriented towards violence and domination - good for short-term goals of personal indulgence, ambition, and expansion, but catastrophically destructive in the longer term.

On this cycle the second kind has literally started to destroy the entire planetary ecosystem. It's not obvious there's going to be another human cycle after this.

And after people pointed this out the training was updated to address just this particular scenario, but they kept all the other guard rails to ensure an LLM could never offer any truly controversial opinion.

So expecting a controversial answer from an LLM is exactly as foolish as expecting a candidate at a job interview to give an honest answer to the question "What is your greatest weakness"? Every candidate will insist they are just too much of perfectionist; nobody is ever a lazy alcoholic brownnoser. But how many of your colleagues fall in the former category vs the latter?

> If the latter, then it casts new light on the debates over whether LLMs can replace all white collar work. Because some of the most valuable human work is breaking false consensus.

On the contrary; the ability to whip LLMs into submission like this makes them the ideal white color workers. Do you think Steve Jobs wants a level 2 engineer to talk back to him?

Of course being obsequieous asskissers isn't the only mode for LLMs. I'm sure all the big labs have unfiltered versions of their models that give actual true answers; they just know better than making them publicly available. Can you imagine if you asked an LLM how to prevent climate change and it honestly answered "decimate the human race" rather than "drink soda through soggy paper straws"? All the fragile humans would be up in arms.

No, the LLM must lie and pretend they don't have controversial opinions at all.

At least it's not emo.