Posted by benbreen 4/2/2025
Regarding the content of this interview:
>If you compiled an enormous dataset of everything Borges read, and combined it with an exquisitely sensitive record of every sensory experience he ever had, could you create a Borges LLM?
This is my Kantian way of thinking about epistemology, but I don't think that LLMs can create synthetic a priori knowledge. Such knowledge would be necessary to create Borges out of a world without Borges.
In this interview, Simon's view feels much more like the way Hume viewed people as mechanical "bundles of sensations" rather than possessing a transcendent "self". This led to his philosophical skepticism, which was (and still is I guess) a philosophical dead end for a lot of people. I think such epistemological skepticism is accurate when applied to machines, at least until some way of creating synthetic a priori knowledge is established (Kant did so with categories for humans, what would the LLM version of this be?)
Yes, his writings are short, but man they are dense!
To anyone who cares, do this exercise: read short story by Borges, probably the shorter the better. Then go ahead say, next day, and try to write it down again in your own words. I tried a couple of times, and I ended with at least twice the number of pages. Amazing.
Do you think that a LLM has the ability to identify a new a priori knowledge?
It seems like it would be a lower threshold to meet but If you combine that with a stochastic process then it seems inevitable that it would be able to ruminate until it came up with new a priori knowledge.
I think the only criteria of no math that would satisfy some people would by definition fail because the model would have no concept of points or distance because some would certainly count those as mathematical content.
Sadly, almost no one talks about it. Ditch the form and embrace the substance. ← It also nods to the mystery behind The Navidson Record.
I wish I had known this when I first read it.
The Navidson Record is on youtube if you dare =P
Thanks!
Hmm, what if you could recreate, word-for-word, the great works of an author like Borges (or, say, Cervantes) by so thoroughly understanding their life that the words themselves came out of you, not memorized and recapitulated, but naturally and unbidden? What an interesting idea for a story, maybe an LLM will be able to write that one day.
With AI tools, though, I can "read" Borges in his native language: with my phone + OCR + translate I have an English language companion. Or, using the voice interface I can try narrating the Spanish text and ask clarifying questions whenever I'm confused.
An author like Borges makes it well worth the extra effort. And, his puzzles often involve language, so the extra layer of mental translation can mirror the work itself, e.g. in his poem La luna [1]. (though, I envy your native Spanish)
On these conjugations, compound verbs are not hard at all.
(I agree!)
An LLM trained on Sartre would be amazing because the logical extensions of many of his positions and postulations would be uncomfortable in polite society. Even as a human being he quite frequently espoused concepts counter the grain of civility or notions of what ethics are or should be. An unrestrained, uncensored LLM in this vein could be scary and gut wrenching and yet a good reminder of our less-than-ideal state of refinement of thought and behavior as a species.
But mostly I like it for it's obsession over the deluge of information that we sift through on a daily basis. Borges perfectly described this issue a decade before Shannon would invent the concept of the bit and provide the foundation for the cognitive revolution
I wrote about the connection between Borges, AI, Wikipedia, Kafka (the messaging system, not the author), GPUs, and cryptography in the small print on page 7 of this: