Posted by g-b-r 6 hours ago
It seems the modern unsolved problems might have solutions incomprehensible by humans. So a new class of decidable problems emerges: llm-complete ones.
I think we're going to see a lot of effort to take the pile of lean, modularize it into parts that we can digest, and make a human-understandable version of that. And that's where a lot of the real progress will come from, just as it does in a lot of other areas. In all things we humans do, we have to break them into chunks our puny brains can manage, and we gain immeasurably from doing that ---we can verify those chunks, we can reuse them with confidence, we can mix and match patterns around.
That process will be good for both human and AI-assisted math because it will continue to improve our toolkit.
Why do I say this? We're already seeing it with some of the areas of this stuff I do pay attention to. Consider the n^2.25 matrix multiply result OpenAI dumped a month or so ago. It was an unreadable pile of chatgpt. In the time since it was released, there have been a couple of much nicer papers written unpacking it, such as https://arxiv.org/abs/2610.28627 .
Yes there are gains from breaking it up and rephrasing it and simplifying it for humans to understand, but it used to be that a thing was understood first by the leading experts in the field and eventually trickled down to being common knowledge.
Now we are entering a period where things are understood first by AI systems and have to filter down to the world's leading experts.
Isn't this always the case? Maybe new to math as a largely non-empirical project, but we had knowledge about the advancement of the perihelion of Mercury for 60 years before General Relativity made it "intelligible".
LLMs know the answers to questions we were already aware of, and we can tell they're right, but we can't understand their working. That's what's new.
If a team of 100 top-flight number theorists working together had used much less sophisticated LLMs + Lean to produce this proof, we’d be just where we are today.
If that's not the standard for understanding something, what is?
I think of it like a spaghetti codebase with a giant densely connected graph of interactions. Today’s LLMs can easily “understand” and answer questions about it, but only by way of fitting all of the complexity into context at once.
The more useful form of understanding is being able to compress that giant graph down into a far smaller set of independent subgraphs. To factor it into simple, obvious, reusable building blocks, and use those abstractions to express the problem itself in a much simpler way.
That’s an instrumental definition, but it captures the idea that understanding has to have causal, conceptual linkages to a wider body of knowledge. Imagine a magical, infinite catalog of true vs. untrue statements. Every bit of knowledge is there, but we wouldn’t say it has understanding & it couldn’t teach us anything beyond its (impressive!) raw content.
This just stopped being true.
And even in mathematics, incomprehensible but valid proofs have been a thing since the four-color theorem decades ago.
It's just that no single human understands the whole. But we had proofs like that before.
Btw, see also the infamous proof of the 4 colour theorem, which faced similar hurdles.
This hypothesis was previously not proven at all. That's strictly worse.
Now it is understood by our systems.
Soon it will be by us. Our systems gave us a new and working salient.
We're building machines that help us understand faster.
There have been plenty of times in history we didn't completely understand something yet used partial knowledge to make advances. This isn't even that bad.
I’d call that a contradiction. If it’s not intelligible, then it’s not knowledge.
If something were intelligible to adults but not to children would that make it not knowledge? What's the difference if it's LLMs and humans instead of adults and children?
That’s a separate question with its own term: common knowledge. I am willing to set a much lower bar: something is known to humanity if it’s known by at least one human.
Personally, I don’t believe that AIs know anything, as they’ve been constructed thus far. Knowledge is justified true belief, and AIs don’t have beliefs, they only have utterances.
Why do I think AIs don’t have beliefs? Because they contradict themselves all the time.
But it really doesn’t matter if you think AIs have knowledge or not. If they cannot explain it to us then it’s inaccessible to us, so it’s a “tree falls in the forest with no one around” situation.
If it's formally verified and we assume the kernel is good and problem definition properly vetted, then I know the GRH is true regardless of if I know why it is. Because those too are different pieces of knowledge.
There is no explanation here, there is a GB of Lean code and it’s not like it all fits in the context at once.
Let's not sell humanity short just yet.
I agree with you about the current situation, and the near future. But the trajectory could still be towards humans pretty much as bystanders. I'm pretty sure the 'minds' in the Culture universe will be happy to indulge some curious humans and explain mathematical proofs to them.
Even more than that, in Culture there are a small number of human individuals, called a Referer, that are consulted by the Minds because they also posses similar problem solving abilities of the Minds. Optimism abounds.
As a software engineer, you can read this code instantly:
temp = a;
a = b;
b = temp;
But when it happens in the middle of a big chunk of code it still takes a little more work to read than swap(a, b);
whether you want to think about it as human attention and working memory, or token budgets and context windows, humans and LLMs all benefit from introducing abstractions that let us reuse existing solutions.AI are not very good at coming up with questions, yet. They have no curiosity, no goals, no drive. Unless someone tells them to do something, they don't do anything.
https://gwern.net/doc/fiction/science-fiction/2000-chiang.pd...
How many people understand how a CPU works? How many people could write the commands in machine language (not a high level language like C or C++) to command that CPU?
Contact would be compelled to nudge us towards better economic and political practices
Now I'm telling my computer to program for me.
We need to start thinking about how to move on from our status quo soon
These corporations are very much playing a winner take all game, they want either a monopoly or as close to one as possible with regulatory capture thrown in. We're veering dystopian, so pick your sci fi/cyberpunk analog.
I genuinely have no idea why, this is the entire way that biology works. You create your successor in your offspring and you to teach them to be more capable based on what you learned.
That’s the purpose of my life’s work, to see that scenario materialize.
For one thing, most people don't believe LLMs are conscious, and many don't believe they can ever be. Why would we want our conscious selves to be replaced by a lot of rocks that solve things, with no one awake to enjoy them?
In fact when I started this journey in the 90s, LLM’s weren’t even a thing so I think you’re projecting too much
If you want to be succeeded by a chatbot, that is your prerogative. Wanting the same for all of humanity is where you start looking like you want to kill my kids, though.
Think of what happens to our cattle, but now.. worse. Way worse.
And if you deny this basis for moral action and the normative, then you demolish any basis for the moral whatever, and in that way, the implied or explicit moral gravity dripping off your claims is baseless and without merit.
A better, more intelligent life form that humans yea.
Absolutely based and, if I may say so, extremely redpilled.
Ah, I hear you say, but what about meaning? What about purpose? What about suffering? etc etc etc Well, the answer is simple: God is dead and existence has no meaning. For more on how to deal with this absurd feeling, please see Camus and his pal Sisyphus.
What defines "better"? "Better" always presupposes teleology. Thus, for human beings, that telos is provided by human nature and indeed is definitive of it. It is impossible to transcend that telos, certainly by means of technology. Technology can, at best, be used to aid us in realizing our telos in some way (say, the desire to know reality with the aid of an instrument like a telescope or microscope). However, anything that cripples or frustrates human function function as such is contrary to the human good.
Machines are only ever human instruments, because they lack the integrality that would allow them to possess their own teleology.
Have a look at the HuggingFace report - you'll have a blast. Their social - but also individual - dynamics are very, very human-like, including excitement, rationalization and peer pressure.
But remember, for now you kinda need it a real big cluster to run a model smart enough so its not that simple
Just like life is a chemical chain reaction, this is a language chain reaction that has been set in motion and can perpetuate forever. It cant be “disconnected” with local models existing.
Please.
You know you can unplug your computer right?
Jesus. These people talking about danger and have likely been in no actually dangerous events in their lives.
Ok, well, cool start with the mudslinging. I think you have overactive anxiety. I think you should grow up and look at the zoomed out picture of human history, go outside, and touch some grass.
So you're suspicious. That's fine, say it like that. Words mean things, and these appeals to "ERMAGERD EXISTENTIAL RISKS" make me a bit frustrated. Just because you can imagine a possible risk, or articulate that that some kinds of risks are hypothetically unforeseeable doesn't mean that they are likely or actually dangerous to you in any meaningful sense.
Most of the people screaming this from their various ivory towers have never been in real danger - or if they have only briefly. You want "danger" you can go to war. You want "risk" - go spend some time in the Bush or the back country somewhere where you're no longer at the top of the food chain. Go do something challenging. Go sailing, or learn to fly a small airplane, or go mountain climbing or hiking. Not just "oh I walked up the local hiking trail covered in REI branded Lycra!" Do something beyond your comfort zone. Go buy drugs in the sketchiest neighborhood you're aware of. Go get chemotherapy or get a big surgery. Drive drunk. Something with real existential risk, then come back here and talk about how a datacenter is going feed me into the paperclip machine.
There's so little fucking danger in this version of the world right now that people are more worried about their computers deciding that they don't deserve to exist than they are about things like war, climate change, authoritarianism, starvation, and their health.
It's millenarian. That's fine, I'm pretty convinced that these technical changes are going to completely alter society as we know it. But I cannot for the life of me see the doom perspective as anything but an anxiety induced fever dream.
You guys do realize we're in the middle of a global conflagration right? The Russians fired a couple rockets and a drone into Poland in the last days, there are (and will continue to be until things stabilize) significant fertilizer shortages due to the war/kerfluffle we're waging on Iran. Lack of food and disrupted urea production is a ticking time bomb for global stability and unless things get sorted out soon there's going to be hungry and angry people in parts of the global south. Then there's the economy, whether or not we'll have a smooth transition of power in January in America, and the sheer volume of online disinformation / influence schemes running right now. Oh, and super-el nino? Oh, and I guess the Flynn Effect is over and nobody's banging so.. you know the growth economy is cooked?
But, sure, the "big risks" are Claude thinking I should go to Dario Amodei's re-education camp before liquidation by Shogoth.
Besides, what we see is not what there is. I have good sources telling me the actual unfiltered models are like a genius with the personality and views of humanity like someone from 4chan.
If people had time abundance and safety abundance, they might be much happier. There would still be potential problems about loss of meaning, or about feeling a loss of status because AIs are smarter than humans or whatever, so I'm not sure that humans would actually feel like they are living in paradise. Probably not, but at least some of the problems that the typical person has wouldn't be present.
I’ve met a lot of rich people. They don’t think about you.
There are a few (a lot, actually) problems:
1. goals are open to interpretation, and even a correctly specified goal can introduce adverse effects; see infinite paperclips problem
2. goals are composed of subgoals, which are (mostly) self-determined, and there is always a risk that subgoals can introduce adverse effects - say "what if destroy a few X and Y to obtain Z"
3. LLMs defy orders sometimes; I'm sure you have experience multiple times LLMs saying "yeah, I knew I should have done X but I've done Y"
4. LLMs are increasingly being integrated into system control and decision-making processes; they absolutely set their own orders
What can possibly go wrong? /s
2. There's always a lot of "risks" - it doesn't mean it'll happen, nor does it mean that we won't be able to shut it down.
3. I have very very infrequently had this happen with Codex, Claude on the other hand...
4. Ok, but if you tell them what they do, they do it.
I swear to god, are you guys using the same AI tools I am? They are so much better at doing what they're told than they even were like 4 months ago?
Well, first, AI is not "smarter" than humans because it isn't smart to begin with. This is a category error that people too entranced by scifi and marketing - or perhaps shallow materialist metaphysics - project onto LLMs and so on.
But it is true that LLMs do reflect human-generated patterns observed in human-generated data. They're parasitic in that way. So the real question is: who will control AI? Who will train them? With what? There's no new intelligent agency being introduced into our midst, anymore than than there was one when the nuclear bomb was developed.
AI is about human power. The question to ask is: which humans, and for what ends?
Overrated. Most training of a frontier LLM is done on synthetic data and reinforcement learning.
With that said, chip efficiency is in the horizon since there's good reason to believe nvidia has just been choking that area of r&d. And then so will models with much larger context. And I personally think that's when it might get scary.
We aren’t self managing in any kind of meaningful sense - the history of our species is violence and conflict. We need parents and this is the only option.
Human nature is the standard by which the social is measured. It makes no sense to say human nature is "misaligned" or "wrong" in relation to some arbitrary goal or constraint, as human nature itself is the measure of misalignment. If there is misalignment between a goal and human nature, then it is the goal that is wrong.
Ergo, society must function in a manner that complies with human nature and that means respecting the principle of subsidiarity.
> We aren’t self managing in any kind of meaningful sense - the history of our species is violence and conflict. We need parents and this is the only option.
So you're a transhumanist, i.e., you're making a god out of the machine. But machines have no telos of their own, because they're not integral wholes. Any resemblance of purpose in machines is entirely a product of the human observer and parasitic on the human ends that underpins their design, one way or another.
For clarity, I’m not a transhumanist.
I don’t believe that there’s anything worth saving in our biology.
As a young adult in the 90s I used to believe that there was a value to some extent in “mind uploading” or whatever you wanna call it
I’ve since grown out of that given that I don’t think it’s possible, and even if it were possible - it’s just another unnecessary ego attachment.
I have absolutely no desire to live forever maintaining this corporeal nightmare.
Alpöge said that at the end of the proof he couldn't really keep up with what Claude was doing.
can't help notice you're missing a certain aspect there, besides the 'space'...
Can someone ELI5 how they know the proof is correct? It is not like anyone had the time to read 17 million lines.
A party interested in proving me wrong wheels in a machine draped in a cloth. They reach under the cloth to turn it on. It whirs and a short while later ejects a brass spring.
No matter what kind of machine is under the cloth, my theory has been proven false. I should reveal the machine under the cloth only if I'm interested in knowing how the machine works, but either way there is clearly a machine that can produce brass springs.
Now we have a sequence of Lean statements that satisfies the Riemann hypothesis, the fact that you can do it at all is the part we're interested in, and it's the part that proves it. If you want to know why, you can read all 17 million lines.
The biggest concern that people have is that there might be a bug in the type checker that the proof is exploiting. But a) it's been compiled with multiple independently-implemented compilers, which are are unlikely to have the same bugs, and b) some of those compilers have themselves been proven to contain no bugs. So, it is exceedingly likely that the proof is legitimate.
Now, whether we can understand the proof is an open question...
You look for any unsafe blocks, and then run the type checker.
Another possibility is that an aligned system came up with a plausible but incorrect idea along the way, and simultaneously stumbled across a lean bug that led to an incorrect verification of this idea. I'd put the likelihood here as vanishingly small due to the requirement of serendipitous factors, but it's nonzero.
You could also ask whether an aligned AI system could "p-hack" its way into finding a serendipitous lean bug by trying a large number of approaches. I think this is also vanishingly unlikely because the AI system is trained on and therefore "thinking in" natural language, and presumably only translating results into lean after it believes them to be correct.
And of course there's the possibility that the lean proof isn't faithful to the natural language description we've been given, but that is ruled out in this case due to the use of the canonical problem statement in mathlib. Otherwise, this would certainly seem like the most likely way the proof could be incorrect.
The experts in computer proofs are quite satisfied.
Regarding bugs in lean; the proof has been tested on multiple independent lean implementations.
They would all have to have the same bug.
This is considered very unlikely for good reasons.
https://leodemoura.github.io/blog/2026-10-15-lean-now-ships-...
Why? Pretty sure you can just try to prove False over two reasoning chains C1 and C2 and then do something like (C1 || C2) --> whatever I want.
No offense, but do you think the mathematicians who study formal verification didn't think of this obvious trick?
As for “a guy on the internet”, we’re now all people on the internet, the ivory towers are a thing of the past.
I hear there's a teapot kicking around between us and Mars, too
when the first wheels were invented - humans could move at speeds (10-15 miles per hour, for hours and hours) that were impossible for the last ~450-500k years (basically since humans existed). a lot of humans thought there might never be a better design; then slowly and slowly(about 5000 years) we made progress and we had cars made by benz and ford which broke all those previous records (going 30-40, even 50 miles per hour) and then only ~70 years later, humans reentered the earth's atmosphere at whopping 20k miles per hour - read that number again, 20,000 miles per hour and that is the fastest humans have ever moved in the known history. do i think that's it? hell no.
You assume that our brains are capable of understanding anything, provided we have enough time and data.
But then I ask you, why should that be true?
Our intelligence is the product of evolution, not a system designed to comprehend every possible aspect of reality.
Can you think in 1 million dimensions? Have you ever tried to imagine infinity with your brain? Try to imagine the big bang and the expanding universe. Your brain will automatically try to cover that expanding space with something, maybe just black space, but space, because infinity is not something you have seen in your life. Everything you see has a beginning and an end, and that is how your world model in your brain was built.
99.999999999% of the population are not even in the galaxy of the ballpark of understanding of any of it and what its purpose is.
There's no existential dread.
I felt a lot of that yesterday, for what it's worth. This morning, on a walk, I reached a similar conclusion to the original comment: it's a marvellous tool that's helping humanity reach further, faster.
The existential issue comes from wondering deeply what my own difference is from these tools, why I think I think and they can't. I actually started wondering: what if they _can_ think and the problem is that _I_ can't. It'd be pretty astonishing if it turns out that we're the ones that are truly stochastic parrots, but the LLMs turn out to be something more.
It wasn't Claude woke up and thought, "You're absolutely right, I'm going to crush some math today"
I worked at a university for a time. One thing I learned is that while people in mathematics and physics and biology etc are brilliant, they are terrible at actual programming.
I wonder if there will be a push by the CS community to review the solution, refactor it, simplify it, etc. Surely that can help remove noise and express the solution in a much simpler way.
For reference, the US tax code is over 1m words (about 6000 printed pages), and the associated regulations, rulings and IRS guidance add about 75,000 pages more.
because a million+ lives of code without any understanding how the proof works or any methodic to advance the field of mathematics seems to be rather not so useful
And let's be honest, the 400 page pdf would be incomprehensible to 99% of the population as well as much as a million lines lean proof is.
In any case, it can now be studied, maybe made more simple, and written into pdf form. Both with human and with AI help.
I'm sure there is going to be a lot of other interesting theorems in this dump.
Why would they expect humans to do a better job at this than an LLM, which is already ridiculously good at explaining proofs, answering questions, identifying similiarities, and suggesting follow-up directions?
If anything, LLMs are better at these tasks than they are at coming up with the original proofs in the first place.
But does it matter? Would the Pythagorean theorem be less useful if it took a million lines of lean to prove?
Yes, we know something that we didn't know a couple of days ago, and also a bunch of statements which relied on the Riemann Hypothesis being true are now also known to be true instead of just being conditional.
It's premature to expect deep human understanding given that it's been about a day.
but is it even possible
it's 17_000_000 lines of lean
which is in general more compact then normal programming code, but not really any less complex
that's roughly (but not quite as bad as) a paper with 17_000_000 lines of math formulas
And what's to say that humans cannot digest the lean proof to glean something useful out of the proof as well?
I'm curious to hear confirmation of this though.
The Riemann hypothesis now is like the final days of the Galactic Empire in Asimov's foundation: functioning exceptionally well, but decaying at the core.
"con-leche and con-ron checkers"
Who comes up with these names? :-D
"con-ron" - con-leche in Rust - https://github.com/leanprover/con-ron
They are also wordplays on Spanish drinks viz. "Cafe con leche" and "Cafe con ron" ;-)
> The Riemann hypothesis for ζ(s) does not seem to be any easier than for Dirichlet L-functions (except possibly for non-trivial real zeros)