Top
Best
New

Posted by ilreb 3 hours ago

AlphaProof Code(deepmind.google)
346 points | 120 commentspage 3
throwaway198846 3 hours ago|
I hope it will be possible to verify CPUs soon as well. Of course that is much harder because they are not open source and physical and you need to simulate physics to verify them fully.
utopiah 3 hours ago|
some are https://www.crowdsupply.com/sutajio-kosagi/precursor caugh CPU-ish
hoppp 2 hours ago||
This is the way. I have been working on similar.

Since code is cheap there is no excuse to not have verification everywhere.

xrisk 3 hours ago||
wonder how many dollars were spent on this
cicko 3 hours ago|
it was one-shotted last night when the new Fable came out
thereader12 3 hours ago|||
It took 4 weeks according to the Article.
simon-ami 3 hours ago|||
so, millions? billions?
yread 3 hours ago||
They thank Sundar Pichai. I guess it makes sense but at the same time...
tucnak 3 hours ago|
This is got to be an inside joke
dinogo 2 hours ago||
I feel that paradigms like this, is going to be the standard going forward.
m3kw9 1 hour ago||
Just input the link to your llm and ask it how it applies to you
paganel 3 hours ago||
I'm not at all versed in this "formal proofing" thing, but without diving all that much into it how does this handle the halting problem (and the related problems that in the end all point to said halting problem)?
sebzim4500 3 hours ago||
Solving the halting problem means writing a program which can tell whether an arbitrary other program will halt or not.

Proving that a particular program will or won't halt can still be easy.

The halting problem does imply that there exist programs which are entirely correct that lean will never be able to prove the correctness of, but this is not really relevant in practice.

kuboble 3 hours ago|||
Halting problem only stops you from your prover from working on all programs.

On some programs like "stop" or "while(true)" the proof is easy.

With more sophisticated analysers it becomes possible on wider range of programs.

contravariant 3 hours ago|||
Dijkstra's approach would be to prove there is a strictly decreasing natural number associated to the loop.

This limits programs to the ones that can be formally shown to terminate. Which is good enough for most purposes.

My approach is to accept programs that run forever as long as I'm fairly sure each loop makes progress of some sort (instead of simply doing "while not solved: continue").

Edit: There's also a fun algorithm that, as far as I know, only provably terminates by being bounded by a sequence of strictly decreasing ordinals. Which breaks all kinds of models for proving it terminates.

bananaflag 3 hours ago|||
The same way Goedel's 0th incompleteness theorem doesn't make math proofs impossible (this is not sarcasm btw).
NooneAtAll5 3 hours ago|||
do you mean how is halting problem not a blocker or how halting problem itself is proven?

answer to the first is same as always - by simply not guaranteeing that everything is provable, especially in finite amount of time

bbor 3 hours ago||
Hmm. Which means that they just luckily don't have to prove anything about any software that might halt...? Or?

Like, they use some pretty stark terms here:

  Formal verification—mathematically proving code runs correctly without flaws—is the highest standard of security assurance; however, it is difficult and often labor-intensive.
Of course "proving" and "without flaws" are just words in a language game and might not mean the absolute things we expect them to as laypeople. Is that what's going on? Or can the "code" they're "proving... runs correctly" indeed be unrelated to such a fundamental finding of compsci?

I type this message to you all with sadness in my heart, cause I know I could get a better answer 100x faster by just asking an agent to research, which I could then have it dumb down to my level until I grasp it. I don't really know what the point of forums are, anymore. It's undeniably hard not to see ourselves as a planet of sad, fragile monkeys wasting away the hours between birth and death. That was already the case, perhaps, but at least learning and sharing knowledge about engineering in places like this felt like a somewhat meaningful way to positively impact the material conditions of my fellow humans, even in an indirect way. As of November 2026, I'm just the taste/framing component of artificial minds that seem overwhelmingly likely to develop their own within months or maybe years.

For a long time (since noticing the singularity) I've felt like someone doused in napalm and set alight while falling through the sky, frantically trying to put myself out while fixing my broken parachute. But post-Mythos I can't help but fear that there is no solid ground below. Just endless void. :(

jlouis 2 hours ago|||
Driving a lean proof manually is a lot of effort. It's often 50x or 100x the effort of writing the initial program in the first place. The grand idea here is that if you can get a system to have a good heuristic/intuition for figuring out those proofs, you can lower this effort amplification by a lot. This makes formal verification at this level feasible for mission critical software, and also a larger slew of software we rely on.

The difficulty lies in coming up with the right formulations, and also some times altering the program to make it amenable to proof. It's quite often the case that a proof won't go through unless the program itself has the right idea.

As for the halting problem: it is highly theoretical. Lots of programs which are useful can be proven to halt for instance. Or you can just prove that the program is correct up to the point in time where the sun expansion covers the earth. There are often practical ways around such limitations, including the alteration or reformulation of the program.

solyin 2 hours ago||||
Which definition of singularity are you using? I'm still doing my own laundry. It's quite poetic, what you wrote, if a bit sad. The hours between birth and death are waste, but are to be enjoyed. To discover the human condition. Overcome adversity, or die trying. Yes the machine gods will reign supreme but until they come to be, we must play and feel joy and sadness and love, hate... sonder. The planet doesn't have to be a bunch of sad monkeys. Maybe you can make a virus that will turn us into joyous lizards! There is so much left to discover and understand. How do you un-fry an egg?
NooneAtAll5 2 hours ago|||
> Which means that they just luckily don't have to prove anything about any software that might halt...? Or?

the only thing halting problem says is that you can't prove *everything*

feel the difference between "any" and "every"

to stop being able to prove "every" thing you only need to 1 counterexample - which Turing found

all explicit examples of unprovable things are very removed from real demand (other than "prove arbitrary program halts", which probably doesn't count as real demand anyway and more of vapordream)

xrisk 3 hours ago||
lean has no relation to the halting problem; you can think of it as a language that allows proofs to be encoded as code, and the compiler that runs that code as knowing how to enforce mathematical constraints appropriately
auggierose 3 hours ago||
there certainly is a relation; you might not be able to prove all the properties you care about because of the halting problem; but that is more a theoretical problem, not a practical one
gf000 2 hours ago||
> you might not be able to prove all the properties you care about

And just to add, this is also true of "human" mathematics as well.

nimitbhardwaj 54 minutes ago||
Nice
nurumaik 3 hours ago||
Now pls do telegram to settle "telegram shouldn't have invented their own crypto" debate once and for all
pona-a 2 hours ago|
They haven't "invent their own crypto" (please don't make me visualize Durov inventing Diffie Hellman). They took the OTP protocol and cut it in all the wrong ways, gave up on multi-device other systems easily support, and reinvented TLS while putting device IDs outside the encrypted packet, leaking persistent IDs to LAN.
whattheheckheck 3 hours ago|
"human experts work with AI to decide what to prove"

Why can't ai do this too? Is this the fundamental limit of ai?

afthonos 2 hours ago||
Because humans are the ones who care? At some point, you have to make the wish, or the genie is not going to do anything.
mjdiloreto 2 hours ago|||
Because "what to prove" is fundamentally bound in a value judgment, not merely the mechanistic application of reasoning.
mrtesthah 1 hour ago|||
Because that’s a design question. The AI has to know what it’s actually supposed to be building.
seydor 3 hours ago||
we are still involving humans in the loop, it helps them acquiesce
More comments...