I still can't imagine hooking up an LLM to real hardware because of exactly this.

[flagged]

[flagged]

Using the name "verifier" suggests some sort of formal verification. As far as I can see, this is just a glorified test rig.

I once saw a bug that only happened on Thursdays, and even then only very intermittently, because Thursday was day 0x04 of the week, with a bit that neatly hit a particular flag bit in a data structure that itself was only hit through a combination of pointer aliasing and a timing race. I wonder how this might be expected to cope with that?