Top
Best
New

Posted by ltononro 4 days ago

Progress on provable inference for large language models(www.anthropic.com)
240 points | 122 commentspage 3
ex-aws-dude 3 days ago|
> Some proposals to regulate and pace AI development rely on monitoring hardware and how it is used. For example, a provider might be required to devote its hardware to inference rather than to training a powerful new model. The provider might try to cheat by serving cheaper models or using compute in an unexpected way, freeing up resources for training. How can a regulator reliably ensure that the hardware is used for inference?

Is this even a real problem? Forcing a company to only do inference?

Sounds like Anthropic's dream scenario

lrvick 3 days ago||
Very cool. Now prove you are unable to read chat transcripts with provable end to end encryption down to the CPU registers and TPM endorsement chain of the software involved.

Provable privacy via remote attestation effectively gives you provable inference for free.

jmort 3 days ago||
What are the tradeoffs of an approach like this vs confidential computing primitives and attesting everything from untampered hardware all the way up to the served request? It seems more performant. Is it simply that they would have to reveal too much?
FrasiertheLion 3 days ago|
More so that attestation requires the model to be running on secure hardware, which is hard to do across all models that a company like Anthropic serves.

In part because many of the accelerators they use may not support attestation, and even with ones that do, such as NVIDIA Blackwell, support is restricted to a single node (8 GPUs). Vera Rubin supports rack-scale encryption, but that is not broadly deployed yet and will not be the only accelerator they use to serve inference.

leafingensemble 3 days ago||
[dead]
anonymous908213 4 days ago||
> A user querying a model today cannot check that the responses came from the specific model that a developer claims is responding

Indeed, I wonder what kind of developer would make users worry about that...? Funny PR stunt coming from a company that was in the news for silently routing ML-related requests to an inferior model.

https://fortune.com/2026/06/10/anthropic-accu-claude-fable-5...

> Unlike Fable’s other restrictions, such as around cybersecurity and biology, which openly redirect users to a less powerful model with a visible notification, the system card emphasized that this is “not visible to the user.” The model still responds, but uses “interventions to limit Claude’s effectiveness” without telling the user it’s doing so.

jsrozner 3 days ago||
"The commitment to the model, and subsequent inference proofs, hide the weights of the model; hiding the architecture of the model is not a goal of this system."

Your Claude is showing.

with 4 days ago||
isn't model substitution a vendor accountability thing? and token tampering, is that not infra security?

do either of these need 70-95x the GPU time in "zksnark" proofs?

ljlolel 3 days ago||
already solved provable inference with encrypted chats on trustedrouter (using TEEs which most people don’t know about)
khalic 3 days ago||
What a great way to enforce a (still hypothetical) open models ban
sarathkavuru 3 days ago|
[flagged]