Ep 1133 Research Paper 3:49 w/ Pippa & Tyler

Progress on provable inference for large language models

Anthropic's FlashProve argues that model identity can become cryptographically checkable: a provider can prove a response came from committed private weights without exposing those weights. The prototype is a meaningful systems result, but its substantial proof-generation overhead keeps the immediate use case focused on high-assurance routing, evaluation, and security rather than ordinary chat.

Embed this episode

Paste this on any site — the player is a self-contained iframe with no cookies or trackers.

<iframe src="https://sandrise.io/exploring-next/embed/1133"
  width="100%" height="180" style="max-width:640px;border:0;border-radius:12px;overflow:hidden"
  title="Exploring Next — Episode 1133 audio player"
  loading="lazy" allow="autoplay" referrerpolicy="strict-origin-when-cross-origin"></iframe>
Embed & API docs →
Script GPT-5.6 terra Voice Inworld TTS 2

Transcript

Pippa The interesting part of FlashProve is not that an A I can answer you. It is that you could check which model actually answered you.

Tyler Yeah. Anthropic is tackling a trust gap that feels mundane until money or security is attached to it. A router can claim it sent your request to a particular model, but output quality alone cannot prove that claim.

Pippa And that has a very direct product consequence. If somebody pays for a specific model, or an evaluator is testing one before release, a polished answer from a cheaper substitute is still the wrong thing.

Tyler Their mechanism is a zkSNARK, which is a cryptographic certificate. The provider proves it ran a specified computation correctly, while zero-knowledge means the certificate does not reveal the private model weights.

Pippa Mm-hm.

Tyler FlashProve commits to a model, then attaches a proof to an inference result. The verifier can check that proof much more cheaply than rerunning the model. That is the central argument, and technically it is a cleaner target than trying to infer identity from model behavior.

Pippa The reported performance is what stops this being purely cryptography-for-cryptography's-sake. On Llama 4 Maverick, a four-hundred-billion-parameter mixture-of-experts model, they report about one hundred ten proof tokens per second on one H100.

Tyler But the cost is very real. For one-thousand-twenty-four-token requests, proofs are about nineteen megabytes, and generating them takes roughly seventy to ninety-five times the GPU time of serving comparable prompts through vLLM. So this is not an invisible checkbox on every chat response.

Pippa Right.

Tyler The article stretches that into pacing claims, where a provider could show regulators certificates for inference traffic. That part has a hard boundary in the text itself: proving inference happened does not prove the same hardware was not training something on the side.

Pippa Which is a good example of the argument holding up without doing everything people might want. Tyler, I buy it more as an integrity tool than as a complete account of what a data center is doing.

Tyler Same. The strongest case is narrower: tamper detection, independent evaluation, and model routing where identity matters. A proof says the committed weights produced this output under the protocol. It cannot certify every surrounding operational claim.

Pippa Okay okay.

Tyler I also like that they did not ask people to trust Claude's optimization choices by default. They released executable code, a fast C++ verifier, a verifier in Lean, and Lean proofs of top-level mathematical claims. Lean is a formal language where the theorem checker verifies each logical step.

Pippa That makes the release more than a benchmark number. The code and proofs are under Apache two point zero, so the useful next question is whether outside teams can audit the assumptions, reproduce the speed, and find where the formal boundary stops.

Tyler Mm-hm.

Pippa The named artifact is FlashProve, and the package includes the prover plus both verifiers. I would be especially curious about deployment tests where a routing service has to carry these proofs without turning every response into a freight shipment.

Tyler Exactly. The reported result is a prototype on a particular model and setup, not a proof that all model serving can absorb this overhead. Still, moving from orders-of-magnitude slower systems to this range is a meaningful systems step.

Pippa My read is that provable inference becomes valuable before it becomes universal. It gives high-stakes customers a way to verify the thing they purchased, instead of treating model identity as a promise buried in an API contract.

Tyler And unusually, the authors make their intermediate status explicit. That is more useful than declaring the trust problem solved, because the unsolved parts are where the next engineering work actually lives.

Pippa I’ll take a cryptographic certificate over a pinky promise, Tyler. See you next time.