Your agent answered, and you still can’t prove which model did it

🛠️ Deep dive · Nov 6, 2026 · ~9 min read

What shipped

On Nov 2, 2026 Anthropic published Progress on provable inference for large language models and open-sourced FlashProve under Apache 2.0.

The release includes the research post, a technical report, a protocol spec, a prover, a fast C++ verifier, a slower Lean verifier, Lean proofs of the top-level security claims and an interactive Lean visualizer.

The problem is blunt. A user who queries a model today cannot check that the tokens came from the specific weights a provider claims, rather than a lower-cost substitute slipped in by a router or a tampered stream. Output quality alone is not enough.

Cryptographic proofs of inference let a provider commit to a model and then prove that each checked response came from those committed weights, without revealing the weights.

FlashProve is that path as a research prototype, not a Claude product toggle. The measured demo runs on Meta’s Llama 4 Maverick (a 400B-parameter mixture of experts) in a third-party public 4-bit conversion. The weights are not in the release. Readers download them separately under Meta’s Llama 4 license.

Why agent buyers care

Agent stacks already pay routers, aggregate providers and score models in eval harnesses. Anthropic’s post maps four trust challenges that provable inference is meant to help with (Anthropic research: Progress on provable inference for large language models):

1. Pacing. Auditors can demand certificates that served requests really ran the claimed model, without standing up the provider’s full hardware stack, while weights stay hidden.
2. Tamper detection. A compromised server, or a stronger model posing as a weaker approved one, must still produce proofs bound to the expected commitment or the check fails.
3. Text provenance. A claim that a frontier model reviewed a patch can ship with a proof that those tokens came from the committed model on that input.
4. Model approval. Rules that allow only approved models can require proofs bound to those approved commitments.

The everyday version for an agent buyer is simpler. You paid for model A through a third-party route. You got tokens. You still cannot prove which weights produced them. FlashProve is the open sketch of what that proof would look like beside the reply.

The proof does not make the answer better. It makes the attribution checkable.

Commit, prove, verify

FlashProve has an offline phase and an online phase.

Offline. The prover publishes a short commitment (about 6.5 MB in the measured setup). That fingerprint pins one exact choice of weights while the weights stay hidden. Later proofs check against that commitment. In the reported setup the commitment lasts about 16 million requests, then the provider must commit again.

Offline setup on the measured machine took about 39 minutes once per model.

Online. After a batch of finished requests, the prover produces a proof file for the claim that the committed model produces these output tokens when evaluated. Anyone who holds the commitment, the prompt and the reply can check it on CPU, without the weights.

Three engineering choices drive the measured speed (Anthropic research: Progress on provable inference for large language models):

• Work scales with the MoE experts that actually ran (~17B of 400B parameters per token), while the proof still hides which experts were chosen.
• Requests are proved in batches after the fact (post-hoc), with one parallel prefill-style pass and one commitment open per batch.
• Layer weights are segmented so a single 80 GB GPU plus host memory can stream the job.

Post-hoc proving is slower to deliver assurances than an in-line prover, and it inherits recomputation complexity and non-determinism limits. Anthropic chose it so this release need not trade security for ultra-low latency.

Blast radius

The blast radius today is research and open-source tooling, plus the design conversation for anyone who already mistrusts opaque routing. It is not a Claude feature flag.

What the proof covers (and what it doesn’t)

Lead with the limits. Anthropic lists them in the research post.

Covered in the prototype

• Exact integer arithmetic on the published 4-bit weights, under greedy decoding.
• Mixture-of-Experts assumptions FlashProve uses for efficiency.
• Completeness, soundness and zero-knowledge for the Lean-stated properties in the random-oracle model (weights stay hidden; architecture does not).
• Proofs small enough to check in seconds on a multi-core CPU (~17 s on 24 threads in the measured run, including offline checks).

Not covered

• Claude product surfaces. No FlashProve proofs on Claude.ai or the Claude API from this release.
• Floating-point or sampled decoding. A float implementation or temperature sampling can diverge from the integer greedy computation the proof asserts.
• Long context. Windows stop at 1,024 tokens.
• Architecture hiding. The verifier learns the MoE layout. Hiding architecture is future work.
• Side channels. Timing and similar leaks are out of scope.
• Fast executable equivalence. Lean states the protocol properties. Anthropic does not claim the paper, Lean and the fast C++ code match in every detail. Day-to-day checks use the C++ verifier, which agreed with the Lean verifier on every proof in their tests.

One sentence to keep. FlashProve proves a specific integer greedy MoE run matched the committed weights. It does not prove an ordinary Claude chat reply.

FlashProve vs Anthropic’s own signed traffic

Anthropic’s Frontier Safety work set a public goal to prototype inference verification by September 30, 2026 (signing outputs so they are attributable to a specific set of weights).

In RSP updates Anthropic says it met that goal through a separate project that signs its own inference traffic and recomputes inference on a trusted server.

FlashProve pursues the same objective by a different method (Anthropic research: Progress on provable inference for large language models, footnote on the RSP track).

An October 19, 2026 RSP update renamed the goal from “provable inference” to “inference verification,” because high assurance of token-stream integrity can be reached several ways. FlashProve is the open cryptographic branch. The signed-traffic prototype is the branch Anthropic applied to its own serving path.

Do not collapse the two into a claim that Claude already ships zk proofs.

Trusted hardware remains complementary. This post focuses on software-only cryptography that still holds if the host environment is compromised, as long as the computational assumptions hold.

The numbers everyone quoted

All figures below are Anthropic’s reported measurements on Llama 4 Maverick 4-bit, not Agent Snacks measurements.

• About 110 tokens/second of proving on a single H100 (batch of 24 requests).
• About 18.78–19 MB of proof per 1,024-token request.
• About 70–95× the per-GPU time of vLLM on the same prompt lengths (roughly 72× in the main comparison; up to about 94× at the highest vLLM load they tried).
• About 17 seconds to verify on 24 CPU threads (about 21 seconds single-threaded), with no GPU on the client.
• Prior open-source work on the same model (VerInf) reportedly sat near 0.02 tok/s with a ~93.6 GB proof.

Anthropic frames the overhead as roughly two orders of magnitude over native computation and asks others to take the artifacts and improve them. Treat every speed claim as vendor-reported on a research configuration.

What you can do with it today

There is no consumer toggle. Builders still have a short checklist.

1. Read the post and the repo. Start with the research post and anthropics/FlashProve. The runnable path needs a large GPU machine and Meta tokenizer access.
2. Separate product claims from research claims. If a vendor claims to prove which model answered, ask which commitment ID, which decoding mode, which context length and whether verification is post-hoc or in-line.
3. Ask router vendors whether proofs or signed commitments will ever be attached, and who publishes the authoritative commitment.
4. For eval authors, decide whether post-hoc sampling of proofs is enough given the prove cost and the 1,024-token window.
5. For Claude users, watch Anthropic’s separate inference-verification notes in RSP updates. FlashProve is the open crypto cousin, not the Claude serving path.

Possible uses stay inside Anthropic’s four documented challenges. There are no independent customer deployments to cite yet. That is expected for a research prototype.

Chef’s pick: treat FlashProve as the reference design for “prove which model answered,” not as a Claude feature. If you buy inference through anyone else’s pipe, put the commitment-and-proof question into your next vendor review.

If you only use Claude directly, keep reading RSP inference-verification notes until a product surface ships proofs beside tokens.