Users of artificial intelligence services, including those powered by large language models and other transformer-based models, need assurance that inference and training follow the declared computation without exposing private models or data. Zero-knowledge proofs can provide this assurance, but proof generation remains expensive. Hardware acceleration research often reports gains for individual cryptographic operations, leaving their effect on the full cost of proving neural-network workloads, particularly large transformer models, unclear. We introduce a four-layer framework linking verifiable AI workloads, proof systems, cryptographic kernels and hardware platforms, use it to review the literature, and add original measurements of six provers and seven kernels to identify which costs matter in practice. Our measurements show that the dominant cost changes with the proof system and input length. In low-rank fine-tuning, the cost of opening commitments to fixed model weights remains nearly constant across input lengths from 64 to 2048, while the work associated with intermediate values grows. A 13-billion-parameter fine-tuning configuration exceeds the capacity of a 40-gigabyte graphics processor but completes on an 80-gigabyte device, showing why capacity must be assessed alongside execution time. Our kernel profiling further identifies instruction issue and synchronization limits that a bandwidth-only model misses. We use these findings to assess graphics processors, reconfigurable logic, custom chips and existing artificial intelligence accelerators. The resulting research agenda connects proof-friendly numeric formats, reuse of fixed-weight commitments, heterogeneous execution and shared benchmarks to the measured bottlenecks. Effective acceleration requires selecting hardware and proof-system optimizations around the cost profile of the target workload.
Figures & tables
Fig. 1: Verifiable AI as a service. The provider attaches a zero-knowledge proof to each result; verification is cheap, while proof generation dominates the cost.
Task
What the proof certifies
System
Model / scale
Reported cost (normalized)
Platform
Verifiable inference
The output was produced by the committed model on the given input (one forward pass or a whole generated sequence)
zkLLM [ 9 ]
up to 13B
< 15 min per 2048-token forward pass ( ≈ 0.39 s/token a ); proof < 200 kB
1 × A100 (40 GB)
zkGPT [ 10 ]
GPT-2
21.8 s per inference ( ≈ 0.68 s/token b )
CPU server (Xeon 6126)
DeepProve [ 11 ]
GPT-2 124M, Gemma 3 270M
174 / 86 tokens per minute, whole sequence (0.34–0.47 / 0.69–0.94 s/token c ); verify 1–3.7 s
Each update step (forward, backward, update) ran on the committed data and weights
VeriLoRA [ 13 ]
3B–13B (LoRA)
122–249 s per step ( ≈ 10 3× plain step); commitment 156–554 s
1 × A100 (80 GB)
TABLE I: Three classes of verifiable-AI tasks and representative zero-knowledge systems. All costs are as reported by the authors, on the stated hardware, at different security levels, fields, quantization settings and input lengths, and are not directly comparable; values in parentheses are normalized per token or per step (notes below). Section IV reports measured cost shares under a common taxonomy.
Fig. 2: The three building blocks that dominate proofs of neural networks.
Fig. 3: Four-layer framework. Each verifiable AI workload (Layer 1) is expressed in a proof system (Layer 2), whose cost is concentrated in a few cryptographic kernels (Layer 3) that run with different efficiency on hardware platforms (Layer 4). Kernel colors give the limiting resource as measured on a GPU (Section IV-B ); B1–B4 mark the bottleneck introduced at each connection between layers; S1–S4 mark where structure specific to AI workloads can be exploited.
Kernel
Bound by
CPU
GPU
FPGA
ASIC
AI acc.
SmartNIC
Main reason
MSM
compute
○
● [ 16 , 17 , 52 ]
◐ [ 18 , 53 , 24 ]
● [ 15 , 19 ]
◐ [ 54 ]
–
Many independent elliptic-curve additions; high arithmetic intensity, parallelizes well.
NTT
memory
○
● [ 16 , 20 ]
◐ [ 24 ]
● [ 15 , 19 ]
● [ 54 ]
–
Butterfly stages with strided access; limited by memory-issue queues and occupancy rather than DRAM bandwidth or integer arithmetic.
Sumcheck
int. issue
◐ [ 21 ]
● [ 22 , 21 , 45 ]
◐ [ 55 ]
◐ [ 56 , 57 , 55 , 58 ]
–
–
Each round streams over all evaluations with little reuse; low-degree rounds saturate integer issue before bandwidth, and the best primitive depends on degree and on-chip memory [ 51 ] .
Lookup
sync., issue
◐
● [ 9 ]
–
–
–
–
Random table accesses plus batched field inversions (LogUp [ 40 ] ); barriers in the inversion, not DRAM, cap the GPU gain.
Hash & Merkle
bit-level
◐
● [ 22 ]
● [ 46 ]
● [ 59 , 58 ]
○
open
Permutation rounds and bitwise operations, limited by instruction issue rather than DRAM; tree reduction suits streaming pipelines.
Encoding
memory
◐
● [ 22 , 45 ]
◐ [ 46 ]
● [ 59 ]
◐ [ 54 ]
–
Sparse linear codes or Reed–Solomon (via NTT); off-chip bandwidth limits the gain [ 46 ] .
TABLE II: Fit between cryptographic kernels and hardware platforms. Ratings are qualitative and cite representative works. The Bound by column gives the resource that limits each kernel on a GPU as measured in Section IV-B (“int. issue”: integer-instruction issue); Fig. 4 shows measured throughput for the CPU and GPU columns.
Prover
Proof system / commitment
Workload
Role
Commit
zkLLM [ 9 ]
sumcheck, lookup; Hyrax-style (BLS12-381)
W1
full scale
993311e
DeepProve [ 11 ]
GKR, lookup; HyperKZG (BN254)
W1, W2
small model
9d1a53e
VeriLoRA [ 13 ]
sumcheck, lookup; Hyrax-style (BLS12-381)
W3
full scale
7cc0977
VeriDP [ 5 ]
GKR; incremental verification
W4
small scale
cf86a46
Expander [ 63 ]
GKR; Orion / Hyrax / KZG
one block
commitment comparison
096581e
ZKTorch [ 64 ]
KZG blocks, accumulation (BN254; CPU)
MLP block
non-sumcheck point
63b9c68
TABLE III: Provers used in the measurements.
Fig. 4: Throughput of the seven Layer-3 kernels (log scale) at 216 – 226 elements on the A100 GPU and the host CPU. Solid lines: GPU execution time with data resident on device; dotted: including host–device copies; dashed: CPU. Color gives the field (or hash); markers separate sumcheck degrees ( d ), Reed–Solomon rates ( r ), the two MSM libraries and hashing from Merkle-tree construction. Hollow markers denote runs whose repetition spread exceeded 10%. No GPU NTT is shown for Goldilocks because the publicly usable library release lacks that field.
Fig. 5: Roofline of the kernels at 224 elements on the A100: thread-level integer-instruction rate, summed over the ALU and FMA pipelines, against integer instructions per DRAM byte. The diagonal is the measured copy bandwidth ( ≈ 1.36 TB/s); the solid and dashed lines are the measured single-pipe ( ≈ 9.66 × 10 12 inst/s) and dual-pipe ( ≈ 1.91 × 10 13 ) issue limits, so points may lie above the solid line. Left of the ridge point ( ≈ 7 inst/byte) bandwidth can limit a kernel, right of it instruction issue; a point far below both is limited by dependencies, synchronization or occupancy. Labels: sc sumcheck, lk lookup, fmul field mult., rs Reed–Solomon, p2 Poseidon2, b3 BLAKE3; bb BabyBear, gl Goldilocks, bn BN254; i2 ICICLE v2, sp sppark.
Fig. 6: Share of proof time by cost category for each measured (workload, prover, scale) cell; bar tops give the median proof time and the lower strip shows one-off setup and verification on a logarithmic scale. Labels give the prover, the model and the input length in tokens (P+G: P prompt plus G generated tokens), the platform (40G: shared A100-40GB; 80G: exclusive A100-80GB; CPU) and the number of repetitions n .
Workload
System / scale
reps
prove (s)
verify (s)
proof
GPU pk (GB)
host pk (GB)
× plain
W1
zkLLM 7B, 1024 tok
n=1
33.6
1.8e-05
n/a
21.2
0.84
341 ×
W1
zkLLM 7B, 2048 tok
n=2
47.8
1.5e-05
n/a
29.3
0.84
249 ×
W1
DeepProve 270M, 64 tok
n=3 † 3%
270
2.8
12.66 MB
21.7
60.2
n/a
W1
DeepProve 270M, 256 tok
n=2 † 6%
509
3.28
15.29 MB
65.6
95.4
n/a
W1
DeepProve 270M, 1024 tok
n=1 † 4%
1278
5.12
18.13 MB
24.4
260.4
n/a
W1
DeepProve 270M, 512 tok
n=3, CPU † 67%
1615
3.8
16.66 MB
–
146.4
n/a
TABLE IV: Measured prover overhead per workload, system, and scale. “prove” is the median sum of the eight taxonomy categories with one-off setup and verification excluded. Fig. 6 shows the decomposition. × plain is the ratio over the unmodified PyTorch step measured on the same machine; n/a marks cells with no compatible baseline. A representative subset of cells is shown here; Fig. 6 shows the full set. The reps column gives the number of repetitions n (CPU: host-CPU-only runs). † DeepProve times trace generation (executing the quantized model to build the witness) separately, before its proving timer; to keep this column comparable across provers we report end-to-end prover time, and the superscript gives the share of it spent in trace generation.
Fig. 7: Engineering-optimization waterfall for zkLLM on W1 (7B, one layer, input length 1024, A100-80GB, exclusive server). Each middle bar shows the per-category change against the previous bar; red-edged segments are regressions. Labels give the median proof time (one-off setup and verification excluded) and the slowdown over the same step in unmodified PyTorch on the same machine ( × plain). Medians of items 4 and 5 pool repetitions from the shared and the exclusive server (item 4 alone on the exclusive server: 30.8 s).
Work
Platform
Target proof system
Accelerated kernels
Reported speed-up (baseline)
AI
GZKP [ 16 ]
GPU
pairing-based SNARK
MSM, NTT
up to 48.1 × (MINA, V100); up to 17.6 × (bellperson, Zcash, four V100)
–
cuZK [ 17 ]
GPU
pairing-based SNARK
MSM
MSM up to 2.94 × , overall up to 4.86 × (prior GPU systems)
–
BatchZK [ 22 ]
GPU
sumcheck + linear-code PCS
sumcheck, Merkle tree, encoder
> 259.5 × throughput (GPU systems); 9.52 proofs/s for verifiable ML
✓
ZKPoG [ 47 ]
GPU
Plonkish, incl. witness generation
end-to-end
22.8 × (CPU), 12.7 × (GPU)
–
Chimera [ 21 ]
GPU+CPU
Spartan
sumcheck
7.96 × sumcheck region; 1.27–1.64 × end-to-end
–
TensorZKP [ 45 ]
GPU tensor cores
HyperPlonk + Brakedown
sumcheck, encoder, field arithmetic
955 × (CPU), 36.2 × (SIMT GPU)
–
TABLE V: Representative hardware acceleration of zero-knowledge provers. Speed-ups are as reported by the authors against the baselines named in parentheses and are not comparable across rows. The last column marks whether a neural-network model or circuit is part of the evaluation.
Fig. 8: Roadmap toward practical verifiable AI. Numbers refer to the open problems in Section VI .
When large AI models are deployed as cloud-based services, clients have no guarantee that responses are correct or were produced by the intended model. Rerunning inference locally is infeasible for large models, and existing cryptographic proof systems -- while providing strong correctness guarantees -- introduce prohibitive prover overhead (e.g., hundreds of seconds per query for billion-parameter models). We present a verification framework and protocol that replaces full cryptographic proofs with a lightweight, sampling-based approach grounded in statistical properties of neural networks. We formalize the conditions under which trace separation between functionally dissimilar models can be leveraged to argue the security of verifiable inference protocols. The prover commits to the execution trace of inference via Merkle-tree-based vector commitments and opens only a small number of entries along randomly sampled paths from output to input. This yields a protocol that trades soundness for efficiency, a tradeoff well-suited to auditing, large-scale deployment settings where repeated queries amplify detection probability, and scenarios with rationally incentivized provers who face penalties upon detection. Our approach reduces proving times by several orders of magnitude compared to state-of-the-art cryptographic proof systems, going from the order of minutes to the order of milliseconds, with moderately larger proofs. Experiments on ResNet-18 classifiers and Llama-2-7B confirm that common architectures exhibit the statistical properties our protocol requires, and that natural adversarial strategies (gradient-descent reconstruction, inverse transforms, logit swapping) fail to produce traces that evade detection. We additionally present a protocol in the refereed delegation model, where two competing servers enable correct output identification in a logarithmic number of rounds.
Pranay Anchuri, Matteo Campanelli, Paul Cesaretti +4
Offchain Labs · University of Tartu, Estonia · CUNY Graduate Center +1
Most large language models (LLMs) run on external clouds: users send a prompt, pay for inference, and must trust that the remote GPU executes the LLM without any adversarial tampering. We critically ask how to achieve verifiable LLM inference, where a prover (the service) must convince a verifier (the client) that an inference was run correctly without rerunning the LLM. Existing cryptographic works are too slow at the LLM scale, while non-cryptographic ones require a strong verifier GPU. We propose TensorCommitments (TCs), a tensor-native proof-of-inference scheme. TC binds the LLM inference to a commitment, an irreversible tag that breaks under tampering, organized in our multivariate Terkle Trees. For LLaMA2, TC adds only 0.97% prover and 0.12% verifier time over inference while improving robustness to tailored attacks by up to 48% over the best prior work requiring a verifier GPU.
Oguzhan Baser, Elahe Sadeghi, Eric Wang +5
Electrical and Computer Engineering, The University of Texas at Austin · Theseus AI Labs · Electrical and Computer Engineering, McGill University +1
Verifying claims about AI workloads is a prerequisite for credible AI governance of covert adversaries (who comply with monitoring only when detection likelihood is high), yet the apparent non-determinism of GPU floating-point arithmetic forces auditors to accept approximate output matches. Covert adversaries can exploit unverifiable degrees of freedom in monitored computation. Attack vectors include steganography, unreported modification of inference software, and covert computation via unreported batch elements. Empirically, we analyze how modern inference engines (vLLM, HF transformers) produce deterministic but non-invariant outputs, without needing to set performance-compromising determinism flags, if the right information is available for re-computation and no atomic functions are called in the backend. We demonstrate that such bitwise-precise re-computation does not require access to identical hardware, via a software-only emulation of LLM inference across multiple NVIDIA GPU variants. Thus, accumulated rounding errors can be an auditable signature of the software and hardware setup used for inference, instead of a constraint on verifiability.