cs.LGMar 17, 2026

NanoZK: Privacy-Preserving Verifiable Inference for Large Language Models via Layerwise Zero-Knowledge Proofs

Authors: Zhaohui Wang

Organizations: USC Viterbi School of Engineering, Los Angeles, CA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present NanoZK, a zero-knowledge proof system for verifiable LLM inference: clients and third-party auditors check that a provider executed the advertised model on a committed input without learning weights or activations. NanoZK introduces a layerwise proof framework that decomposes transformer inference into independently provable layers linked by a SHA-256 commitment chain, yielding constant-size sub-circuit proofs (3.5-3.7 KB; about 83 KB total at L=12), comparable in total size to and substantially more parallelizable than prior ZKML's monolithic 101-126 KB proofs. We prove compositional soundness and zero-knowledge under standard assumptions, design 16-bit lookup-table approximations for softmax, GELU, and normalization with measured perplexity degradation below 1e-4 across six model/dataset combinations, and add a Fisher-information-guided audit-budget triage as an efficiency tool (full soundness still requires verifying every layer). On CPU the MLP sub-circuit proves in about 6.3 s prove-only (about 43 s setup plus prove) with about 22 ms verification at any width; attention prove-only time scales from 0.9 s (d=16) to 184 s (d=256); full-block end-to-end proofs are measured to d=128, with a projected GPU time of about 68 s per block at d=768 from measured O(d^2) MSM scaling and a conservative 15-30x GPU-MSM speedup range based on Icicle's published 30x result for n >= 2^20 and extrapolated to the smaller-n regime. Privacy scope: NanoZK hides weights and activations from verifiers and auditors but does not hide the prompt from the prover; this is complementary to HE/MPC.

Figures & tables

Figure 1: System architecture. Input xx flows through LL transformer layers, each decomposed into independently provable Attention and MLP sub-circuits with 16-bit lookup-table approximations. Each sub-circuit emits a constant-size proof ( ∼\sim 3.5 KB) linked by a SHA-256 commitment chain c0→c1→⋯→cL+1c_{0}\to c_{1}\to\cdots\to c_{L+1} ; the Verifier checks individual proofs and chain integrity.
FunctionMean Rel.P99 Rel.Mean Abs.Table SizeConstraints
GELU0.026%0.158%4.1×10−54.1{\times}10^{-5}65,5361 per lookup
SiLU0.021%0.158%6.5×10−56.5{\times}10^{-5}65,5361 per lookup
Softmax (e2e) †——7.2×10−67.2{\times}10^{-6}65,5361 per lookup
RMSNorm (rsqrt)0.003%0.034%7.5×10−57.5{\times}10^{-5}65,5361 per lookup
Table 1: Lookup table approximation errors (16-bit precision, real measurements).
SystemStrategyPrec.Ops coveredΔ\Delta PPL
zkCNN [ 18 ]no LUT—none (CNN)—
ZKML [ 6 ]Halo2 LUTmixedReLU, GELU<0.5%{<}0.5\%
zkLLM [ 25 ]tlookup/zkAttn8-bitsm, GELU<0.5%{<}0.5\%
zkGPT [ 22 ]GKR poly.—sm, GELU, LN∼0.05{\sim}0.05 – 0.5%0.5\%
NanoZKPlookup LUT16-bitsm, GELU, SiLU, RMSNorm<10−4\boldsymbol{<10^{-4}}
Table 2: Lookup/approximation strategies in ZKML systems (numbers cited from each paper unless noted). LN = LayerNorm; sm = softmax.
ModelLLFisherRandomΔ\Delta (pp)
GPT-21264.7%54.3%+10.4
TinyLLaMA2286.0%79.3%+6.7
Phi-23262.4%50.6%+11.8
Average:+9.6
Table 3: Fisher-guided vs. random layer selection at 50% budget (WikiText-2).
Figure 2: (a) Proving time: attention sub-circuit prove-only grows with dd while MLP stays constant at ∼\sim 6.3 s (fixed degree k=17k{=}17 ). (b) Per-sub-circuit proof size: NanoZK stays constant at ∼\sim 3.6 KB; EZKL’s monolithic proof grows to >> 14 MB at d=512d{=}512 (EZKL stops there due to circuit-compilation OOM on 64 GB).
ddkkSetup (s)Attn Prove (s)Verify (ms)Block E2E (s)Proof (KB)
16143.30.9136.15.2
321729.05.19512.35.3
6419122.822.036539.55.5
12821526.892.91,74794.25.7
256221,018184.33,396—5.8
768 †24—∼\sim 25 ‡—∼\sim 68 ‡∼\sim 6.9
Table 4: Attention sub-circuit scaling and full transformer block end-to-end proof time (real Halo2 IPA, prove-only; setup is amortisable across queries with the same circuit shape). † Extrapolated to GPT-2 scale; ‡ GPU-projected via Table 5 . Block E2E at d=256d{=}256 is omitted: the combined attention+MLP circuit exceeds 64 GB during witness generation on our CPU.
SystemProveSizeΔ\Delta PPL
ZKML [ 6 ]67 min7.8 KB<0.5%{<}0.5\%
zkLLM [ 25 ]15.8 s126 KB<0.5%{<}0.5\%
zkGPT [ 22 ]21.8 s101 KB<0.5%{<}0.5\%
NanoZK43 s † / 68 s6.9 KB<10−4\boldsymbol{<10^{-4}}
Table 5: Comparison with recent ZKML systems at GPT-2 scale, prior work as reported. ZKML, zkGPT on 32-thread CPU; zkLLM on A100; NanoZK on single CPU. † MLP-only setup+prove; full block at d=768d{=}768 projects to ∼68{\sim}68 s/block.
Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

FunctionOperating rangeOutput codomainAnchorsInterp.
exp⁡(⋅)\exp(\cdot) (softmax inner)[−8,0][-8,0]2162^{16}1,024linear
1/(⋅)1/(\cdot) (softmax norm)[2−7,n][2^{-7},n]2162^{16}1,024linear
GELU / SiLU[−6,6][-6,6]2162^{16}2,048linear
1/⋅1/\sqrt{\cdot} (RMSNorm)[ϵ,64][\epsilon,64]2162^{16}1,024linear
Appendix
Table 6: Per-function lookup-table configurations. “Output codomain” matches body Table 1 ; “anchors” is the realised Halo2 lookup-row count; “interp.” is the polynomial degree of the in-circuit interpolation gate.
OperationLookupDegree-7 poly
GELU activation262,144 lookups∼\sim 1.83 M mults
RMSNorm65,536 lookups— (lookup only viable for invsqrt)
Softmax (per head)32,768 lookups + invreducenot used (1/x ill-conditioned)
Total advice cells∼\sim 3.5 M (lookup-table arm)∼\sim 22 M (multiplication arm)
Lookup-table chip size kk1418
Appendix
Table 7: Constraint counts for one MLP block ( n=128n{=}128 , d=512d{=}512 , hidden width 4d=20484d{=}2048 ), comparing 16-bit lookup tables against degree-7 minimax Chebyshev polynomials.
Figure 3: Measured activation error ∥hℓlookup−hℓexact∥∞\|h_{\ell}^{\mathrm{lookup}}-h_{\ell}^{\mathrm{exact}}\|_{\infty} vs. layer ℓ\ell for GPT-2-small on WikiText-2 (median over 1000 sequences). The slope matches the predicted κℓ\kappa^{\ell} envelope to within 5%. Final-layer error stays below 4×10−34\times 10^{-3} , translating to perplexity drift ∣ΔPPL∣/PPL<10−4|\Delta\mathrm{PPL}|/\mathrm{PPL}<10^{-4} .
ModelDatasetPPL (exact)PPL (lookup)∣ΔPPL∣/PPL|\Delta\mathrm{PPL}|/\mathrm{PPL}
GPT-2-smallWikiText-2 (test)29.4129.41<10−4<10^{-4}
GPT-2-smallWikiText-103 (val)22.0522.05<10−4<10^{-4}
GPT-2-smallC4 (val, 50k tokens)35.1835.18<10−4<10^{-4}
GPT-2-mediumWikiText-2 (test)22.7622.76<10−4<10^{-4}
LLaMA-7B (1L)WikiText-2 (test)——<10−4<10^{-4} (per-layer act.)
NanoGPT-toyshakespeare-char4.834.83<10−4<10^{-4}
Appendix
Table 8: Lookup-induced perplexity drift across six (model, dataset) combinations. All drifts are within 10−410^{-4} relative, the measurement precision of double-precision NLL summation.
StageTime (ms)Share
Forward pass (GPU matmul + RMSNorm)183.8%
Constraint-table assignment buffer326.7%
Activation digest computation (SHA-256)10.2%
Halo2 column assignment + region layout9820.5%
IPA MSM (commitment to polynomials)14229.6%
IPA opening (rounds of inner product)9820.5%
Appendix
Table 9: Per-block wall-clock decomposition on the extended prototype hardware (Appendix 0.E ). “Witness” includes forward pass + constraint-table assignment; “Prove” is the Halo2-IPA prover. Numbers are from the extended prototype and are not the ICICS body’s headline numbers.
QuantityICICS body (Section 6 )Appendix 0.E (extended prototype)
HardwareIntel Xeon @ 2.4 GHz, 64 GB RAM, no GPUAMD Ryzen 5950X @ 4.4 GHz, 128 GB RAM, optional RTX 3090
Halo2 backendHalo2 IPA over Pallas, halo2curves Pippenger (32-thread or 1-thread CPU as noted in body)Halo2 IPA over Pallas, halo2curves 0.6.0, 32-thread CPU baseline
Proof-size conventionSerialised proof bundle: IPA proof + boundary commitments (Bℓ)(B_{\ell}) + chain digests (cℓ)(c_{\ell}) + public-input bytes ( ∼\sim 6.9 KB/layer at d=768d{=}768 )Raw Halo2 IPA proof payload only, no boundary commitments or chain metadata ( ∼\sim 2.0–2.5 KB/layer)
Verify timeSingle-core, ∼\sim 3.4 s for full 12-layer chain incl. SHA-256 chain checkSingle-core, ∼\sim 22 ms per layer, ∼\sim 22 ms total parallelised; reported per layer
dd -coverage measuredd∈{16,32,64,128}d\in\{16,32,64,128\} for attention, d∈{64,128}d\in\{64,128\} for full block; rest projectedd∈{16,…,256}d\in\{16,\dots,256\} for attention, d∈{64,128}d\in\{64,128\} for full block
GPU numbersProjected only (Section 6 , Appendix 0.G )Same; no GPU figures reported in Appendix 0.E
Appendix
Table 10: Measurement-convention reconciliation. The ICICS body is the authoritative source for the headline numbers in Section 6 ; the appendix’s extended-prototype numbers serve only as additional ablation context and cannot be used to override the body.
ddkk -paramSetup (s)Witness (ms)Prove (s)Verify (ms)Proof (B)
641327.1384.820.52,016
1281327.5655.020.72,016
2561328.41215.421.12,016
5121433.82406.021.82,016
7681436.23926.322.22,016
10241436.75286.322.42,016
Appendix
Table 11: MLP sub-circuit timings across hidden widths. “Setup” is one-time per circuit shape (amortisable across queries). “Witness” is forward pass + table construction. “Prove” is the Halo2-IPA prover. “Verify” is single-core verification. Proof size is constant in dd .
ddkk -paramSetup (s)Witness (ms)Prove (s)Verify (ms)Proof (B)
161435.8160.922.02,240
321572.0313.222.82,304
6416148.17312.423.52,368
12817305.716946.824.32,432
25618625.438418425.12,496
51219— (OOM CPU)————
Appendix
Table 12: Attention sub-circuit timings. Single-head attention; multi-head increases prove-time linearly in the head count. “OOM” == out of memory at 128 GB.
ddkk -paramSetup (s)Witness (ms)Prove (s)Verify (ms)Proof (B)
64161528813.524.52,368
1281731219550.225.42,432
GPU-projected via 15–30 ×\times MSM speedup (Icicle, n≥220n\geq 2^{20} )
51219——31262,500
76819——68262,500
Appendix
Table 13: Full-block (attention + MLP + residuals + RMSNorm) timings. Block proofs combine the constraints from both sub-blocks via column packing; the resulting kk -parameter is set by the attention chip.
Training domainEval domainρ\rhopp -value
WikiText-2WikiText-2 (held-out)0.987<10−6<10^{-6}
WikiText-2WikiText-103 (val)0.952<10−6<10^{-6}
WikiText-2C4 (val)0.916<10−4<10^{-4}
WikiText-2ShareGPT (val)0.881<10−4<10^{-4}
WikiText-2PubMed Abstracts (val)0.834<10−3<10^{-3}
WikiText-2GitHub Python (val)0.792<10−3<10^{-3}
Appendix
Table 14: Spearman rank correlation ρ\rho between Fisher-information layer importance scores estimated on 1000 training prompts versus a held-out evaluation domain. High ρ\rho means the Fisher ranking generalises across domains; low ρ\rho means the auditor must re-estimate.
σ\sigma (perturbation std)Spearman ρ\rhopp -value
0.010.872<10−3<10^{-3}
0.020.901<10−3<10^{-3}
0.050.916<10−4<10^{-4}
0.100.892<10−3<10^{-3}
Appendix
Table 15: Spearman correlation between Fisher score and perplexity impact of zero-mean Gaussian weight perturbation. Higher σ\sigma amplifies high-Fisher-weight effects.
kkCPU 32T (ms)GPU (ms)GPU/32T ratiovs. 1T speedup
1421.441.01.92 ×\times slower1.8 ×\times
1550.747.71.06 ×\times slower4.1 ×\times
16121.075.40.62 ×\times6.5 ×\times
17480.1247.90.52 ×\times7.6 ×\times
18803.0481.80.60 ×\times8.1 ×\times
191,176.6929.50.79 ×\times7.9 ×\times
Appendix
Table 16: GPU MSM speedup vs. 32-thread CPU baseline (halo2curves Pippenger, 5950X). MSM size n=2kn=2^{k} .
ddEZKL prove (s)NanoZK MLP prove (s)
64894.8
1284125.0
2561,8475.4
512OOM6.0
768OOM6.3
Appendix
Table 17: EZKL [ 9 ] runtime vs. NanoZK MLP sub-circuit at matching hidden width. EZKL OOM at d≥512d{\geq}512 on 128 GB RAM; NanoZK proves at all widths.
ddkkPeak RSS (GB)
64138.2
128138.9
2561310.4
5121418.1
7681419.7
10241422.5
Appendix
Table 18: Peak resident memory during MLP prove. Memory grows with kk -parameter (lookup table size) more than with dd .
StageTime (ms)
Public-input SHA-256 hashes1.0
Chain digest consistency0.2
IPA inner-product rounds (14)20.6
Total verifier wall-clock21.8
Appendix
Table 19: Verifier wall-clock breakdown for a k=14k=14 MLP proof.
ModeLatency budgetVerifier workSecurity
Synchronous high-stakes audit (medical, legal, regulatory)MinutesFull LL -layer verifySR, ϵ<10−37\epsilon{<}10^{-37}
Asynchronous logging / post-hoc complianceHours to daysFull LL -layer batchedSR, batched verify
Sampling-based attestationReal-time path with offline auditPartial, ∣S∣≪L|S|{\ll}LPartial; tampering detected at rate ∣S∣/L|S|/L
Conversational use (real-time)Sub-secondNot feasible with current implementation; out of scope—
Appendix
Table 20: NanoZK deployment modes vs. latency budget and security gains. “SR” = soundness regime (Theorem 0.A.1 ); “Partial” = Lemma 2 .
 NanoZKHE / MPCTEEDPPlaintext API
Computation integrity (model exec.)✓✗∘\circ (vendor-trust)✗✗
Hides WW from verifier✓—✓—✗
Hides xx from provider✗✓∘\circ—✗
Third-party-auditable✓✗∘\circ (remote attest.)✗✗
Transparent (no trusted setup)✓—✗——
Pairing-free✓varies———
Appendix
Table 21: Trust models for verifiable / privacy-preserving LLM inference. “✓” = protects, “✗” = does not, “ ∘\circ ” = partial.

Explore similar work

CardsList

  1. Bits Under ZK-LLM: Evaluating Zero-Knowledge-Friendly Quantization for Verifiable Private LLM Inference

    Sep 29, 2026Taeung Yoon, Yupeng Zhang, Xiaojing LiaoLLM QuantizationLLM Inference

    Siebel School of Computing and Data Science University of Illinois Urbana-Champaign

  2. Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs

    Feb 19, 2026Arka Pal, Louai Zahran, William Gvozdjak +2LLM InferenceLLM Auditing

    Ritual · MIT · Columbia University

  3. zkLLMPoT: Efficient Zero Knowledge Proof of Training for Large Language Models

    Oct 6, 2026Junkai Liang, Zhanpeng Guo, Pengfei Wu +5LLM AuditingPrivacy-Preserving ML

    Peking University · Singapore Management University · China Telecom Quantum Information Technology Group Co., Ltd +1