A Note on Non-Composability of Layerwise Approximate Verification for Neural Inference

Or Zamir

cs.CRFeb 17, 2026

A Note on Non-Composability of Layerwise Approximate Verification for Neural Inference

Authors: Or Zamir

Organizations: Tel Aviv University

Abstract

A natural and informal approach to verifiable (or zero-knowledge) ML inference over floating-point data is: ``prove that each layer was computed correctly up to tolerance $δ$ ; therefore the final output is a reasonable inference result''. This short note gives a simple counterexample showing that this inference is false in general: for any neural network, we can construct a functionally equivalent network for which adversarially chosen approximation-magnitude errors in individual layer computations suffice to steer the final output arbitrarily (within a prescribed bounded range).

Explore similar work

Bit-Exact AI Inference Verification Without Performance Tradeoffs
May 29, 2026Naci CankayaNeural Network Verification LLM Inference
Machine Intelligence Research Institute
Towards Verifiable AI with Lightweight Cryptographic Proofs of Inference
Mar 19, 2026Pranay Anchuri, Matteo Campanelli, Paul Cesaretti +4Neural Network Verification Reasoning Verification
Offchain Labs · University of Tartu, Estonia · CUNY Graduate Center +1
NanoZK: Privacy-Preserving Verifiable Inference for Large Language Models via Layerwise Zero-Knowledge Proofs
Mar 17, 2026Zhaohui WangPrivacy-Preserving ML Privacy-Preserving Language Models
USC Viterbi School of Engineering, Los Angeles, CA, USA

May 29, 2026J/K move · Enter open · S save

Bit-Exact AI Inference Verification Without Performance Tradeoffs

Naci Cankaya

Machine Intelligence Research Institute

Neural Network Verification LLM Inference LLM Auditing

Open paper AskPDF arXiv

Verifying claims about AI workloads is a prerequisite for credible AI governance of covert adversaries (who comply with monitoring only when detection likelihood is high), yet the apparent non-determinism of GPU floating-point arithmetic forces auditors to accept approximate output matches. Covert adversaries can exploit unverifiable degrees of freedom in monitored computation. Attack vectors include steganography, unreported modification of inference software, and covert computation via unreported batch elements. Empirically, we analyze how modern inference engines (vLLM, HF transformers) produce deterministic but non-invariant outputs, without needing to set performance-compromising determinism flags, if the right information is available for re-computation and no atomic functions are called in the backend. We demonstrate that such bitwise-precise re-computation does not require access to identical hardware, via a software-only emulation of LLM inference across multiple NVIDIA GPU variants. Thus, accumulated rounding errors can be an auditable signature of the software and hardware setup used for inference, instead of a constraint on verifiability.

Abstract

Explore similar work

Bit-Exact AI Inference Verification Without Performance Tradeoffs

Towards Verifiable AI with Lightweight Cryptographic Proofs of Inference

NanoZK: Privacy-Preserving Verifiable Inference for Large Language Models via Layerwise Zero-Knowledge Proofs