Model signing authenticates the full-precision weights W an LLM is released with, yet users run a quantized artifact Q whose producing transformation is not uniquely determined under current disclosure. We separate the non-environment inputs from the execution environment E and measure a quantity that, to our knowledge, has not been measured before: the Independent Verifiability Rate (IVR), the fraction of a popularity-sampled GGUF ecosystem that a third party can re-derive bit-for-bit. Across the 200 most-downloaded Hugging Face GGUF repositories (198 in scope), no artifact or model card records the build environment, so under a recorded-inputs protocol the IVR is 0/198. Yet the artifacts are not irreproducible. On four in-sample repositories that disclose their inputs, the card's information alone reproduces no Q4_K_M file, but two coordinates the card omits - the build's floating-point-contraction setting and, for hybrid models, the per-tensor quantization recipe - re-derive 7/8 target-quant files bit-for-bit on x86 (5/8 on arm64); the fp16 re-derives from the provider's checkpoint, but the calibration imatrix must be shipped and hash-bound, since recomputing it did not match. An older release additionally pins the instruction set and did not re-derive. Reproducing the divergence reported with llama.cpp PR #25353, we show that E is a necessary coordinate and that one compiler setting yields a unique output across the tested platforms at negligible quality cost (+0.06% perplexity). As a security characterization, benign floating-point non-determinism separates from uniformly placed nibble-flip tampering above a detection floor that grows with the environments a verifier must allow, but a targeted embedding edit evades the statistic, so the proposed protocol rests on exact re-derivation; adversarial-fp16 attacks lie outside it. We release the audit data and a re-runnable harness.
Figures & tables
Threat and regime
What re-derivation gives a verifier
Cheating quantizer, BE′=0
Sound detection: a hash mismatch implies tampering, with no false positives (A2, A4).
Cheating quantizer, BE′>0
A nonzero diff no longer implies tampering; uniformly placed nibble flips are only characterized , by a measured detection floor, and a targeted edit can evade the statistic (§ VI ).
Cheating quantizer via C (a chosen imatrix, Q=T )
Nothing beyond binding: the manifest fixes which C was used and re-derivation reproduces Q ; vetting C is outside re-derivation (§ II ).
Malicious W (adversarial fp16, Q=T )
Nothing: the quantization is honest, so re-derivation reproduces the artifact (out of scope, A1).
TABLE I: Security claim by regime. A cheating quantizer starts from a benign W but ships Q=T(Xˉ;E) ; BE′=0 means the honest output is unique over the environments rebuilders use (§ III ).
What the repositories lack
Environment/version record
198
Retrievable fp16 source (base-model rule)
33
Retrievable imatrix (2 also lack fp16)
19
Only the environment/version record
148
Representative file scored
Scale-search k/i-quant
154
TABLE II: Census decomposition over the 198 repositories. Restricting to text and multimodal LLMs by Hugging Face pipeline tag (excluding speech, video, image-generation, embedding, and classification models) leaves \textscIVR=0/165 .
arm64 default
arm64 contr. off
x86 (either)
target (tensors)
quant
card / +rec
card / +rec
card / +rec
Qwen3.5-0.8B (335)
Q4_K_M
140 / 193
260 / 335 ∗
260 / 335 ∗
Qwen3.5-2B (335)
Q4_K_M
140 / 193
260 / 335 ∗
260 / 335 ∗
Qwen3.5-4B (441)
Q4_K_M
184 / 252
341 / 441 ∗
341 / 441 ∗
Qwen3.5-0.8B (335)
Q8_0
298 / 334 †
298 / 334 †
299 / 335 ∗
Qwen3.5-2B (335)
Q8_0
298 / 334 †
298 / 334 †
299 / 335 ∗
TABLE III: Trial re-derivation: per-tensor bit-exact matches (tensor count in parentheses). card =named release + published imatrix + stock preset; +rec =card plus the deployed file’s per-tensor type map. On x86 the default build is already uncontracted, so its two builds coincide. ∗ =fully bit-exact; —=the stock preset already equals the recipe; † =one residual tensor whose all-zero blocks store byte-different, numerically identical codes; ‡ = b3772 builds the kernels -march=native (SIMD ISA pinned; the arm64/gcc-15 builds, not shown, equal arm64/clang; x86 default 170 , contraction off 141 ); n/t=not tested; = =tensor-data hash differs in all four rebuilds (three distinct honest outputs). No model card records either coordinate.
Fig. 1: (a) Source-reconstructible rates under the two rules (both ignore E ) and the Independent Verifiability Rate under the recorded-inputs protocol; (b) the causes of non-verifiability. Both over the 198 GGUF-bearing repositories of the top 200, from metadata and model cards.
1.5B, 7B, 14B; Q4_K_M ; arm64 default vs. contraction off
Quality (§ V )
b10448
1.5B; Q4_K_M ; wikitext-2, 584 chunks
Pilot re-derivation (§ IV )
b3772
1.5B publisher f16 and imatrix; Q4_K_M ; both platforms and builds
Trial re-derivation (§ IV )
b3772 , b9222 , b10603
7B, Qwen3.5-0.8B/2B/4B, granite-4.2-3b; publisher full-precision GGUF and imatrix; Q4_K_M , Q8_0 (all 23 types for the 0.8B); stock preset vs. pinned recipe; x86_64 and arm64, default and contraction off
Imatrix regeneration (§ IV )
b9222
Qwen3.5-0.8B; card-linked corpus; arm64 (both builds) and x86_64
TABLE IV: Experimental setups. Models are Qwen2.5-Instruct unless stated; builds are CPU-only llama.cpp in a default and a -ffp-contract=off variant on arm64/clang (Apple M4) and x86_64/gcc (Linux). At b10448 the ggml quant kernels compile without -march=native , so the x86 default build’s quant path is baseline x86-64 (no FMA, hence uncontracted) while arm64 (NEON, FMA available) contracts by clang’s default; this is why -ffp-contract=off is the setting that makes the two architectures agree for the scale-search types (the Q8_0 all-zero-block residual at b9222 , § IV , is a separate architecture dependence). A _M preset stores its k-quant tensors in the named format, so § VI writes Q4_K for Q4_K_M .
Fig. 2: (a) Cross-architecture bit-exactness at 1.5B: default builds byte-differ for 3 of 4 types; -ffp-contract=off restores bit-identity ( 4/4 ). (b) The fraction of divergent bytes βbyte (arm64 default vs -ffp-contract=off , the single-machine proxy of § V ; Q4_K_M ) grows with model size.
Fig. 3: Security characterization (Qwen2.5-1.5B-Instruct): (a) natural vs. nibble-flip perturbation distributions (Q4_K); (b) per-tensor detection floor at a known release (arm64 contraction pair; tampered = honest rebuild plus uniformly placed one-step flips; AUC ≥0.99 in every seed) and the natural tail noise it must exceed; (c) Q4_K across environment pairs: benign difference (fraction of weights above τ ), each pair’s per-tensor floor, and the model-level threshold calibrated on the known-release pairs.
Developers download quantized large language models (LLMs) from public registries to run them locally. None of the registries we examined documented functional testing of their files. We tested all 305 eligible code-capable GGUF tags (the llama.cpp format) across 15 model lines in the official Ollama library, plus 22 popular Hugging Face community files. A 15-task code screen identified suspects, which we retested on all 164 HumanEval+ tasks. Seven official tags (2.3%) solved at most six of them. All were low-bit quantizations of models with at most four billion parameters. We rebuilt five suspect files from the original weights. All five registry files matched our uncalibrated rebuilds tensor for tensor, so none was corrupted. Exact reproducibility did not imply function. With the weights, toolchain and tensor mix held fixed, adding an importance matrix raised Qwen2.5-Coder-3B scores at three quantization levels from zero to 76-113 of 164 tasks. The same uncalibrated recipe left the smaller Qwen2.5-Coder-1.5B functional, showing that the recipe alone does not predict failure. None of 66 sampled tags that passed the screen was non-functional. We release the audit data and the quantcheck tool and recommend screening low-bit tags before publication, then calibrating or withholding failures.
Verifying claims about AI workloads is a prerequisite for credible AI governance of covert adversaries (who comply with monitoring only when detection likelihood is high), yet the apparent non-determinism of GPU floating-point arithmetic forces auditors to accept approximate output matches. Covert adversaries can exploit unverifiable degrees of freedom in monitored computation. Attack vectors include steganography, unreported modification of inference software, and covert computation via unreported batch elements. Empirically, we analyze how modern inference engines (vLLM, HF transformers) produce deterministic but non-invariant outputs, without needing to set performance-compromising determinism flags, if the right information is available for re-computation and no atomic functions are called in the backend. We demonstrate that such bitwise-precise re-computation does not require access to identical hardware, via a software-only emulation of LLM inference across multiple NVIDIA GPU variants. Thus, accumulated rounding errors can be an auditable signature of the software and hardware setup used for inference, instead of a constraint on verifiability.
Most large language models (LLMs) run on external clouds: users send a prompt, pay for inference, and must trust that the remote GPU executes the LLM without any adversarial tampering. We critically ask how to achieve verifiable LLM inference, where a prover (the service) must convince a verifier (the client) that an inference was run correctly without rerunning the LLM. Existing cryptographic works are too slow at the LLM scale, while non-cryptographic ones require a strong verifier GPU. We propose TensorCommitments (TCs), a tensor-native proof-of-inference scheme. TC binds the LLM inference to a commitment, an irreversible tag that breaks under tampering, organized in our multivariate Terkle Trees. For LLaMA2, TC adds only 0.97% prover and 0.12% verifier time over inference while improving robustness to tailored attacks by up to 48% over the best prior work requiring a verifier GPU.
Oguzhan Baser, Elahe Sadeghi, Eric Wang +5
Electrical and Computer Engineering, The University of Texas at Austin · Theseus AI Labs · Electrical and Computer Engineering, McGill University +1