Model signing authenticates the full-precision weights W an LLM is released with, yet users run a quantized artifact Q whose producing transformation is not uniquely determined under current disclosure. We separate the non-environment inputs from the execution environment E and measure a quantity that, to our knowledge, has not been measured before: the Independent Verifiability Rate (IVR), the fraction of a popularity-sampled GGUF ecosystem that a third party can re-derive bit-for-bit. Across the 200 most-downloaded Hugging Face GGUF repositories (198 in scope), no artifact or model card records the build environment, so under a recorded-inputs protocol the IVR is 0/198. Yet the artifacts are not irreproducible. On four in-sample repositories that disclose their inputs, the card's information alone reproduces no Q4_K_M file, but two coordinates the card omits - the build's floating-point-contraction setting and, for hybrid models, the per-tensor quantization recipe - re-derive 7/8 target-quant files bit-for-bit on x86 (5/8 on arm64); the fp16 re-derives from the provider's checkpoint, but the calibration imatrix must be shipped and hash-bound, since recomputing it did not match. An older release additionally pins the instruction set and did not re-derive. Reproducing the divergence reported with llama.cpp PR #25353, we show that E is a necessary coordinate and that one compiler setting yields a unique output across the tested platforms at negligible quality cost (+0.06% perplexity). As a security characterization, benign floating-point non-determinism separates from uniformly placed nibble-flip tampering above a detection floor that grows with the environments a verifier must allow, but a targeted embedding edit evades the statistic, so the proposed protocol rests on exact re-derivation; adversarial-fp16 attacks lie outside it. We release the audit data and a re-runnable harness.
Figures & tables
Threat and regime
What re-derivation gives a verifier
Cheating quantizer, BE′=0
Sound detection: a hash mismatch implies tampering, with no false positives (A2, A4).
Cheating quantizer, BE′>0
A nonzero diff no longer implies tampering; uniformly placed nibble flips are only characterized , by a measured detection floor, and a targeted edit can evade the statistic (§ VI ).
Cheating quantizer via C (a chosen imatrix, Q=T )
Nothing beyond binding: the manifest fixes which C was used and re-derivation reproduces Q ; vetting C is outside re-derivation (§ II ).
Malicious W (adversarial fp16, Q=T )
Nothing: the quantization is honest, so re-derivation reproduces the artifact (out of scope, A1).
TABLE I: Security claim by regime. A cheating quantizer starts from a benign W but ships Q=T(Xˉ;E) ; BE′=0 means the honest output is unique over the environments rebuilders use (§ III ).
What the repositories lack
Environment/version record
198
Retrievable fp16 source (base-model rule)
33
Retrievable imatrix (2 also lack fp16)
19
Only the environment/version record
148
Representative file scored
Scale-search k/i-quant
154
TABLE II: Census decomposition over the 198 repositories. Restricting to text and multimodal LLMs by Hugging Face pipeline tag (excluding speech, video, image-generation, embedding, and classification models) leaves \textscIVR=0/165 .
arm64 default
arm64 contr. off
x86 (either)
target (tensors)
quant
card / +rec
card / +rec
card / +rec
Qwen3.5-0.8B (335)
Q4_K_M
140 / 193
260 / 335 ∗
260 / 335 ∗
Qwen3.5-2B (335)
Q4_K_M
140 / 193
260 / 335 ∗
260 / 335 ∗
Qwen3.5-4B (441)
Q4_K_M
184 / 252
341 / 441 ∗
341 / 441 ∗
Qwen3.5-0.8B (335)
Q8_0
298 / 334 †
298 / 334 †
299 / 335 ∗
Qwen3.5-2B (335)
Q8_0
298 / 334 †
298 / 334 †
299 / 335 ∗
TABLE III: Trial re-derivation: per-tensor bit-exact matches (tensor count in parentheses). card =named release + published imatrix + stock preset; +rec =card plus the deployed file’s per-tensor type map. On x86 the default build is already uncontracted, so its two builds coincide. ∗ =fully bit-exact; —=the stock preset already equals the recipe; † =one residual tensor whose all-zero blocks store byte-different, numerically identical codes; ‡ = b3772 builds the kernels -march=native (SIMD ISA pinned; the arm64/gcc-15 builds, not shown, equal arm64/clang; x86 default 170 , contraction off 141 ); n/t=not tested; = =tensor-data hash differs in all four rebuilds (three distinct honest outputs). No model card records either coordinate.
Fig. 1: (a) Source-reconstructible rates under the two rules (both ignore E ) and the Independent Verifiability Rate under the recorded-inputs protocol; (b) the causes of non-verifiability. Both over the 198 GGUF-bearing repositories of the top 200, from metadata and model cards.
1.5B, 7B, 14B; Q4_K_M ; arm64 default vs. contraction off
Quality (§ V )
b10448
1.5B; Q4_K_M ; wikitext-2, 584 chunks
Pilot re-derivation (§ IV )
b3772
1.5B publisher f16 and imatrix; Q4_K_M ; both platforms and builds
Trial re-derivation (§ IV )
b3772 , b9222 , b10603
7B, Qwen3.5-0.8B/2B/4B, granite-4.2-3b; publisher full-precision GGUF and imatrix; Q4_K_M , Q8_0 (all 23 types for the 0.8B); stock preset vs. pinned recipe; x86_64 and arm64, default and contraction off
Imatrix regeneration (§ IV )
b9222
Qwen3.5-0.8B; card-linked corpus; arm64 (both builds) and x86_64
TABLE IV: Experimental setups. Models are Qwen2.5-Instruct unless stated; builds are CPU-only llama.cpp in a default and a -ffp-contract=off variant on arm64/clang (Apple M4) and x86_64/gcc (Linux). At b10448 the ggml quant kernels compile without -march=native , so the x86 default build’s quant path is baseline x86-64 (no FMA, hence uncontracted) while arm64 (NEON, FMA available) contracts by clang’s default; this is why -ffp-contract=off is the setting that makes the two architectures agree for the scale-search types (the Q8_0 all-zero-block residual at b9222 , § IV , is a separate architecture dependence). A _M preset stores its k-quant tensors in the named format, so § VI writes Q4_K for Q4_K_M .
Fig. 2: (a) Cross-architecture bit-exactness at 1.5B: default builds byte-differ for 3 of 4 types; -ffp-contract=off restores bit-identity ( 4/4 ). (b) The fraction of divergent bytes βbyte (arm64 default vs -ffp-contract=off , the single-machine proxy of § V ; Q4_K_M ) grows with model size.
Fig. 3: Security characterization (Qwen2.5-1.5B-Instruct): (a) natural vs. nibble-flip perturbation distributions (Q4_K); (b) per-tensor detection floor at a known release (arm64 contraction pair; tampered = honest rebuild plus uniformly placed one-step flips; AUC ≥0.99 in every seed) and the natural tail noise it must exceed; (c) Q4_K across environment pairs: benign difference (fraction of weights above τ ), each pair’s per-tensor floor, and the model-level threshold calibrated on the known-release pairs.
Developers increasingly run large language models locally by pulling quantized GGUF artifacts from public registries, yet nothing in the distribution pipeline functionally tests these conversions before they reach users. We executed 327 quantized code-capable model artifacts: 305 from the official Ollama library, spanning 15 model lines at every eligible quantization level at or under 8 GB, and 22 from the most-downloaded community repositories on HuggingFace. Each ran a 15-task smoke suite calibrated so that healthy artifacts pass while a known-broken one fails; suspects then faced full 164-task evaluation, a second inference backend, an independent distributor's conversion of the same model and quantization as referee, and, for community files, re-testing under the artifact's own template. The official library carries five silently defective artifacts, a batch of four Qwen2.5-Coder-3B conversions and one phi3.5-mini conversion, that solve zero of 164 tasks and zero of the smoke suite on both backends while independent conversions of the same models work: 1.6% of official artifacts, 2 of 29 model-and-size conversion groups. The adjudication chain cleared small-model artifacts that a naive threshold would condemn as broken when they are merely collapsed by extreme quantization, and it exposed two older community conversions that degrade badly on CUDA yet pass on Metal: not defective files but backend-dependent failures, a third phenomenon no registry currently tests for. Two confirmed defects produce output whose surface statistics sit inside the healthy range, invisible to any low-noise heuristic short of execution. We release the audit dataset, the quantcheck acceptance-testing tool, and disclosure reports for every confirmed defect (https://github.com/aditi-p31/quantcheck), and argue that model registries need the acceptance gate that package registries already run.
Verifying claims about AI workloads is a prerequisite for credible AI governance of covert adversaries (who comply with monitoring only when detection likelihood is high), yet the apparent non-determinism of GPU floating-point arithmetic forces auditors to accept approximate output matches. Covert adversaries can exploit unverifiable degrees of freedom in monitored computation. Attack vectors include steganography, unreported modification of inference software, and covert computation via unreported batch elements. Empirically, we analyze how modern inference engines (vLLM, HF transformers) produce deterministic but non-invariant outputs, without needing to set performance-compromising determinism flags, if the right information is available for re-computation and no atomic functions are called in the backend. We demonstrate that such bitwise-precise re-computation does not require access to identical hardware, via a software-only emulation of LLM inference across multiple NVIDIA GPU variants. Thus, accumulated rounding errors can be an auditable signature of the software and hardware setup used for inference, instead of a constraint on verifiability.
Most large language models (LLMs) run on external clouds: users send a prompt, pay for inference, and must trust that the remote GPU executes the LLM without any adversarial tampering. We critically ask how to achieve verifiable LLM inference, where a prover (the service) must convince a verifier (the client) that an inference was run correctly without rerunning the LLM. Existing cryptographic works are too slow at the LLM scale, while non-cryptographic ones require a strong verifier GPU. We propose TensorCommitments (TCs), a tensor-native proof-of-inference scheme. TC binds the LLM inference to a commitment, an irreversible tag that breaks under tampering, organized in our multivariate Terkle Trees. For LLaMA2, TC adds only 0.97% prover and 0.12% verifier time over inference while improving robustness to tailored attacks by up to 48% over the best prior work requiring a verifier GPU.
Oguzhan Baser, Elahe Sadeghi, Eric Wang +5
Electrical and Computer Engineering, The University of Texas at Austin · Theseus AI Labs · Electrical and Computer Engineering, McGill University +1