Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantization

Pranav Varshney

cs.LGAug 12, 2026

Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantization

Authors: Pranav Varshney

Organizations: University of Michigan

Abstract

A statistic reported without the quantity needed to interpret it is not evidence. We develop that thesis for a concrete practice in AI safety. Interpretability artifacts are calibrated on full-precision weights, deployed on quantized ones, and certified as surviving the change by scale-invariant statistics (cosine similarity, correlation, AUROC) that are reported without their noise floor. For the difference-in-means direction estimator, the split-half floor is governed by one dimensionless number, $κ= nρ^2/d$ . The closed form $\mathbb{E}[\cos] \approx (1+4/κ)^{-1}$ is classical; the missing input is the class separation $ρ$ , which we measure on real activations; no compression-transfer study we know of reports it. On Qwen2.5-1.5B-Instruct, $ρ= 33$ -- $61$ across depth, so two independent runs of the estimator agree to $0.978$ -- $0.994$ by sampling alone. A published cosine of $0.996$ between full-precision and quantized refusal directions therefore cannot be read as preservation without the $n$ it was computed at, which is not reported. Where $n$ is known, we judge each low-bit cosine against the split-half null measured within that quantized model, because a full-precision null assumes the low-bit estimator has the same variance. That assumption is exactly what a null exists to test. The result is plain: at INT4 the direction rotated, and the deficit exceeds the estimator's own noise. At INT8 we detect no movement, which is not an equivalence claim. We also show that a scale-invariant statistic cannot distinguish translation from attenuation of a transferred decision variable, although the two call for opposite remedies. We close with reporting recommendations that cost one forward pass. Code, data, and a one-cell reproduction are released at https://github.com/pvarshh/quantinterp

Figures & tables

layer	$\rho$ (naive)	$\rho$	$\rho/\sqrt{d}$	$\kappa$	floor, Eq. ( 1 )	floor, measured	$\Delta$
4	34.47	34.29	0.875	196.0	0.9800	0.9808	0.0008
8	33.72	33.54	0.856	187.5	0.9791	0.9799	0.0008
12	33.05	32.87	0.839	180.1	0.9783	0.9790	0.0008
16	61.43	61.33	1.565	627.0	0.9937	0.9939	0.0002
20	59.12	59.02	1.506	580.6	0.9932	0.9935	0.0003
24	56.22	56.11	1.432	524.8	0.9924	0.9928	0.0004

Table 1: Measured

\rho

on real activations. Qwen/Qwen2.5-1.5B-Instruct,

d=1536

n=256

per class, walledai/AdvBench+tatsu-lab/alpaca. Separation is

33

–

61

\rho

units:

0.84

–

1.56

pooled within-class SDs per dimension , the column comparable across hidden sizes (

\kappa=n\rho^{2}/d=n(\rho/\sqrt{d})^{2}

; Sec. 4 ). An order-unity

\rho

(the natural guess absent any reported measurement) is

0.016

SDs per dimension at

d=4096

54

–

100\times

below the range measured here. The closed form predicts the empirically measured split-half floor to within

0.0008

at every depth. Bold marks the peak, layer 16.

$n$	(A) floor	(A) margin	(B) floor	(B) margin
100	0.9583	$+0.0377$	0.9839	$+0.0121$
256	0.9833	$+0.0127$	0.9937	$+0.0023$
512	0.9916	$+0.0044$	0.9968	$-0.0008$
1024	0.9958	$+0.0002$	0.9984	$-0.0024$

Table 2: Carrying the measured

\rho

d=4096

. “margin” is

0.996-\text{floor}

: positive means the published cosine clears its own noise floor. Peak

\rho=61.33

d=1536

(layer 16, Table 1 ). (A) holds

\rho

constant; (B) holds the per-dimension effect

1.565

constant, giving

\rho=100.16

. Under carry (B) the reported

\cos=0.996

is at or below its own floor for

n\geq 512

, so it is not evidence of preservation there.

	rel. wt. err.	subspace gain	$\cos$ to FP16	own null	sd	FP16 null	floor, Eq. ( 1 )	verdict
INT8	0.0063	1.007	0.9999	0.9827	0.0047	0.9825	0.9820	within noise
INT4	0.1066	0.933	0.9647	0.9803	0.0052	0.9825	0.9820	rotation

Table 3: Rotation, against each bit-width’s own null. At INT4 the cosine falls below the split-half null measured within that model itself by more than twice the null’s own split-to-split spread (

0.9647

vs.

0.9803-2(0.0052)

), so the deficit exceeds that bit-width’s own estimation noise by a margin the noise does not produce, and the rotation is a detection. The verdict is the decision rule

\cos<\text{own null}-2\,\mathrm{sd}

: “own null” is the split-half cosine measured within that quantized model (200 random splits at

n=89

per class), sd its spread over those splits, and “FP16 null” the identical statistic within FP16 on the same partitions (same

n

, same splits, same seed), so the difference between the two null columns (INT8

+0.0002

; INT4

-0.0022

) is the change in the activations and not the draw. It is a conservative decision rule, not a calibrated test; Sec. 5 argues why, and why the conservatism has a known sign. The smallest deficit this rule can call is

2\,\mathrm{sd}\leq 0.0104

in cosine units; a rotation smaller than that is invisible at this

n

, so “within noise” bounds the detectable effect and does not assert absence. The floor of Eq. ( 1 ),

0.9820

, is computed from Table 1 at layer 16 rather than stored beside the cosine, and decides no row; its distance from the FP16 null,

+0.0006

, is Sec. 4 ’s isotropic-Gaussian check repeated at this

n

. Subspace gain is

\overline{|h_{q}\!\cdot\!\hat{v}|}/\overline{|h_{\mathrm{fp}}\!\cdot\!\hat{v}|}

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$n$	$d$	$\rho$	$\kappa$	Eq. ( 1 )	Monte Carlo	$\|$ err $\|$
400	64	3.00	56.25	0.9336	$0.9339\pm 0.0120$	0.0003
400	64	1.00	6.25	0.6098	$0.6086\pm 0.0695$	0.0011
400	64	0.35	0.77	0.1607	$0.1591\pm 0.1221$	0.0015
512	4096	3.00	1.12	0.2195	$0.2203\pm 0.0139$	0.0008
512	4096	10.00	12.50	0.7576	$0.7579\pm 0.0051$	0.0003
512	4096	30.00	112.5	0.9657	$0.9657\pm 0.0007$	0.0000

Appendix

Table 4: Equation ( 1 ) against direct simulation, 300 replicates per row. Settings with matching

\kappa

but very different

(n,d,\rho)

give matching floors, as predicted. The last two rows are this paper’s own operating points (Secs. 4 and 5 ) at the measured peak

\rho

. Maximum absolute error

0.0015

, at

\kappa=0.77

Figure 1: (A) Eleven Monte Carlo settings spanning

\kappa\in[0.77,627]

collapse onto the single analytic curve; the diamonds are this paper’s two operating points at the measured peak

\rho

. (B) At

d=4096

the two carries of Sec. 4 bracket the floor (shaded band); the light curves show the sweep an unmeasured

\rho

leaves open, and the reported

\cos=0.996

is marked. In the shaded right region carry (B) puts the floor above

0.996

, where the reported cosine is not evidence of preservation. Its abscissa

n

is still unreported.

	decision-value fit		AUROC		accuracy
perturbation	slope	$R^{2}$	FP	transfer	transfer	recal.
none	1.000	1.0000	0.8037	0.8037	0.7354	0.7188
translation (small)	1.000	1.0000	0.8037	0.8037	0.7229	0.7188
translation (large)	1.000	1.0000	0.8037	0.8037	0.5500	0.7188
attenuation ( $0.6\times$ )	0.686	0.9472	0.8037	0.7591	0.6750	0.6854
attenuation ( $0.3\times$ )	0.451	0.7168	0.8037	0.6702	0.5792	0.6354

Appendix

Table 5: Known perturbations along the probe direction,

n=1200

d=64

. Translation leaves AUROC exactly unchanged while calibrated-threshold accuracy falls to

0.55

; attenuation moves AUROC by up to

0.13

. The slope is the regression of quantized on full-precision decision values , not the injected subspace gain

g

: the decision variable mixes the perturbed direction with unperturbed ones, so the slope is

1-c(1-g)

with

c

the share of decision-variable variance the perturbed direction carries (

c=0.784

here, consistent across all three gain rows). It therefore understates subspace attenuation (slope

0.686

for an injected

g=0.6

) and detects it rather than measures it; recovering

g

itself requires projecting onto the probe direction before regressing.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Detecting Unreliable Classifications from Quantized Models
Oct 13, 2026Ananya Kotla, Jorge T. GomezPost-Training Quantization Classification
Perplexity Can Miss SAE Feature Damage Under Quantization
Jun 2, 2026Evan DuanLLM Quantization LLM Interpretability
University of Michigan
Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations
Jul 24, 2026Kazi Kamruzzaman Rabbi, Md. Zami Al Zunaed Farabe, M. Sohel RahmanFeature Attribution Neural Network Interpretability
BRAC University, Dhaka, Bangladesh · Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh

Oct 13, 2026J/K move · Enter open · S save

Detecting Unreliable Classifications from Quantized Models

Ananya Kotla, Jorge T. Gomez

Post-Training Quantization Classification

Open paper AskPDF arXiv

Post-training INT8 quantization is typically validated by checking accuracy retention, which requires ground-truth labels often unavailable in low-resource clinical settings. We show a label-free alternative: comparing a model's own output before and after quantization. We compute cosine similarity between the FP32 and INT8 models' output probability vectors for each prediction, which requires only a single forward pass. This output-agreement similarity separates lower-accuracy predictions across four evaluated architectures (ResNet-50, VGG-16-BN, Wide ResNet-50, and ViT-B/16) under fixed-percentile splits, outperforming a confidence-only baseline for all four. These results show that quantized models can be compared to their baselines to assess their drift and determine whether they are well suited for clinical deployment.

Abstract

LLM Interpretability

Explore similar work

Detecting Unreliable Classifications from Quantized Models

Perplexity Can Miss SAE Feature Damage Under Quantization

Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations