Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders

Hyojung Han; Jongmin Kim; Seung-Hun Jeon

cs.IROct 6, 2026

Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders

Authors: Hyojung Han, Jongmin Kim, Seung-Hun Jeon

Organizations: ThakiCloud

Abstract

Mixed-precision post-training quantization needs a per-module sensitivity signal; for a text embedder the obvious one -- the retrieval quality a module costs when quantized -- needs relevance labels that deployments rarely have. We measure a label-free substitute: quantization-induced representation drift, obtained by quantizing one module, re-encoding the corpus, and recording how far the output embeddings moved from their full-precision positions. What is specific is the observable: the deployed output representation a dense retriever ranks with. Across five development embedders, configuration-level drift orders sampled mixed-precision plans against held-out retrieval quality at a macro Spearman of 0.911, the sensitivity transports across calibration corpora and retrieval domains in the usable regime, module drifts compose rank-consistently but not numerically, and relevance-derived sensitivity adds no consistent value. The method is one additive allocation under a hard packed-byte budget, with no labels and no search. On three embedders held untouched until method, baselines and hypotheses were frozen and sealed, the pre-registered directional hypothesis against the prior LieQ criterion holds (3/3 at the main budget, no collapse) and drift scores above a two-sided LieQ steelman in 2/3; but at the main budget drift is numerically lower than same-budget uniform precision on all three (-0.99, -0.85, -1.01 points), having reduced module and whole-model drift as designed. Output drift is thus a robust coarse sensitivity signal, not a universally optimal allocation objective: it avoids the catastrophic failures of the transferred signed-geometry adaptation and can remain usable at stressed budgets where uniform collapses, but fine-grained redistribution around a strong uniform operating point remains unresolved.

Figures & tables

Model	Group	Pool	$B_{2\leftrightarrow 3}$	$B_{3}$	Pred. pool budget
			Realized packed bpw
E5-base	64	174	3.117	3.612	3.612 ( $=S_{3}$ )
BGE-M3	32	154	3.708	4.200	4.200 ( $=S_{3}$ )
EmbGemma	128	174	2.750	3.250	3.750
Qwen3-0.6B	32	174	3.500	4.000	4.000 ( $=S_{3}$ )
SKILLRET	32	174	3.500	4.000	4.000 ( $=S_{3}$ )

Table 1: Per-model configuration. “Group” is the quantization group size, “Pool” the realized number of distinct feasible configurations used for the predictability analysis after deduplication, and the two budget columns give the actual packed bits per weight that each budget name resolves to on that model. The budget names are not bit widths:

B_{2\leftrightarrow 3}

resolves to a different packed cost on every model.

Model	Budget on grid	$\rho$	$\tau_{b}$	Top-10% recall
E5-base	yes	0.892	0.720	0.41
BGE-M3	yes	0.973	0.865	0.73
EmbGemma	no	0.891	0.738	0.65
Qwen3-0.6B	yes	0.916	0.769	0.53
SKILLRET	yes	0.884	0.732	0.59
Macro ( $n=5$ )	—	0.911	0.765	0.58

Table 2: Spearman

\rho

and Kendall

\tau_{b}

between whole-model embedding drift and held-out NDCG@10 over a pool of up to 176 requested mixed-precision configurations per model (154–174 distinct feasible plans after deduplication), and the recall of the true top decile by the drift ranking. “Budget on grid” marks cells whose frozen budget coincides with that model’s uniform 3-bit packed size, which constrains the feasible set.

Figure 1: Spearman correlation between whole-model drift and held-out NDCG@10, restricted to progressively better configurations. “top 25%” means the quarter of the pool with the highest true NDCG. The correlation over the full pool is carried in part by collapsed configurations; among good configurations it is substantially weaker. The top-decile column has

n\approx 16

per model and is shown for shape only.

Model	FP32	Uniform	LieQ-signed	LieQ-abs	InfoQ-X	Drift (ours)
E5-base	34.58	31.24	24.40	31.24 ≡U	rejected ‡	32.29
BGE-M3	31.50	27.67	0.96 §	21.38	rejected ‡	30.15
EmbGemma	38.52	36.13	36.13	36.13 ≡U	rejected ‡	37.12
Qwen3-0.6B	34.97	31.00	1.31 §	27.05	rejected ‡	31.31
SKILLRET	33.74	29.01	1.20 §	29.01 ≡U	rejected ‡	30.62

Table 3: Development comparison (the five models the method was designed against; the three untouched confirmation models are reported separately in § 4.3 , and no aggregate win count against LieQ is given over these five). Held-out NDCG@10 at

B_{3}

, all arms under the same packed-byte budget. § marks a cliff-crossing cell, where one allocation falls below the usability threshold, so the difference is not a quality margin. LieQ-signed is the literal port of the prior directional criterion; LieQ-abs replaces its score with

|\Delta r|

and is a two-sided steelman fixed in advance of computing it, after a systematic sign reversal was observed on the development models. ≡U marks a LieQ-abs allocation that is byte-identical to uniform precision — in those cells the unsigned criterion does not discriminate between modules, and the drift margin should be read as a margin over uniform rather than over LieQ. ‡ InfoQ-X was rejected by a pre-specified estimator-stability gate before any retrieval score was computed — an empty cell here is a resolved negative result, not a missing measurement. LieQ-adapt uses LieQ’s own diagnostic [ 33 ] — the training-induced concentration of a projection’s output spectrum, measured against an untrained weight matrix of matched statistics — as the sensitivity score, with the quantizer, bit set, group size, packing overhead, budget, allocator and calibration inputs held identical to ours. It is an adaptation rather than a reproduction: LieQ allocates per layer, we allocate over twelve units. Fifteen arms are run, and the one reported here is fixed a priori as closest to the method as described, not chosen on the held-out score; the arm and the selection rule are given in Table 4 , and the post-hoc best of the fifteen is reported separately as a LieQ-family upper bound in Table 17 . InfoQ-X is reported only if it satisfies the estimator-stability criterion fixed in advance (Appendix B ); InfoQ-X did not satisfy the pre-specified estimator-stability gate, so no retrieval score is reported for it; see Appendix B . Our own static-geometry operationalization is not a published system and is reported in Appendix I (Table 18 ) rather than here.

Model	Arm	Gain (pp)	Seed $\rho$	Agreement $\rho$
E5-base	init/ $4^{-b}$ /lay	-6.842	1.000	-0.762
BGE-M3	init/ $4^{-b}$ /lay	-26.708	1.000	-0.497
EmbGemma	init/ $4^{-b}$ /lay	0.000	0.976	-0.427
Qwen3-0.6B	init/ $4^{-b}$ /lay	-29.697	0.976	-0.524
SKILLRET	init/ $4^{-b}$ /lay	-27.813	1.000	-0.545

Table 4: Which LieQ-adapt arm is reported and how reproducible its score is. The reported arm is fixed in advance, not selected on the held-out score. Fifteen arms are run — three sensitivity variants crossed with five allocation rules, the latter being LieQ’s own top-

m

four-/two-bit split at a matched packed budget plus two bit-penalty forms crossed with two scopes — and the arm in Table 3 is the one chosen a priori as closest to the method as described: an isotropic untrained reference, LieQ’s own per-head split, LieQ’s criterion applied to the layer bands with the vocabulary bands held at uniform (LieQ is layer-wise and has no vocabulary-band notion), and the

4^{-b}

bit penalty that matches uniform-quantization error scaling. Choosing instead the best of the fifteen by held-out NDCG would be a LieQ-family post-hoc upper bound , not a baseline; that quantity is reported separately in Appendix I (Table 17 ) together with the arm that uses LieQ’s own two-level allocation rule, and our method is given no comparable advantage. Arm names are abbreviated: the prefix is the sensitivity variant ( init = isotropic untrained reference, w

d

= every projection split at a fixed width

d

, colmatch = a reference matched to the trained weight’s column statistics), top-

m

is LieQ’s own allocation rule, and

2^{-b}

4^{-b}

with all / lay name the bit-penalty form and whether embedding bands participate. These are display abbreviations for one ledger key and not separate recipes: the reported arm is written init/L2_mse/layers in the ledger and renders here as init /

4^{-b}

/ lay , where L2_mse is the

4^{-b}

penalty — the error of a uniform quantizer scales as

4^{-b}

, so the two names denote the same quantity. Seed

\rho

is the rank correlation of the module scores across draws of the untrained reference matrix, which is the only stochastic element of the diagnostic. Agreement

\rho

is the rank correlation between the two sensitivity criteria over the twelve allocation units; since the allocator, bit set, group size, packing overhead, budget and calibration inputs are identical between the arms, it isolates the disagreement between criteria from everything else.

Figure 2: Full-precision retention of the drift allocation against the actual packed budget. Retention is quantized NDCG@10 divided by full-precision NDCG@10 on the same corpus. The same normalized budget lands in very different regimes on different architectures.

Model	Budget	Group	bpw	FP32	Uniform	LieQ-signed	LieQ-abs	Drift (ours)	Stratum
mxbai-large	$B_{2\leftrightarrow 3}$	32	3.559	38.40	5.30	27.99	27.99	28.84	graded
mxbai-large	$B_{3}$	32	4.056	38.40	37.41	34.83	37.41 ≡U	36.42	graded
MPNet-base	$B_{2\leftrightarrow 3}$	64	3.117	33.57	1.04	7.61	7.61	6.09	dead
MPNet-base	$B_{3}$	64	3.613	33.57	30.52	16.75	16.75	29.67	cliff
GTE-base	$B_{2\leftrightarrow 3}$	64	3.000	36.02	1.42	1.01	0.79	2.44	dead
GTE-base	$B_{3}$	64	3.500	36.02	28.87	1.21	27.51	27.85	cliff

Table 5: Confirmation on the three embedders held untouched until the method, baselines, budgets, thresholds and hypotheses were frozen and sealed. Held-out NDCG@10, all arms under the same packed-byte budget, one run per cell under the frozen protocol. ≡U marks a LieQ-abs allocation byte-identical to uniform. The stratum is the pre-registered pairwise classification by each arm’s own retention against the

0.5\times

full-precision usability threshold: graded (drift and LieQ-signed both usable), cliff (one usable), dead (neither).

Model	Device	$\max\|\Delta\|$ vs ledger	$\hat{D}$ drift / uniform	$D$ drift / uniform	$\Delta$ NDCG drift $-$ uniform [95% CI]
mxbai-large	mps	0.060	0.043 / 0.059	0.048 / 0.058	-1.04 [-1.85, -0.28]
MPNet-base	mps	0.042	0.097 / 0.131	0.129 / 0.143	-0.90 [-1.90, 0.18]
GTE-base	cuda	0.000	0.131 / 0.289	0.145 / 0.326	-1.01 [-2.71, 0.64]

Table 6: Independent re-evaluation of the untouched

B_{3}

cells with the ledger’s own bit plans and held-out protocol on a different machine.

\max|\Delta|

is the largest absolute difference from the ledger over the full-precision, uniform, drift and searched arms.

\hat{D}_{\mathrm{held\text{-}out}}

is the sum of single-module held-out drifts under the plan (the allocator itself minimised the calibration-corpus analogue),

D

the measured whole-model balanced drift. The last column is the query-level paired bootstrap of drift minus uniform NDCG@10 (5,000 resamples, 323 queries; percentile interval).

Model	Probe $\to$ held-out		Four-probe stability
	$\rho$	cells	$\rho$	identical plans	median Hamming	NDCG spread
E5-base	0.992	2	0.993	12/12	0.0	0.00
BGE-M3	0.996	2	0.997	8/12	0.0	0.05
EmbGemma	0.986	2	0.989	7/12	0.0	0.37
Qwen3-0.6B	0.991	2	0.993	9/12	0.0	11.43
SKILLRET	0.994	2	0.996	12/12	0.0	0.00

Table 7: Transport of module sensitivity across retrieval domains. The left block correlates module drift measured on the probe corpus against the same cells measured on the held-out corpus. The right block varies the probe over four corpora and reports the mean pairwise sensitivity correlation, how many probe pairs yield a byte-identical bit plan, the median Hamming distance between plans, and the spread of held-out NDCG across probes. Probe bank and evaluation bank are disjoint.

		$B_{2\leftrightarrow 3}$		$B_{3}$
Model	$\rho$	Identical	Max $\Delta$ NDCG	Identical	Max $\Delta$ NDCG
E5-base	0.992	100.0% †	0.000 †	100.0%	0.000
BGE-M3	0.998	33.3%	0.053	100.0%	0.000
EmbGemma	0.987	100.0%	0.000	16.7%	0.372
Qwen3-0.6B	0.991	50.0% †	11.425 †	100.0%	0.000
SKILLRET	0.994	100.0% †	0.000 †	100.0%	0.000

Table 8: Four-probe domain stability. Module sensitivity is measured independently on SciFact, FiQA, ArguAna and NFCorpus; every resulting plan is evaluated on SciDocs, which appears in no probe.

\rho

is the median pairwise Spearman of the

12\times 3

module-by-bit sensitivities over the six probe pairs. Identical is the fraction of probe pairs selecting a byte-identical bit plan at that budget, and Max

\Delta

NDCG the largest held-out disagreement any pair produces. † marks cells the budget collapses (best probe retains under half of full-precision). Those cells are reported separately rather than dismissed: all of their resulting systems lie in the collapsed regime, so a plan disagreement there is not comparable to one between usable systems, but it is not evidence that the signal is stable either — the Qwen3-0.6B cell shows the signal-to-decision chain becoming more consequential past the cliff. The macro row is computed over the non-collapsed cells and the all-cell macro is given in the text.

Figure 3: (a) Predicted drift

\sum_{i}d_{i,b_{i}}

, the sum of single-module drifts, against the drift measured by quantizing the whole configuration at once; the dashed line is

y=x

. (b) Empirical CDF of the relative residual

|R|/D_{\mathrm{whole}}

for the same configurations. The cloud in (a) sits systematically below

y=x

— the compressive bias quantified in Table 9 — and (b) shows how far the disagreement reaches: a tenth of configurations are off by more than a third. Panel (a) read alone invites “near-additive”; (b) is why the text below claims ordering rather than magnitude. Module drifts and configuration drifts are measured on the same corpus, so the spread here is composition error and not cross-corpus transfer.

	Pairwise inversion rate			Calibration $D=\alpha+\beta\hat{D}$
Cell	All (%)	Top 25%	Seed nbhd.	$\beta$	$R^{2}$	nMAE/IQR
E5-base $B_{2\leftrightarrow 3}$	10.7%	18.5%	13.3%	0.34	0.84	1.02
E5-base $B_{3}$	6.4%	16.7%	12.0%	0.66	0.93	0.23
BGE-M3 $B_{2\leftrightarrow 3}$	12.4%	23.1%	15.5%	0.46	0.77	0.45
BGE-M3 $B_{3}$	12.4%	6.2%	6.7%	0.51	0.78	0.47
EmbGemma $B_{2\leftrightarrow 3}$	5.2%	11.4%	9.9%	0.65	0.90	0.20

Table 9: How module drift composes. The inversion rate is

\Pr[(\hat{D}_{A}-\hat{D}_{B})(D_{A}-D_{B})<0]

over configuration pairs, restricted to three regions: the whole pool, the top quartile by true held-out NDCG, and the budget-preserving neighbourhood of the additive plan — the region a local search actually explores. The calibration columns fit

D=\alpha+\beta\hat{D}

; nMAE/IQR is the mean absolute residual divided by the interquartile range of measured drift, a scale-free magnitude figure that does not inherit the small-denominator inflation of

|R|/D

Figure 4: Budget-matched candidate selection. Each curve reports the best held-out NDCG@10 reached after evaluating

k

configurations, selecting by label-free whole-model drift alone; NDCG is never consulted during selection. “no signal” picks uniformly at random from the same candidates. “additive” is the drift allocation, which costs 36 single-module encodings rather than

k

configuration evaluations. Curves are single trajectories; seed variance is not measured (§ 8 ).

Budget	Model	Uniform	Additive	Rand+drift	Seed+drift	$\Delta_{\mathrm{search}}$	$\Delta_{\mathrm{seed-rand}}$
$B_{2\leftrightarrow 3}$	E5-base	1.63	6.79	4.20	7.18	0.390	2.985
	BGE-M3	0.97	23.75	18.98	25.10	1.346	6.118
	EmbGemma	1.49	33.86	30.20	33.34	-0.524	3.138
	Qwen3-0.6B	0.97	3.38	8.68	7.96	4.581	-0.724
	SKILLRET	1.52	6.15	8.60	14.74	8.582	6.135
$B_{3}$	E5-base	31.24	32.29	26.94	31.18	-1.111	4.242

Table 10: Held-out NDCG@10 of three arms at one evaluation budget of 100 configurations.

\Delta_{\mathrm{search}}

is seeded search minus the additive plan;

\Delta_{\mathrm{seed-rand}}

is seeded search minus random search. Selection within both search arms uses drift only; NDCG is never consulted during selection.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Samples $N$	Projections	Median seed $\rho$	Median Hamming /12	Runtime
1024	64	0.309	7.0	4 min
1024	128	0.390	7.0	9 min
1024	256	0.319	8.0	17 min
1024	512	0.429	8.0	34 min
2048	512	0.491	7.0	71 min
4096	512	0.439	6.0	2.7 h

Appendix

Table 11: Stability of the InfoQ-X adaptation as the projection count and estimator sample count increase, reported as the median over 5 development models. Median seed

\rho

is the seed-pairwise Spearman correlation of the

12\times 3

module-by-bit sensitivity scores; median Hamming is the seed-pairwise distance between the twelve selected bit assignments; exact-plan rate is the fraction of seed pairs whose selected plans are byte-identical. The rank correlation column is the load-bearing one. An earlier version of this paper emphasised the exact-plan rate instead; Appendix F shows that was a misattribution, because a well-behaved signal degraded to this level of rank agreement also fails to reproduce an allocation. The exact-plan column is retained as a description of what the ladder did, not as evidence about the estimator. Runtime is the median wall clock per model per rung on one B200. The convergence criterion and the maximum sampling budget were fixed before these runs.

Model	Budget	Pooled	Balanced	Query-only	Doc-only
E5-base	$B_{2.5}$	27.29	27.29	27.29	27.29
E5-base	$B_{3.0}$	33.99	33.99	33.99	33.99
BGE-M3	$B_{2.5}$	23.75	23.75	23.75	23.75
BGE-M3	$B_{3.0}$	30.15	30.15	30.15	30.15
EmbGemma	$B_{2.5}$	34.89	34.89	34.89	34.89
EmbGemma	$B_{3.0}$	36.99	36.99	36.93 ∗	36.99

Appendix

Table 12: Held-out NDCG@10 of the allocation produced by each pooling of the query and document drift, built from one shared measurement per model. Development models above the rule, untouched models below it. Cells where a variant’s plan differs from the pooled plan are marked ∗ .

Model	Median seed-pair $\rho$	Exact-plan rate
E5-base	0.9997	100.0%
BGE-M3	0.9997	100.0%
EmbGemma	0.9997	100.0%
Qwen3-0.6B	0.9997	100.0%
SKILLRET	0.9997	100.0%

Appendix

Table 13: Drift sensitivity under calibration resampling, at

B_{3}

. Statistics are the same three the stability ladder reports for InfoQ-X, computed over seed pairs.

Model	Drift gain (pp)	Permuted median (pp)	Permuted pct.	Random pct.
E5-base	1.051	-14.776	100.0	100.0
BGE-M3	2.478	-25.826	99.5	100.0
EmbGemma	1.478	0.764	89.5	100.0
Qwen3-0.6B	0.301	-22.372	100.0	100.0
SKILLRET	1.607	-17.491	100.0	100.0

Appendix

Table 14: Signal against allocator structure at

B_{3}

. “Permuted median” is the median gain over uniform of 200 allocations built from randomly reassigned sensitivities; a positive value means non-uniform spending helps on that model regardless of the criterion. Percentile columns place the drift allocation within each null distribution.

Score rank agreement $\rho$	Pairs	Median plan Hamming /12	Exact-plan rate
0.930	16	0.0	75.0%
0.875	16	0.0	56.2%
0.730	16	3.0	6.2%
0.589	16	4.0	6.2%
0.319	16	6.0	0.0%
0.064	16	8.0	0.0%

Appendix

Table 15: Allocator response to score perturbation, measured on the drift scores of 4 development cells at

B_{3}

. Each row perturbs the real scores to a target rank correlation, draws two independent perturbations, and compares the two resulting bit plans — the same quantity the stability ladder reports for InfoQ-X. Exact-plan agreement is a step function of rank agreement, not a property of any particular estimator.

Units	Single-quantize evals	Probe time	Retention	Gain over uniform (pp)	Probe transport $\rho$
6	18	2.3 min	0.970	-0.072	0.990
12	36	4.4 min	0.983	0.397	0.994
24	72	8.6 min	0.980	0.276	0.990

Appendix

Table 16: Allocation-unit granularity on E5-base at

B_{3}

. Probe time is the cost of measuring module sensitivity, which grows linearly in the number of units. Retention is held-out NDCG@10 divided by full precision. Probe transport

\rho

is the rank correlation of the sensitivities measured on two different probe corpora at that granularity. The cross-granularity ordering correlations are in the text; they are computed in a common coarse frame because the unit names differ between settings.

Model	Pre-specified	LieQ-native alloc.	Post-hoc best	Arms
E5-base	24.40	2.02	24.40	15
BGE-M3	0.96	1.81	1.81	15
EmbGemma	36.13	9.79	36.86	15
Qwen3-0.6B	1.31	1.11	1.31	15
SKILLRET	1.20	1.82	1.82	15

Appendix

Table 17: LieQ-adapt variants at

B_{3}

. “Pre-specified” is the arm fixed in advance and reported in Table 3 . “LieQ-native alloc.” applies LieQ’s own two-level top-

m

rule at a matched packed budget; over twelve coarse units a binary four-/two-bit split cannot spend the budget as precisely as the integer program can, which is a consequence of the granularity mismatch rather than of the criterion. “Post-hoc best” selects the highest held-out score among all arms and is therefore an upper bound for the LieQ family , not a competitor — our method is not given the same selection. The gap between the pre-specified column and the post-hoc column measures how much of that family’s headroom is invisible without looking at the test metric.

Model	Uniform	Pre-specified	Post-hoc bound	Variants
E5-base	31.24	31.24	31.24	12
BGE-M3	27.67	26.41	30.14	12
EmbGemma	36.13	30.82	35.31	12
Qwen3-0.6B	31.00	5.14	29.97	12
SKILLRET	29.01	4.96	28.10	12

Appendix

Table 18: Our own static-geometry operationalization at

B_{3}

, retained for reference now that a prior geometry-based baseline (LieQ-adapt, Table 3 ) is available. Layer importance is read from full-precision activation spectra with no quantization intervention and routed through the same allocator. It is not a reproduction of any published system. “Pre-specified” is a single variant fixed in advance and is the honest baseline; “post-hoc bound” is the best of the listed variants selected on the held-out set , which makes it an upper bound for the family rather than a competitor — our method is not given the same advantage. Unlike LieQ, these proxies read the spectrum in absolute terms, with no untrained reference, so they do not measure training-induced concentration.

Model	All	Top 50%	Top 25%	Top 10%	Within $\pm 2$ of uniform
E5-base	0.892	0.588	0.328	0.615	0.698
BGE-M3	0.973	0.861	0.664	0.800	0.762
EmbGemma	0.891	0.826	0.762	0.142	0.822
Qwen3-0.6B	0.916	0.581	0.728	0.022	0.663
SKILLRET	0.884	0.552	0.709	0.792	0.735
Macro	0.911	0.682	0.638	0.474	0.736

Appendix

Table 19: Drift-to-NDCG correlation stratified by configuration quality, with the median sample size per band (§ 4.1 ). A single pooled

\rho

over a pool spanning usable and collapsed models overstates how well drift discriminates in the region an allocator actually operates in.

Cell	Pearson $r$	Spearman $\rho$	Median $\|R\|/D$	P90 $\|R\|/D$
E5-base $B_{2\leftrightarrow 3}$	0.917	0.940	11.2%	40.9%
E5-base $B_{3}$	0.966	0.977	12.4%	27.2%
BGE-M3 $B_{2\leftrightarrow 3}$	0.878	0.921	16.1%	69.1%
BGE-M3 $B_{3}$	0.883	0.908	19.6%	67.7%
EmbGemma $B_{2\leftrightarrow 3}$	0.949	0.983	10.4%	41.3%
EmbGemma $B_{3}$	0.945	0.986	7.5%	25.8%

Appendix

Table 20: Correlation and relative residual of the additive prediction (§ 5 ).

R=D_{\mathrm{whole}}-\sum_{i}d_{i,b_{i}}

is the composition residual and

|R|/D_{\mathrm{whole}}

its relative magnitude, over the 200 evaluated configurations per cell. Read alongside Table 9 , which reports the same disagreement on scale-free axes; the relative figures here inherit a small-denominator inflation that those do not.

Budget	Model	Neighbours	Beating additive	Share
$B_{2\leftrightarrow 3}$	E5-base	100	2	2%
$B_{2\leftrightarrow 3}$	BGE-M3	100	11	11%
$B_{2\leftrightarrow 3}$	EmbGemma	85	0	0%
$B_{2\leftrightarrow 3}$	Qwen3-0.6B	100	21	21%
$B_{2\leftrightarrow 3}$	SKILLRET	100	19	19%
$B_{3}$	E5-base	100	0	0%

Appendix

Table 21: How many budget-preserving neighbours of the additive plan actually beat it, by measured held-out NDCG@10 (§ 6.2 ). Where the count is zero, no selector — however good — can improve on the seed within that neighbourhood.

Model	Budget	Packed bpw	FP32 NDCG	Uniform ret.	Drift ret.	Drift operating regime
E5-base	$B_{2\leftrightarrow 3}$	3.117	34.58	0.047	0.197	collapse
E5-base	$B_{3}$	3.612	34.58	0.904	0.934	useful
BGE-M3	$B_{2\leftrightarrow 3}$	3.708	31.50	0.031	0.754	useful
BGE-M3	$B_{3}$	4.200	31.50	0.878	0.957	useful
EmbGemma	$B_{2\leftrightarrow 3}$	2.750	38.52	0.039	0.879	useful
EmbGemma	$B_{3}$	3.250	38.52	0.938	0.964	useful

Appendix

Table 22: Compression frontier, with the realized packed bits per weight for each budget name (§ 3 , § 4.2 ). The realized cost of a given budget name differs across models, which is why the names must not be read as bit widths. A cell is marked collapse when the drift allocation retains under half of full-precision quality, and saturated when uniform quantization already retains over 95%, leaving little for any allocator to win.

Explore similar work

Sep 14, 2026cs.IR

Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families

Weight-only post-training quantization is the cheapest way to shrink a retrieval embedder, and the received advice for applying it -- protect the embedding table, allocate bits by module sensitivity, prefer a ranking-aware objective over weight reconstruction -- was carried into LLM quantization largely intact. We test that advice on retrieval embedders directly, quantizing five checkpoints from four architecture families across a grid of bit widths and group sizes, and isolating the embedding, attention and feed-forward blocks at each width. Every heuristic fails to transfer as stated. The embedding table never emerges as the dominant isolated protection priority in any family, despite being the largest tensor in several of them. Module sensitivity does not survive as a transferable ordering: at INT4/g16 the spread between modules is too small to allocate against, at INT3 the ordering becomes family-dependent and joint damage stops being the sum of its parts, and at INT2 comparable reconstruction error accompanies retention ranging from 1.3 to 65.9 percent of full precision. A cheap reconstruction proxy is useful for screening uniform bit widths but substantially less reliable for choosing which tensors to protect; its apparent strength across the whole grid is a range-extension artifact. A distilled 109M student at INT3 holds 78.04 NDCG@10 in 68.4 MB and dominates the extreme-PTQ arm of its own 0.6B teacher, 297.9 MB at 64.46, on both size and quality -- but only inside the task it was distilled for. Sizes are byte counts of files that exist rather than arithmetic estimates, and the measurement repository carries the byte provenance for every one of them.

Hyojung Han

ThakiCloud

Post-Training Quantization Large Language Model Quantization Embedder Reconstruction Error Text Analysis and Detection Generation Provenance

Ask PDF

Oct 6, 2026cs.LG

Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training

Quantization-aware training (QAT) leverages lower-precision arithmetic to reduce the cost of LLM deployment, but aggressive quantization degrades final model performance. A common remedy is mixed-precision training, in which high precision is assigned to some of the layers to maintain performance while keeping the cost constrained. This approach then requires precision assignments for model layers during training. We provide a new approach, called Q-PACE, consisting of a second-order sensitivity model that predicts the loss increase as a sum of quantization noise MSE weighted by per-layer curvature coefficients. During training, we periodically re-compute these coefficients using perturbations across layers, and re-assign precision. Pretraining and supervised fine-tuning experiments on LLMs of up to 4B parameters show that Q-PACE consistently improves over existing mixed-precision training recipes, and achieves comparable loss at substantially lower total memory budgets. We further find that quantization sensitivity is highly predictable by depth and layer type, and its stability during training allows for infrequent, cheap recalibration.

Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2

ISTA · EPFL, ISTA

Large Language Model Training

Ask PDF

Jul 25, 2026cs.LG

MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models

Mixed-precision quantization improves the accuracy of post-training quantization by allocating higher bitwidths to sensitive layers, but existing methods solve the allocation for a single fixed memory budget. In practice the budget varies across deployments and is unknown at calibration time. Adaptive quantization addresses this with one offline calibration that serves any budget, yet current methods score layer sensitivity in a manner that does not consider its dependency on quantization levels of other layers. We show that a layer's sensitivity depends strongly on the bitwidths of its upstream layers and that this dependence shifts the resulting preferred bit allocation. We propose MixQuant, a technique-agnostic adaptive framework that wraps any base quantizer. MixQuant marginalizes each layer's distortion over random quantized upstream configurations to obtain budget-agnostic scores, calibrates the quantizer's parameters on plans the allocator itself produces, and penalizes allocations that leave layers at the lowest bitwidths. A single greedy pass then serves any budget at deployment. Across Llama-3.2-3B, Llama-2-7B, and Mistral-7B under AWQ and GPTQ, MixQuant outperforms adaptive and mixed-precision baselines in every setting, improving average accuracy by up to 8 points and reducing perplexity from 12.43 to 10.70 at the tightest budget, while matching an ILP solver at negligible deployment cost.

Ashitabh Misra, Madhav Agrawal, Arham Jain +1

University of Illinois at Urbana-Champaign

Mixed-Precision Quantization Mix

Ask PDF