Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization

Renjie Xie; Juncheng Yang; Aoting Hu; Yan Liang; Mingxi Zhang; Zheheng Hong; Wei Xu; Yanzhi Chen

cs.LGOct 14, 2026

Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization

Authors: Renjie Xie, Juncheng Yang, Aoting Hu, Yan Liang, Mingxi Zhang, Zheheng Hong, Wei Xu, Yanzhi Chen

Organizations: Nanjing University of Posts and Telecommunications · Tylogi AI Lab / TAIL · Anhui University of Technology · Southeast University ·

Abstract

Mixture-of-experts (MoE) language models activate only a few experts per token, yet all expert weights must be stored. Mixed-precision quantization reduces this storage cost by assigning higher bit-widths to sensitive experts and lower bit-widths to resilient ones, but expert rankings alone do not determine how strongly to weight their errors. To address this gap, we propose NatScale, a contrast-calibrated allocation framework for mixed-precision MoE quantization. It uses a model-specific power transformation to calibrate gradient-based expert weights to an information-theoretic contrast target while preserving rankings. We jointly evaluate each expert's gate, up and down configurations to estimate its output error. We then combine these errors and calibrated weights with an existing prior based on weight spectra, variance and precision decay to select configurations under a storage budget. Across three models, NatScale achieves the lowest native-to-quantized KL divergence and the highest top-token agreement in all nine settings against five allocation baselines. It also yields the lowest perplexity in seven settings. These findings highlight the importance of objective-aware precision allocation.

Figures & tables

Figure 1: Overview of NatScale . (a) Contrast Calibration: Maps gradient scores

g_{i}

via a power transformation

w_{i}=g_{i}^{\beta_{\tau}}

, where the exponent

\beta_{\tau}

is calibrated using a symmetric KL divergence target

\tau

to control expert storage competition. (b) Joint Error Estimation: Evaluates quantization distortion by jointly quantizing gate, up, and down projections over replay inputs while incorporating structural priors. (c) Budgeted Allocation: Synthesizes calibrated weights, structural priors, and joint errors into expert costs, solving the global storage allocation by linear-programming (LP) relaxation and feasible rounding under a storage budget

B

Qwen3.6-35B-A3B	2.0 routed bpw			2.5 routed bpw			3.0 routed bpw
Method	PPL $\downarrow$	KLD $\downarrow$	Top1 $\uparrow$	PPL $\downarrow$	KLD $\downarrow$	Top1 $\uparrow$	PPL $\downarrow$	KLD $\downarrow$	Top1 $\uparrow$
MxMoE ( Duanmu et al., 2025 )	8.0690	0.2727	79.87	7.2930	0.1489	85.30	7.0630	0.0870	88.93
FRI-MxMoE ( Qi et al., 2026 )	8.4293	0.3118	78.14	7.9069	0.2359	80.90	7.6321	0.1992	82.55
AlphaQ ( Yang et al., 2026 )	7.7790	0.2047	81.76	7.3119	0.1236	85.70	6.9932	0.0671	89.49
MODE ( Chen et al., 2026 )	8.4724	0.3291	78.30	7.6407	0.1901	83.68	7.2792	0.1196	87.21
GEMQ (alloc.) ( Deng et al., 2026 )	7.9142	0.2189	81.89	7.3333	0.1160	87.01	7.1582	0.0762	89.85

Table 1: WikiText-2 results at three nominal routed-weight budgets. Lower perplexity (PPL) and native-to-quantized KL divergence (KLD) are better; higher native top-token agreement (Top1, %) is better. Bold indicates the best result in each column. All methods use the same candidate quantized weights, but retain method-specific allocation constraints and payload/header accounting (Appendix D ). NatScale uses the fixed contrast target

\tau=1

nat.

Figure 2: Sensitivity to the contrast target

\tau

. Changes in KLD (%) and Top1 (percentage points) are measured relative to NatScale at

\tau=1

for each model and nominal routed-bpw budget. Solid curves show NatScale allocations at different targets; same-color horizontal dashed lines show the corresponding AlphaQ reference. The vertical dotted line marks the fixed default.

	UD reference tier (NatScale byte ceiling)
Method	IQ1_M			Q2_K_XL			IQ3_XXS
	Size (GB)	KLD $\downarrow$	Top1 $\uparrow$	Size (GB)	KLD $\downarrow$	Top1 $\uparrow$	Size (GB)	KLD $\downarrow$	Top1 $\uparrow$
UD (reported)	74.5	0.314739	79.691	78.9	0.224607	82.715	82.0	0.165120	85.414
GSQ-RCO †	75.840	0.221337	82.998	—	—	—	83.618	0.137169	86.898
NatScale (ours)	74.529	0.196894	84.721	78.859	0.146297	86.543	81.952	0.124359	87.623

Table 2: Whole-model Qwen3.8-Flash-Next comparison by Unsloth Dynamic (UD) reference tier. NatScale uses each tier’s UD byte ceiling. GSQ-RCO release sizes differ ( † ). Total size includes fixed tensors and metadata, with GB defined as

10^{9}

bytes. Top1 is native-model agreement (%). UD scores are externally reported, while NatScale and GSQ-RCO are evaluated locally using the same WikiText-2 stream and native reference. Bold indicates the lowest KLD and highest Top1 within each tier. Additional details appear in Appendix H .

Method	AIME	GPQA	TQA	LCB
MxMoE	63.33	73.23	93.54	57.14
FRI-MxMoE	50.00	67.68	92.03	51.43
AlphaQ	62.50	73.74	93.04	58.86
MODE	60.83	74.75	93.42	62.86
GEMQ (alloc.)	15.83	63.13	93.80	47.43
NatScale (ours)	68.33	78.28	94.05	61.14

Table 3: Downstream performance on Qwen3.8-Flash-Next at 2.5 routed bpw (%). Thinking is disabled. AIME is mean response accuracy on AIME 2025; GPQA is GPQA Diamond; TQA is TruthfulQA binary accuracy; and LCB is LiveCodeBench v6 pass@1. GEMQ denotes the allocation-only adaptation. Bold indicates the highest point estimate per task.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Expert precision and layer budgets for Qwen3.6-35B-A3B at nominal 2.5 routed bpw. Panels (a)–(e) show actual expert storage; (f) gives its cumulative distribution; (g) shows routed storage by layer. The five methods use the same order, storage definition and color scale across the three model figures.

Figure 4: Expert precision and layer budgets for Qwen3.8-Flash-Next at nominal 2.5 routed bpw. Heatmaps, cumulative storage distributions and layer curves follow the definitions and shared scales in Figure 3 . NatScale uses the joint-error default.

Figure 5: Expert precision and layer budgets for DeepSeek-V4-Flash-0731 at nominal 2.5 routed bpw. All 43 layers and 256 routed experts per layer are shown in their original order. Heatmaps, cumulative distributions and layer curves use the same definitions and scales as the two Qwen figures.

Model	Target bpw	$\tau$ (nat)	KLD $\downarrow$	Top1 (%) $\uparrow$	PPL $\downarrow$
Qwen3.6- 35B-A3B	2.0	0.5	0.202861	82.8668	7.703205
		0.7071	0.201441	82.9946	7.682874
		1	0.198816	82.9824	7.676734
		1.4142	0.196132	83.1650	7.676333
		2	0.195436	83.0852	7.689436
	2.5	0.5	0.100756	87.7958	7.125175

Appendix

Table 4: All 45 threshold settings, including the nine defaults. Top1 is native-reference agreement (%). Bold marks each metric’s best value among the five contrasts within each model and budget.

bpw	Variant	KLD $\downarrow$	Top1 (%) $\uparrow$	PPL $\downarrow$
2.0	NatScale w/o local RMS error	0.243853	81.1204	8.156114
2.0	NatScale w/o bit decay	0.310342	79.1062	8.450841
2.0	NatScale w/o gradient weights	0.222994	81.9155	7.854326
2.0	NatScale w/o gradient smoothing	0.245525	81.0257	8.210989
2.0	NatScale w/o exposure magnitude	0.194500	82.4719	7.719427
2.0	NatScale (ours, $\tau=1$ )	0.198816	82.9824	7.676734

Appendix

Table 5: NatScale component variants for Qwen3.6-35B-A3B under fixed candidate weights and payload ceilings. Top1 is native agreement (%). Bold marks the best listed variant within each budget and metric; the joint-error default appears last.

bpw	Variant	KLD $\downarrow$	Top1 (%) $\uparrow$	PPL $\downarrow$
2.0	NatScale w/o local RMS error	0.378134	78.5598	5.302109
2.0	NatScale w/o bit decay	0.456162	76.5707	5.582535
2.0	NatScale w/o gradient weights	0.337510	79.8438	5.154556
2.0	NatScale w/o gradient smoothing	0.492273	75.1075	5.816586
2.0	NatScale w/o exposure magnitude	0.330178	80.1420	5.035539
2.0	NatScale w/ MSE, w/o bit decay	0.389347	78.4050	5.372212

Appendix

Table 6: NatScale component variants for Qwen3.8-Flash-Next, including error-power and spectral/variance controls. Bold marks the best variant within each budget and metric.

bpw	Variant	KLD $\downarrow$	Top1 (%) $\uparrow$	PPL $\downarrow$
2.0	NatScale w/o local RMS error	0.655823	73.9828	7.726484
2.0	NatScale w/o bit decay	1.102132	65.5826	11.946395
2.0	NatScale w/o gradient weights	0.912658	69.1134	10.175868
2.0	NatScale w/o gradient smoothing	0.609329	74.8965	7.378263
2.0	NatScale w/o exposure magnitude	0.787991	71.1932	8.884293
2.0	NatScale (ours, $\tau=1$ )	0.729311	72.1534	8.364731

Appendix

Table 7: NatScale component variants for DeepSeek-V4-Flash-0731. Bold marks the best variant within each budget and metric.

Setting	Joint error			Separate error
	KLD $\downarrow$	Top1 (%) $\uparrow$	PPL $\downarrow$	KLD $\downarrow$	Top1 (%) $\uparrow$	PPL $\downarrow$
Qwen3.8-Flash-Next
2.0 bpw	0.329276	80.2170	5.067797	0.326225	80.2569	5.073339
2.5 bpw	0.194086	84.8573	4.834395	0.195639	84.9175	4.831680
3.0 bpw	0.124340	87.6714	4.743009	0.123854	87.6423	4.741410

Appendix

Table 8: Joint versus separate projection errors on Qwen3.8-Flash-Next at the NatScale default. Both variants use raw RMS and

\tau=1

under each setting’s fixed payload ceiling. Top1 is native agreement (%). Bold marks the better of the two scores within each row and metric, using unrounded values.

Error score	AIME	GPQA	TQA	LCB
Qwen3.8-Flash-Next
NatScale w/ separate errors	65.00 (78/120)	74.75 (148/198)	93.42 (738/790)	62.86 (110/175)
NatScale (joint, ours)	68.33 (82/120)	78.28 (155/198)	94.05 (743/790)	61.14 (107/175)

Appendix

Table 9: Downstream comparison of joint and separate-error NatScale on Qwen3.8-Flash-Next at nominal 2.5 routed bpw, with thinking disabled. Cells show percentages and correct counts over the full denominator. AIME is mean response accuracy on AIME 2025; GPQA is Diamond; TQA is TruthfulQA binary; LCB is LiveCodeBench v6 pass@1. Bold marks the higher point estimate on each task.

Mixed-Precision Quantization

Mixed-precision quantization assigns different numeric precisions across a model to balance accuracy with memory, speed, energy, and hardware constraints.

See all 168 papers

AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Jun 3, 2026Wanqi Yang, Yuexiao Ma, Alexander Conzelmann +4Mixed-Precision Quantization Mixture-of-Experts Language Models
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center +5
Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
Apr 7, 2026Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4Mixture-of-Experts Quantization Neural Network Quantization
Rensselaer Polytechnic Institute · IBM Research
ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models
Oct 22, 2026Jiaqi Chen, Xianglong Yan, Chengzhu Bao +2Mixture-of-Experts Quantization Post-Training Quantization
Shanghai Jiao Tong University

Jun 3, 2026J/K move · Enter open · S save

AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization

Wanqi Yang, Yuexiao Ma, Alexander Conzelmann, Xiawu Zheng+3

Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center+5

Mixed-Precision Quantization Mixture-of-Experts Language Models Mixture-of-Experts Quantization Neural Network Quantization

Open paper AskPDF arXiv

Mixture-of-Experts (MoE) architectures scale model capacity through sparse expert activation, but their deployment remains memory-bound because all expert weights must reside in memory. Mixed-precision quantization can substantially reduce this footprint by assigning different bit-widths to different experts. Existing approaches, however, typically rely on calibration data to estimate expert importance and determine bit allocation. For frontier MoE LLMs, the original training data, and hence the true training distribution, is proprietary and inaccessible. As a result, calibration sets are inevitably imperfect surrogates, and this can misestimate expert utilization and lead to suboptimal bit allocation. Motivated by the substantial cross-expert quality variability observed in modern MoE models, and by the success of Heavy-Tailed Self-Regularization (HT-SR) theory at predicting neural network model quality without access to training or testing data, we propose AlphaQ, a calibration-free bit-allocation method for MoE quantization. AlphaQ draws on HT-SR theory and follows a simple principle: experts with more heavy-tailed weight spectra are typically better trained and hence should receive higher bit-widths, while experts with weaker heavy-tailed structure can be quantized more aggressively. AlphaQ operationalizes this principle by measuring expert-wise spectral heavy-tailedness and solving a budget-constrained optimization problem that minimizes total quantization error under a global bit-budget constraint. Across several MoE models, AlphaQ consistently outperforms calibration-based baselines under matched bit budgets. Notably, on Qwen1.5-MoE, AlphaQ achieves near full-precision accuracy with an average expert precision of only 3.5 bits, while delivering more than 4

\times

memory compression. Our code is available at https://github.com/Superone77/AlphaQ.

Abstract

Mixed-Precision Quantization

Explore similar work

AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees

ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models