ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models

Jiaqi Chen; Xianglong Yan; Chengzhu Bao; Xie Chen; Yulun Zhang

cs.AIOct 22, 2026

ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models

Authors: Jiaqi Chen, Xianglong Yan, Chengzhu Bao, Xie Chen, Yulun Zhang

Organizations: Shanghai Jiao Tong University

Abstract

Mixture-of-Experts (MoE) large language models reduce computation through sparse activation but still incur substantial storage costs. Mixed-precision post-training quantization (PTQ) reduces these costs by quantizing different model components at different bit widths, but existing methods often struggle to preserve generation quality under tight bit budgets. We identify two key challenges: (1) nonlinear quantization errors at extremely low precision are underestimated when cross-layer propagation is ignored; (2) direct router fine-tuning may cause routing drift, and calibration gains may not generalize to downstream tasks. To address these issues, we propose ProQuant, a weight-only PTQ framework for MoE models. Prune-or-Prioritize Mixed-Precision Allocation (PoP) jointly optimizes expert pruning and bit-width assignment. It allows higher-precision experts to substitute for severely degraded ones while improving bit-budget utilization. Routing-Consistent RMSNorm Finetune (RoCo) adapts expert input representations through mathematically equivalent transformations. It compensates for quantization errors to reduce routing drift. Extensive experiments demonstrate that ProQuant outperforms existing methods across diverse generative tasks. Code is available at https://github.com/ThrowBug/ProQuant.git.

Figures & tables

Figure 1: Performance comparison of MoE mixed-precision quantization methods on Qwen3-30B-A3B-Instruct-2507 under 2.5-bit average expert quantization.

Figure 2: Relative hidden-state MSE at each decoder layer, measured against the corresponding full-precision model outputs. All experts in layer 0 are quantized to 1–4 bits, while those in other layers use 2 bits.

Figure 3: Overview of ProQuant, illustrating the workflows of (a) Prune-or-Prioritize Mixed-Precision Allocation (PoP) and (b) Routing-Consistent RMSNorm Finetune (RoCo).

Figure 4: Top-32 expert Jaccard similarity across MoE layers in Qwen3-30B-A3B-Instruct-2507, comparing C4 with MATH-500 and GPQA-Diamond using activation-count and mean-routing-score rankings.

Figure 5: Architecture of RoCo. (a) Fine-tuning process and (b) parameter folding process. Superscripts $i$ and $p$ denote “in” and “post”, respectively.

Figure 6: Layer-wise Jaccard similarity between the full-precision and quantized models’ top-32 experts ranked by mean routing score on three datasets.

Model	Bits	Method	GSM8K	MATH-500	GPQA-D	IFEval	MMLU-R	BFCL-v3	AVG
Qwen3-30B-A3B- Instruct-2507	16.0	BF16	96.36	95.40	55.05	87.66	88.18	63.02	80.94
	2.5	MxMoE	93.86	51.80	36.36	81.17	81.26	30.01	62.41
		PMQ	95.45	39.80	30.81	82.33	79.56	29.82	59.63
		GEMQ	93.93	45.60	31.31	82.66	81.56	30.22	60.88
		ProQuant	95.53	93.20	53.03	87.19	85.44	56.31	78.45
	2.0	GPTQ	91.28	76.00	36.36	81.17	81.75	45.29	68.64

Table 1: Performance of quantized Qwen3-30B-A3B-Instruct-2507 and Qwen3.5-35B-A3B on six generation-based benchmarks. We compare ProQuant with representative mixed-precision methods for MoE LLMs at varying average expert bit-widths. GPQA-D and MMLU-R denote GPQA-Diamond and MMLU-Redux, respectively.

Model	Bits	Method	Wiki. PPL $\downarrow$	C4 PPL $\downarrow$	PIQA $\uparrow$	ARC-E $\uparrow$	ARC-C $\uparrow$	HS $\uparrow$	WING $\uparrow$	AVG $\uparrow$
Qwen3-30B-A3B-Instruct-2507	16.0	BF16	7.26	12.49	80.09	83.88	62.37	79.72	73.40	75.89
	2.0	GPTQ	9.83	15.45	78.29	80.18	56.83	73.70	71.74	72.15
		MxMoE	9.23	13.97	79.54	79.21	55.20	77.84	72.30	72.82
		PMQ	10.61	14.84	79.71	78.87	54.69	77.73	72.93	72.79
		GEMQ	8.22	13.49	79.65	77.99	56.31	76.97	72.61	72.79
		ProQuant	8.94	14.57	79.22	79.08	57.17	75.19	70.64	72.26

Table 2: Perplexity and 0-shot log-likelihood accuracy on Qwen3-30B-A3B-Instruct-2507. Wiki., ARC-E, ARC-C, HS, and WING denote WikiText-2, ARC-Easy, ARC-Challenge, HellaSwag, and WinoGrande, respectively.

Candidate Bits	GSM8K	MATH-500	GPQA-D	IFEval	MMLU-R	BFCL-v3	AVG
$\{0,2,3\}$	94.47	90.20	45.96	84.47	79.82	52.05	74.49
$\{1,2,3\}$	93.33	85.80	36.87	80.96	71.40	42.40	68.46
$\{0,1,2,3\}$	93.93	85.40	36.36	77.88	69.44	38.87	66.98

Table 3: Ablation of candidate bit-width sets on Qwen3-30B-A3B-Instruct-2507 at a 2.0-bit average expert precision.

Method	GSM8K	MATH-500	GPQA-D	IFEval	MMLU-R	BFCL-v3	AVG
ProQuant	93.93	86.00	32.83	77.07	67.26	46.69	67.30
w/o RoCo	94.16	85.80	26.77	75.35	63.75	42.79	64.77
Direct RMSNorm FT	93.33	86.60	30.81	74.96	65.88	43.89	65.91

Table 4: Fine-tuning strategy ablation on Qwen3-30B-A3B-Instruct-2507 at a 1.5-bit average expert precision.

Model	Bits	$\boldsymbol{\rho}$	GSM8K	MATH-500	GPQA-D	IFEval	MMLU-R	BFCL-v3	AVG
Qwen3-30B-A3B- Instruct-2507	2.0	0.10	95.00	88.00	48.99	82.10	82.56	49.71	74.39
		0.15	94.69	88.20	42.93	84.21	81.30	52.66	74.00
		0.20	95.22	87.80	44.44	84.92	80.12	51.67	74.03
		0.25	95.45	91.80	46.46	82.57	79.49	54.38	75.03
		0.30	94.24	90.60	45.45	83.97	79.00	51.04	74.05

Table 5: Sensitivity to

\rho

on Qwen3-30B-A3B-Instruct-2507 with 2.0-bit experts, without RoCo finetuning.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 7: (a) Model size comparison of Qwen3-30B-A3B-Instruct-2507 and Qwen3.5-35B-A3B under different expert average bit-width settings. (b) End-to-end quantization time comparison of different methods on Qwen3-30B-A3B-Instruct-2507 under a 2.0-bit expert average bit-width setting.

Mixture-of-Experts Quantization

Mixture-of-Experts quantization compresses expert weights and computations to reduce memory use and speed up inference while preserving model quality.

See all 32 papers

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
Apr 7, 2026Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4Mixture-of-Experts Quantization Neural Network Quantization
Rensselaer Polytechnic Institute · IBM Research
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
May 21, 2026Jianing Deng, Song Wang, Dongwei Wang +4Mixed-Precision Quantization LLM Quantization
University of Pittsburgh · University of Central Florida · University of Arizona +1
AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Jun 3, 2026Wanqi Yang, Yuexiao Ma, Alexander Conzelmann +4Mixed-Precision Quantization Mixture-of-Experts Language Models
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center +5

Apr 7, 2026J/K move · Enter open · S save

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees

Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai, Naigang Wang+3

Rensselaer Polytechnic Institute · IBM Research

Mixture-of-Experts Quantization Neural Network Quantization Post-Training Quantization Sparse Mixture-of-Experts

Open paper AskPDF arXiv

Sparse Mixture-of-Experts (MoE) allows scaling of language and vision models efficiently by activating only a small subset of experts per input. While this reduces computation, the large number of parameters still incurs substantial memory overhead during inference. Post-training quantization has been explored to address this issue. Because uniform quantization suffers from significant accuracy loss at low bit-widths, mixed-precision methods have been recently explored; however, they often require substantial computation for bit-width allocation and overlook the varying sensitivity of model performance to the quantization of different experts. We propose a theoretically grounded expert-wise mixed precision strategy that assigns bit-width to each expert primarily based on their change in routers l2 norm during training. Experts with smaller changes are shown to capture less frequent but critical features, and model performance is more sensitive to the quantization of these experts, thus requiring higher precision. Furthermore, to avoid allocating experts to lower precision that inject high quantization noise, experts with large maximum intra-neuron variance are also allocated higher precision. Experiments on large-scale MoE models, including Switch Transformer and Mixtral, show that our method achieves higher accuracy than existing approaches, while also reducing inference cost and incurring only negligible overhead for bit-width assignment.

Abstract

Mixture-of-Experts Quantization

Explore similar work

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs

AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization