Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors

Yifan Liu

cs.LGOct 14, 2026

Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors

Authors: Yifan Liu

Organizations: UIUC MCS Student

Abstract

We investigate whether per-layer, per-task-phase (prompt/reasoning/response) expert-routing statistics in a Mixture-of-Experts (MoE) language model can guide layer-wise mixed-precision quantization of routed-expert FFN tensors (down/gate and up). To attain layerwise routing statistics, we build a routing-capture and profiling pipeline for Qwen-3.6-35B-A3B in ik_llama.cpp, derive a composite sensitivity score from routing entropy/decisiveness, coefficient of variation (CV), Jaccard/phase-fragility, and top-1 expert strength, and use coordinate descent to tune a layer allocation that respects a fixed average bits-per-weight budget. Notably, the profiling pipeline is applicable to standard top-k MoE architectures. Across independent experimental conditions (about 4.0 bpw flat, about 2.7 bpw steep, and about 2.7 bpw random allocation as a control), reallocation fails to outperform the uniform baseline within each quality tier. In the flat regime, reallocation is statistically neutral (z about 0.9). For the steep tier, the random reallocation performs within 1 SE of the routing-guided reallocation at the same tier, indicating a structural loss at this quality rung caused by mandatory downgrades and Kullback-Leibler Divergence (KLD) tail injection. This aggressive-budget condition shows a decisive loss (z greater than 4 SE) replicated across three independent weight vectors. We argue that the deeper issue is that routing-behavior statistics are an unvalidated proxy for true per-layer quantization cost.

Figures & tables

segment	$n_{\text{positions}}$	med. top-1 frac.	med. $H_{\text{norm}}$	expert coverage
prompt	8,508	0.0558	0.863	0.89–1.00
reasoning	128,287	0.0322	0.907	0.996–1.00
response	26,721	0.0366	0.912	0.996–1.00

Table 1: Per-phase routing statistics over 30 coding prompts (39 profiled layers each). Prompt-phase routing is markedly less diffuse (lower

H_{\text{norm}}

) and leaves some experts never selected (coverage down to 0.89), unlike the reasoning/response phases.

Figure 1: Per-expert routing behavior across the three task phases. Top row shows aggregate expert selection counts per phase. Middle row shows per-layer expert-usage heatmaps, one row per profiled layer. Bottom row shows KDEs of top-1 gate probability. Data spans 30 coding prompts, 39 profiled layers, and 256 routed experts. Prompt-phase routing is visibly less diffuse than the reasoning and response phases.

Regime	Ladder (rungs)	Down base	GU base	Combined bpw/layer
Flat ( $\sim$ 4 bpw)	iq3_ks $\cdot$ iq3_k $\cdot$ iq4_kss $\cdot$ iq4_ks $\cdot$ iq4_k	iq4_ks	iq3_k	11.13
Steep ( $\sim$ 2.7 bpw)	iq2_ks $\cdot$ iq2_k $\cdot$ iq2_kl $\cdot$ iq3_ks $\cdot$ iq3_k	iq3_ks	iq2_kl	8.57

Table 2: Quantization regimes and corresponding ladder configurations. gu = the bundled gate/up tier.

Allocation	KLD ( $\pm$ SE)	Top-1 (%)
Uniform (baseline)	0.014725 $\pm$ 0.000430	97.5
C1	0.015276 $\pm$ 0.000400	97.4
C2	0.015327 $\pm$ 0.000405	97.3
C3	0.015362 $\pm$ 0.000407	97.3
C4	0.015363 $\pm$ 0.000403	97.4
C5	0.015507 $\pm$ 0.000423	97.3

Table 3: Flat regime (

\sim

4 bpw, compressed ladder). Uniform baseline vs. the 8 coordinate-descent allocations C1–C8 ordered by KLD, on the full 331K-token corpus. No allocation beats the uniform baseline. The best candidate is statistically neutral. PPL is a wash across all candidates at

\sim

2.00 and is omitted.

Allocation	KLD ( $\pm$ SE)	KLD p99	Top-1 (%)
Uniform U (baseline)	0.034321 $\pm$ 0.000608	0.437	95.8
cv 0.5	0.038249 $\pm$ 0.000659	0.490	95.6
cv 1.5	0.038142 $\pm$ 0.000665	0.487	95.6
cv 3.0	0.038223 $\pm$ 0.000662	0.492	95.6

Table 4: Steep regime (

\sim

2.7 bpw, low-bit, IQ2 floor tier). Uniform baseline U vs. the 3 score-guided allocations on the full corpus. All candidates lose decisively. The KLD tail (p99) rises on every candidate.

Allocation	KLD ( $\pm$ SE)	KLD p99	Top-1 (%)
Uniform U (baseline)	0.034321 $\pm$ 0.000608	0.437	95.8
random, seed 42	0.037728 $\pm$ 0.000642	0.483	95.7
random, seed 1337	0.037808 $\pm$ 0.000617	0.495	95.6
ref: score-guided (cv 0.5–3.0)	0.03814–0.03825	0.487–0.492	95.6

Table 5: Random-selection control (

\sim

2.7 bpw). Depth, tiers, and budget are matched to the score-guided candidates. Only layer selection is randomized across 2 seeds. Random lands within

\sim

1 SE of the score-guided candidates, showing the composite score carries no selection signal over random.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	bpw/tensor	Used in
IQ2_KS	2.1875	steep ladder (floor)
IQ2_K	2.375	steep ladder
IQ2_KL	2.69	steep ladder (gu base)
IQ3_KS	3.19	both ladders (steep down base)
IQ3_K	3.44	both ladders (flat gu base)
IQ4_KSS	4.0	flat ladder

Appendix

Table 6: Per-tensor bits-per-weight of the IQK quantization tiers used in this work, as reported by ik_llama.cpp. All other tensors (attention, SSM, shared experts, embeddings, MTP head) are held at fixed tiers (Q8_0 or IQ6_K) outside the reallocation budget.

Round	Axis	Winning multiplier	KLD	Changed
1	$w_{c}$ (CV)	$\times 1.5$	0.015276	yes
1	$w_{p}$ (phase)	$\times 0.5$	0.015363	yes
1	$w_{t}$ (top-1)	$\times 1.5$	0.015327	yes
2	$w_{c}$ (CV)	$\times 0.75$	0.015276	yes
2	$w_{p}$ (phase)	$\times 1.0$	0.015276	no
2	$w_{t}$ (top-1)	$\times 0.75$	0.015276	yes

Appendix

Table 7: Flat-regime coordinate-descent trajectory across 3 free axes and 2 rounds. The objective is full-corpus KLD. The KLD column reports the candidate evaluated at each step. A step whose candidate is worse than the current best (e.g., round-1

w_{p}

) is recorded but not accepted. Thus, the accepted KLD never regresses.

Field	Description
kld	Mean KLD (nats/token), full corpus
kld_se	Standard error of KLD (across chunks)
kld_p99	99th-percentile per-position KLD
kld_p95	95th-percentile per-position KLD
kld_median	Median per-position KLD
kld_max	Maximum per-position KLD

Appendix

Table 8: Dedupe cache value schema. Fields present in the low-bit cache ( results_lowbit.json ). The flat-regime cache ( results_full.json ) omits the KLD-tail fields.

Quantization Sensitivity Analysis

See all 20 papers

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
May 21, 2026Jianing Deng, Song Wang, Dongwei Wang +4Mixed-Precision Quantization LLM Quantization
University of Pittsburgh · University of Central Florida · University of Arizona +1
Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
Apr 7, 2026Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4Mixture-of-Experts Quantization Neural Network Quantization
Rensselaer Polytechnic Institute · IBM Research
AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Jun 3, 2026Wanqi Yang, Yuexiao Ma, Alexander Conzelmann +4Mixed-Precision Quantization Mixture-of-Experts Language Models
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center +5

May 21, 2026J/K move · Enter open · S save

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs

Jianing Deng, Song Wang, Dongwei Wang, Zijie Liu+3

University of Pittsburgh · University of Central Florida · University of Arizona+1

Mixed-Precision Quantization LLM Quantization Mixture-of-Experts Language Models LLM Inference Acceleration Mixture-of-Experts Quantization Post-Training Quantization+1

Open paper AskPDF arXiv

Mixture-of-Experts Large Language Models (MoE-LLMs) achieve strong performance but incur substantial memory overhead due to massive expert parameters. Mixed-precision quantization mitigates this cost by allocating expert-wise bit-widths based on their importance, approaching the accuracy-memory Pareto frontier and enabling extreme low-bit quantization. However, existing methods rely on layer-wise importance estimation and overlook router shifts induced by quantization, resulting in suboptimal allocation and routing. In this work, we propose Global Expert-level Mixed-precision Quantization (GEMQ) to overcome these limitations via (1) a global linear-programming formulation that captures model-wide expert importance based on quantization error analysis, and (2) efficient router fine-tuning to adapt routing to quantized experts. These components are integrated into a progressive quantization framework that iteratively refines importance estimation and allocation. Experiments demonstrate that GEMQ significantly reduces memory and accelerates inference with minimal accuracy degradation. Source code is available at https://github.com/jndeng/GEMQ .

Abstract

Quantization Sensitivity Analysis

Explore similar work

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees

AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization