DPA-MoE: A Runtime Precision Control Framework for MoE Inference

Jaeyun Kim; Hoonki Lee; Susie Shin; Hyunmin Kim; Dahyun Lee; Sangyeob Kim

cs.CLOct 15, 2026

DPA-MoE: A Runtime Precision Control Framework for MoE Inference

Authors: Jaeyun Kim, Hoonki Lee, Susie Shin, Hyunmin Kim, Dahyun Lee, Sangyeob Kim

Organizations: Agency for Defense Development · Yonsei University

Abstract

Mixture-of-Experts (MoE) models reduce computation by routing each token to a small set of experts, but each selected expert is still used at a fixed precision. We present DPA-MoE, a runtime precision control framework that makes routed experts any-precision. DPA-MoE encodes expert feed-forward weights as prefix-decodable chunks using SharedScale AnyBCQ, profiles per-expert reconstruction residuals offline, and allocates chunks at inference time using live router gate scores. This post-routing allocation lets a single encoded MoE checkpoint operate at multiple effective weight budgets without retraining or re-encoding. Experiments on Qwen3-30B-A3B and DeepSeekMoE-16B show that DPA-MoE improves KL divergence and downstream accuracy over uniform chunk allocation at matched budgets, with modest steady-state throughput overhead in our prototype.

Figures & tables

Figure 1: DPA-MoE adds post-routing precision allocation to MoE: routing chooses which experts run, while DPA-MoE chooses how much of each selected expert is used.

Figure 2: SharedScale AnyBCQ encoding and dynamic precision inference. Left (Progressive Expert Encoding): chunks are added sequentially, with all active $\alpha$ planes jointly re-optimized at each step to produce a single shared scale table. Right (Dynamic Precision Inference): at inference time, selecting $k$ chunks loads the first $k$ bitplanes and scale columns from the shared table — no additional scale fetches are required when switching precision.

Figure 3: Two-Stage Precision Allocation. Offline profiling builds a residual table $\mathcal{R}[L,e,k]$ . At inference time, DPA combines this table with live gate scores and allocates a fixed chunk budget across routed experts.

Panel of Figure 4 — Figure 4: Mean KL divergence vs. effective model weight (GB) for DPA (ours), Uniform- $k$ allocation, and static quantization baselines on Qwen3-30B-A3B (left) and DeepSeekMoE-16B-Chat (right).

Panel of Figure 5 — Figure 5: Macro-average benchmark accuracy (%) vs. effective model weight (GB) for DPA (ours), Uniform- $k$ allocation, and static quantization baselines on Qwen3-30B-A3B (left) and DeepSeekMoE-16B-Chat (right).

	Qwen3-30B-A3B						DeepSeekMoE-16B-Chat
	Avg Acc (%)		TPS		TTFT (s)		Avg Acc (%)		TPS		TTFT (s)
$\bar{k}$	Uni	DPA	Uni	DPA	Uni	DPA	Uni	DPA	Uni	DPA	Uni	DPA
3	63.38	66.12	58.70	55.29	1.001	1.311	63.86	65.79	116.54	106.96	0.519	0.691
4	66.92	67.90	51.53	48.72	1.150	1.560	64.57	66.79	105.68	97.06	0.597	0.842
5	67.86	68.41	46.29	44.23	1.277	1.773	67.20	67.43	95.17	89.64	0.679	0.978
6	67.67	68.67	41.00	38.79	1.427	1.898	67.28	68.01	88.68	82.18	0.758	1.101

Table 1: Macro-average benchmark accuracy, generation throughput (TPS), and time-to-first-token (TTFT) on Qwen3-30B-A3B and DeepSeekMoE-16B-Chat under the Full-Memory setting.

$\bar{k}$	Uniform	Residual	Gate	DPA
	- $k$	-only	-only	(ours)
3	0.558387	0.514394	0.321453	0.306188
4	0.261631	0.249736	0.145686	0.137787
5	0.118282	0.113541	0.067866	0.064224
6	0.041207	0.041108	0.030344	0.028952

Table 2: Mean KL divergence on OLMoE-1B-7B-0125 with each allocation signal isolated. Lower is better.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Enc.	Bits	Scale	Total	%
AnyBCQ	4,608	2,520	7,128	35.4
SS-AnyBCQ	4,608	576	5,184	11.1

Appendix

Table 3: Per-expert storage breakdown in KB for full 2–8 bit support on Qwen3-30B-A3B.

$k$	diag/ex.	cross/ex.	Pearson	Spearman
2	0.402	0.598	0.987	0.999
3	0.881	0.119	0.972	1.000
4	0.916	0.084	0.980	1.000
5	0.733	0.268	0.972	1.000
6	0.831	0.169	0.955	1.000
7	0.871	0.129	0.979	1.000

Appendix

Table 4: Cross-term analysis per precision level

k

Panel of Figure 6 — Figure 6: Per-task benchmark results for Qwen3-30B-A3B.

Panel of Figure 7 — Figure 7: Per-task benchmark results for DeepSeekMoE-16B-Chat.

Method	Training-free	Router intact	Token-aware	Fine-grained $k$	No re-fetch on upgrade
MxMoE ( Duanmu et al., 2025 )	✓	✓	✗	✗	✗
HOBBIT ( Tang et al., 2024 )	✓	✓	$\triangle$	✗	✗
DynaExq ( Chu et al., 2025 )	✓	✓	✗	✗	✗
D 2 MoE ( Wang et al., 2025a )	✗	✓	✓	✗	✓
DPA-MoE (ours)	✓	✓	✓	✓	✓

Appendix

Table 5: Comparison with existing dynamic and mixed expert-precision methods.

\triangle

: HOBBIT switches precision only for cache-miss experts, so the decision is gated by cache state rather than by the gate score itself.

OLMoE-1B-7B-0125						Qwen1.5-MoE-A2.7B
		Mean KLD		Avg Acc (%)				Mean KLD		Avg Acc (%)
Config	GB	Uni	DPA	Uni	DPA	Config	GB	Uni	DPA	Uni	DPA
$k{=}2$	2.42	1.06908	1.06908	54.63	54.63	$k{=}2$	6.30	0.25540	0.25540	60.98	60.98
$k{=}3$	3.33	0.56371	0.32776	60.60	62.40	$k{=}3$	7.93	0.10721	0.04213	62.60	62.78
$k{=}4$	4.23	0.26034	0.14780	63.34	64.17	$k{=}4$	9.56	0.05210	0.01630	62.04	63.45
$k{=}5$	5.14	0.11913	0.06908	64.52	64.67	$k{=}5$	11.19	0.01953	0.00694	62.63	63.52

Appendix

Table 6: Mean KL divergence and macro-average benchmark accuracy versus effective weight-memory budget on the two additional MoE architectures.

$\bar{k}$	DPA (%)	Uniform- $k$ (%)	$\Delta$
3	72.48	72.22	$+0.26$
4	73.10	73.08	$+0.02$
5	73.44	73.54	$-0.10$
6	74.45	73.78	$+0.67$
7	74.23	73.25	$+0.98$
Avg	73.54	73.17	$+0.37$

Appendix

Table 7: PIQA accuracy under dynamic top-

p

routing (

p{=}0.25

, average

4.0

active experts per token) on OLMoE-1B-7B-0125-Instruct, averaged over 5 runs.

Scope	Field	Value
Models	Qwen3-30B-A3B	48 layers; top-8 routing over 128 experts.
	DeepSeekMoE-16B-Chat	28 layers; top-6 routing over 64 routed experts plus 2 shared experts.
	OLMoE-1B-7B-0125	16 layers; top-8 routing over 64 experts; no shared experts.
	Qwen1.5-MoE-A2.7B	24 layers; top-4 routing over 60 routed experts plus 4 shared experts.
Artifacts	Base HF checkpoints	Qwen/Qwen3-30B-A3B @ ad44e77 ; deepseek-ai/deepseek-moe-16b-chat @ eefd8ac ; allenai/OLMoE-1B-7B-0125 @ 9b0c1aa ; Qwen/Qwen1.5-MoE-A2.7B @ 1a758c5 .
	Instruct checkpoint	allenai/OLMoE-1B-7B-0125-Instruct @ b89a7c4 ; used only for the dynamic top- $k$ study of Appendix F .

Appendix

Table 9: Models, source artifacts, and compute.

Scope	Field	Value
Encoding	Target tensors	Expert FFN weights only; non-expert tensors use RTN Q8_0.
	Encoding	SharedScale AnyBCQ with prefix-decodable chunks.
	Chunk range	$k_{\min}{=}2$ , $K{=}8$ .
	Quantization shape	Group size 128; asymmetric reconstruction; learned $\beta$ ; no any-precision $\alpha$ .
	Optimization	FP32 reconstruction; Adam learning rates $10^{-4}/10^{-4}/10^{-4}$ ; 20 optimization iterations.
	Initialization	20 rounds of alternating refinement per BCQ sign bitplane.

Appendix

Table 10: Encoding and residual-profiling configuration.

Scope	Field	Value
KLD	Metric	Mean KL divergence against the BF16 reference.
	Dataset	WikiText-2 raw test text.
	Input	10 sequences; 4096 tokens per sequence.
	Runtime	Context 4096; batch 1; microbatch 1.
	Operating points	Static Q2_K/Q5_K_M/Q8_0; Uniform $k\in\{2,3,4,5,6,7,8\}$ ; DPA $\bar{k}\in\{2.5,3,3.5,4,4.5,5,5.5,6,6.5,7\}$ .
	Execution	Full pinned expert chunks; CUDA capture where available; VRAM caps: 40 GB for Qwen3, 30 GB for DeepSeekMoE.

Appendix

Table 11: Reconstruction-fidelity and downstream benchmark configuration.

Scope	Field	Value
Workload	Metrics	Generation TPS and time-to-first-token.
	Datasets	WikiText-2, C4-en, and C4-ko.
	Samples	100 sampled prompts per dataset.
	Input/output	1024-token prompts; 256 generated tokens.
	Repetitions	6 warm-up runs and 1 measured run per sample.
Runtime	Batching	Context 4096; batch 2048; microbatch 256.

Appendix

Table 12: Inference-speed measurement configuration.

Artifact	Source / Version	Terms	Use in this paper
Qwen3-30B-A3B	Qwen/Qwen3-30B-A3B@ad44e77	Model-card terms; Apache-2.0 at access.	Base MoE checkpoint used for encoding and evaluation; weights are not redistributed.
Qwen3 GGUF	unsloth/Qwen3-30B-A3B-GGUF@d5b1d57	Inherited model-card terms.	Static Q2_K/Q5_K_M/Q8_0 comparison; artifacts are downloaded separately.
DeepSeekMoE-16B	deepseek-ai/deepseek-moe-16b-chat@eefd8ac	DeepSeek model-card terms.	Base MoE checkpoint used for encoding and evaluation; weights are not redistributed.
DeepSeekMoE GGUF	mradermacher/deepseek-moe-16b-chat-GGUF@fe95dea	Inherited model-card terms.	Static Q2_K/Q5_K_M/Q8_0 comparison; artifacts are downloaded separately.
OLMoE-1B-7B	allenai/OLMoE-1B-7B-0125@9b0c1aa	Model-card terms at access.	Base MoE checkpoint for the additional-architecture sweep; weights are not redistributed.
OLMoE-1B-7B Instruct	allenai/OLMoE-1B-7B-0125-Instruct@b89a7c4	Model-card terms at access.	Router trajectory for the dynamic top- $k$ study; weights are not redistributed.

Appendix

Table 13: Model artifacts, versions, terms, and intended use.

Artifact	Source / Version	Terms	Use in this paper
WikiText-2	Raw train/test splits ( Merity et al., 2017 )	Dataset/source terms.	Calibration, profiling, KLD, and speed prompts.
C4 / mC4	C4-en ( Raffel et al., 2020 ) ; Korean mC4 ( Xue et al., 2021 )	Web-derived dataset/source terms.	Speed prompts only; possible inherited web-text risks are discussed in Ethical considerations.
ARC, PIQA, WinoGrande	Standard evaluation splits ( Clark et al., 2018 ; Bisk et al., 2020 ; Sakaguchi et al., 2021 )	Dataset/source terms.	Multiple-choice downstream evaluation; no new labels or human-subject collection.
llama.cpp	ggml-org/llama.cpp ( Gerganov and others, 2023 )	MIT.	Prototype runtime substrate; upstream notices are retained in the supplementary package.
AnyBCQ	Reference code ( Park et al., 2026 )	Apache-2.0.	Encoding substrate and kernel reference; upstream notices are retained in the supplementary package.

Appendix

Table 14: Dataset and software artifacts, terms, and intended use.

Mixture-of-Experts Inference

Mixture-of-experts inference studies how to run sparsely activated expert models efficiently while preserving their quality.

See all 173 papers

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Nov 19, 2025Kexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision Quantization GPU Acceleration
University of Connecticut · University of California, Santa Cruz · Independent Researcher
Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
Apr 9, 2026Baihui Liu, Kaiyuan Tian, Wei Wang +3Mixture-of-Experts Inference Mixture of Experts
National Key Laboratory of Parallel and Distributed Computing, College of Computer Science and Technology, National University of Defense Technology
Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
Apr 7, 2026Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4Mixture-of-Experts Quantization Neural Network Quantization
Rensselaer Polytechnic Institute · IBM Research

Nov 19, 2025J/K move · Enter open · S save

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference

Kexin Chu, Dawei Xiang, Zixu Shen, Yiwei Yang+2

University of Connecticut · University of California, Santa Cruz · Independent Researcher

Mixed-Precision Quantization GPU Acceleration Mixture-of-Experts Inference LLM Inference Acceleration Mixture-of-Experts Quantization Post-Training Quantization+1

Open paper AskPDF arXiv

Mixture-of-Experts (MoE) has become a practical architecture for scaling LLM capacity while keeping per-token compute modest, but deploying MoE models on a single, memory-limited GPU remains difficult because expert weights dominate the HBM footprint. Existing expert offloading and prefetching systems reduce the resident set, yet they often pay expert-loading costs on the critical path when activation becomes dense. Post-training quantization (PTQ) lowers the footprint without transfers, but prevailing pipelines fix expert bit-widths offline and assume routing remains stable, even though MoE expert utilization is heavy-tailed and the hot set can shift across workloads. We present DynaExq, a runtime-aware mixed-precision serving system that treats single-GPU MoE inference under a hard HBM envelope as an online, budget-constrained precision allocation problem. The key insight is to keep the experts that dominate runtime traffic resident at higher precision, while maintaining a low-precision fallback for the remaining experts, so the system can reduce transfer volume and avoid the waiting latency that limits offloading and prefetching under dense activation. DynaExq estimates long-horizon expert hotness from router traces, selects a per-layer high-precision resident set via a budget-feasible top-

n

rule, and applies promotions and demotions asynchronously through stable expert handles so the forward pass always executes on a fully materialized expert version. Across Qwen3-MoE-30B/80B and six benchmarks, DynaExq improves accuracy over static PTQ on Qwen3-80B (73.09% to 77.57%) under comparable device-memory budgets and achieves up to 2.73x higher throughput than offloading/prefetch baselines at batch size 32.

Abstract

Mixture-of-Experts Inference

Explore similar work

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference

Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees