Mixture-of-Experts Quantization

Follow this topic Ask about this topic

Mixture-of-Experts (MoE) quantization studies how to represent the many expert networks in a sparsely routed model with fewer bits, making large models cheaper to store and run. Methods include post-training quantization and quantization-aware training, expert- or block-level mixed-precision allocation, and techniques that handle outliers or share low-rank and quantized structures across experts. Serving systems also adjust precision at runtime, offload experts, and use fused or hardware-specific kernels to balance memory, latency, and accuracy. Evaluations use measures such as perplexity, task accuracy, memory use, and throughput, while an open challenge is finding reliable sensitivity estimates: routing frequency and other inexpensive proxies do not always predict which precision choices preserve quality across workloads.

Latest papers

32 papers8 papers in the last four weeks, up 167% on the four weeks before. 0.0% of all new papers.

Training an MoE LLM in MXFP4: How We Closed the Gap to BF16
Oct 23, 2026Han Wang, Zhitao Wang, Natalia Frumkin +4LLM Quantization Low-Precision Training
Advanced Micro Devices, Inc.
ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models
Oct 22, 2026Jiaqi Chen, Xianglong Yan, Chengzhu Bao +2Mixture-of-Experts Quantization Post-Training Quantization
Shanghai Jiao Tong University
DPA-MoE: A Runtime Precision Control Framework for MoE Inference
Oct 15, 2026Jaeyun Kim, Hoonki Lee, Susie Shin +3Mixture-of-Experts Inference Adaptive Inference
Agency for Defense Development · Yonsei University
Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors
Oct 14, 2026Yifan LiuQuantization Sensitivity Analysis Mixture-of-Experts Quantization
UIUC MCS Student
Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization
Oct 14, 2026Renjie Xie, Juncheng Yang, Aoting Hu +5Mixed-Precision Quantization Mixture-of-Experts Quantization
Nanjing University of Posts and Telecommunications · Tylogi AI Lab / TAIL · Anhui University of Technology +4
DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving
Oct 9, 2026Mengqi Guo, Zhe Wei, Yuan Yuan +3Mixture-of-Experts Quantization Mixture-of-Experts Inference
Huawei Technologies Ltd.
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
Oct 6, 2026Xin Wang, Hao Yu, Zhengyang Zhuge +9LLM Quantization Quantization-Aware Training
Alibaba Token Hub, Alibaba Group · Ohio State University
Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts
Sep 29, 2026Kwanhee Lee, Namhoon Lee, Dan AlistarhMixture-of-Experts Quantization Mixture-of-Experts Inference
POSTECH & ISTA · POSTECH · ISTA
Colla-Q: Toward Collaborative Experts in MoE Quantization via Minimax Precision Balancing
Sep 16, 2026Eunju Shin, Jongbin RyuMixed-Precision Quantization Mixture-of-Experts Quantization
Ajou University, South Korea
Quality-Constrained Routing over a Fixed Pool of Quantized Mixture-of-Experts Instances
Sep 11, 2026Zhenghong Huang, Hongfan Wu, Jiheng ZhangMixture-of-Experts Inference Mixture-of-Experts Quantization
The Hong Kong University of Science and Technology Clear Water Bay, Kowloon, Hong Kong
Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs
Aug 31, 2026Deokjae Lee, Sihun Chu, Hyun Oh SongMixed-Precision Quantization LLM Quantization
Department of Computer Science and Engineering, Seoul National University · Neural Processing Research Center · 3Neural Processing Research Center +2
Benchmarking Composable Compression Techniques in Mixture-of-Experts LLMs
Aug 22, 2026Afsara Benazir, Chen Chen, Rongxiao Qu +3Mixture-of-Experts Inference LLM Evaluation
University of Virginia · Sony AI
Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inference
Aug 15, 2026Amjad SaabMixture-of-Experts Inference Mixture-of-Experts Language Models
Appendture Dubai, United Arab Emirates
Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving
Aug 9, 2026Matteo GrellaTernary Quantization Mixture-of-Experts Inference
Crisis24
RotaryQuant: Fitting 120B MoE Models on Consumer Hardware via Fused Compressed-Space Attention
Aug 8, 2026Anthony. Lui, Mohamed. Elsaied, N. P. SavaniExpert Offloading Mixture-of-Experts Language Models
Cognizant AI & Analytics, London, UB8 3PH, UK · Southern Methodist University (SMU), Engineering Management, Lyle School of Engineering, Dallas, TX, USA · Cognizant AI Lab, Responsible AI Office, San Francisco, USA +1
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
Jul 17, 2026Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1LLM Quantization Mixture-of-Experts Language Models
University of Illinois Urbana-Champaign
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Jun 15, 2026Yuanteng Chen, Peisong Wang, Zhilei Liu +9Mixed-Precision Quantization LLM Quantization
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Zhongguancun Academy
Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models
Jun 4, 2026Hancheol Park, Geonho Lee, Tairen Piao +1LLM Quantization Mixture-of-Experts Quantization
Nota Inc., South Korea.
AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Jun 3, 2026Wanqi Yang, Yuexiao Ma, Alexander Conzelmann +4Mixed-Precision Quantization Mixture-of-Experts Language Models
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center +5
BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization
May 22, 2026Jiayu Zhao, Zihan Teng, Minhao Fan +4Mixed-Precision Quantization LLM Quantization
School of Microelectronics, University of Science and Technology of China · College of Computing and Data Science, Nanyang Technological University · School of Electrical and Electronic Engineering, Nanyang Technological University
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
May 21, 2026Jianing Deng, Song Wang, Dongwei Wang +4Mixed-Precision Quantization LLM Quantization
University of Pittsburgh · University of Central Florida · University of Arizona +1
RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compression
May 14, 2026Zhengjia Zhong, Shuyan Ke, Zaizhou Lin +3Residual VQ Vector Quantization
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, China.
TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
May 10, 2026Hongyaoxing Gu, Xinzhe Chen, Lijuan Hu +1Efficient Inference Mixture-of-Experts Inference
Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences
CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
Apr 12, 2026Xiangyang Yin, Xingyu Liu, Tianhua Xia +5Mixture-of-Experts Quantization Neural Network Quantization
Courant Institute of Mathematical Sciences, New York University · Tandon School of Engineering, New York University · Cerebras Systems Inc.
MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization
Apr 8, 2026Zhixiong Zhao, Zukang Xu, Zhixuan Chen +1Mixture-of-Experts Inference Mixture-of-Experts Quantization
Houmo AI
Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
Apr 7, 2026Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4Mixture-of-Experts Quantization Neural Network Quantization
Rensselaer Polytechnic Institute · IBM Research
DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge
Mar 19, 2026Yuegui Huang, Zhiyuan Fang, Weiqi Luo +3Mixture-of-Experts Inference Mixture-of-Experts Quantization
Sun Yat-sen University, Guangzhou, China · Tencent, Shenzhen, China
ButterflyViT: 354 $\times$ Expert Compression for Edge Vision Transformers
Mar 6, 2026Aryan KarmoreEfficient ViTs Vision Transformer
Indian Institute of Information Technology, Nagpur
Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization
Feb 27, 2026Chenwei Jia, Baoting Li, Xuchong Zhang +3VLM Quantization Neural Network Quantization
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University
pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training
Feb 26, 2026Wenzheng Zhang, Bingzheng Liu, Yang Hu +3LLM Quantization Low-Bit Quantization
School of Computer Science, Peking University · College of Future Information Technology, Fudan University · Center for Information Research, Academy of Military Sciences +2

Oct 23, 2026J/K move · Enter open · S save

Training an MoE LLM in MXFP4: How We Closed the Gap to BF16

Han Wang, Zhitao Wang, Natalia Frumkin, Xuanwu Yin+3

Advanced Micro Devices, Inc.

LLM Quantization Low-Precision Training LLM Training Mixture-of-Experts Quantization

Open paper AskPDF arXiv

Training large language models (LLMs) in sub-8-bit arithmetic promises substantial gains in memory bandwidth and compute throughput, yet FP4 formats introduce quantization error severe enough to destabilize optimization if left unaddressed. We present ALTO, an open-source training recipe for GPT-OSS-20B, a 20-billion-parameter mixture-of-experts (MoE) model, on the MLPerf Small MoE Training benchmark under the MXFP4 microscaling format. The recipe combines established techniques from Abecassis, Felix et al. [arXiv:2509.25149] (hybrid 1D/2D block quantization, randomized Hadamard transforms, and stochastic rounding) with a simplified weight de-oscillation scheme adapted from TetraJet-v2 [arXiv:2510.27527]. On C4 validation at global batch size 16, MXFP4 + RHT + SR + de-oscillation reaches a validation loss of 3.3350 at 16,128 steps, only 0.007 above the BF16 baseline (3.3283), closing roughly 50% of the gap left by MXFP4 without de-oscillation (3.3418). Measured by steps to reach the MLPerf quality target (validation loss 3.34), de-oscillation cuts the FP4 convergence overhead relative to BF16 from +20.0% to +5.0%. We also report negative end-to-end results from differential gradient estimation, outlier clipping, and macro-block scaling, despite each showing operator-level SNR gains. All experiments use simulated MXFP4 kernels on AMD MI300X hardware and wall-clock speed has not been optimized.