Mixture-of-Experts Quantization

Follow this topic Ask about this topic

Mixture-of-Experts (MoE) quantization studies how to represent the many expert networks in a sparsely routed model with fewer bits, making large models cheaper to store and run. Methods include post-training quantization and quantization-aware training, expert- or block-level mixed-precision allocation, and techniques that handle outliers or share low-rank and quantized structures across experts. Serving systems also adjust precision at runtime, offload experts, and use fused or hardware-specific kernels to balance memory, latency, and accuracy. Evaluations use measures such as perplexity, task accuracy, memory use, and throughput, while an open challenge is finding reliable sensitivity estimates: routing frequency and other inexpensive proxies do not always predict which precision choices preserve quality across workloads.

Latest papers

32 papers8 papers in the last four weeks, up 167% on the four weeks before. 0.0% of all new papers.

ButterflyMoE: Compression-Scalable Ternary Experts via Structured Butterfly Orbits
Jan 20, 2026Aryan KarmoreTernary Quantization Mixture-of-Experts Language Models
Indian Institute of Information Technology, Nagpur
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Nov 19, 2025Kexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision Quantization GPU Acceleration
University of Connecticut · University of California, Santa Cruz · Independent Researcher

Jan 20, 2026J/K move · Enter open · S save

ButterflyMoE: Compression-Scalable Ternary Experts via Structured Butterfly Orbits

Aryan Karmore

Indian Institute of Information Technology, Nagpur

Ternary Quantization Mixture-of-Experts Language Models LLM Compression Mixture-of-Experts Quantization

Open paper AskPDF arXiv

In current Mixture of Experts (MoE) architectures, linear memory scaling is present, the memory grows as the number of experts increases.

N

independent expert weight matrices require

\mathcal{O}(N \cdot d^2)

memory which exceeds the memory budget of edge devices. Current compression methods like quantization, pruning, and low-rank factorization reduce constant factors, but the scaling bottleneck is still unresolved. We introduce ButterflyMoE, a method which treats experts not as independent matrices but as geometric reorientations of a shared quantized substrate. Diversity amongst the experts arises from viewing different angles of the shared capacity and not from redundant storage. Learned rotations are applied to a shared ternary prototype. With this, each expert yields

\mathcal{O}(d^2 + N \cdot d \log d)

memory-reducing per-expert cost from

\mathcal{O}(d^2)

\mathcal{O}(d \log d)

. The key insight is that training these rotations with quantization reduces activation outliers and stabilizes extreme low-bit training where other static methods collapse. Across language modeling benchmarks, ButterflyMoE achieves 80

\times

memory reduction at 8 experts with a highly favorable memory-accuracy tradeoff.At this 80x compression ButterflyMoE outperforms an equal memory dense baseline, showing that orbital parameterization extracts fundamentally more utility per byte. When scaled up to 256 experts, ButterflyMoE asymptotically compresses the memory by 150

\times

. ButterflyMoE reduces the constant factor of linear scaling with compression ratio growing with the expert count.

Latest papers

ButterflyMoE: Compression-Scalable Ternary Experts via Structured Butterfly Orbits

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference