LLM Quantization

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

33 papers in the last four weeks, up 43% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 251

All topics

MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Jun 15, 2026Yuanteng Chen, Peisong Wang, Zhilei Liu +9Mixed-Precision Quantization LLM Quantization
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Zhongguancun Academy
ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training
Jun 14, 2026Janghwan Lee, Sihwa Lee, Jinseok Kim +4LLM Quantization Quantization-Aware Training
1Hanyang University, 2Rebellions Inc., Republic of Korea.
MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
Jun 14, 2026Yangjia Hu, Haodong Wang, Zicong Hong +8LLM Quantization 4-Bit Quantization
HKUST · EPFL · MetaX Integrated Circuits Co., Ltd
Signed Symmetric Quantization for Few-Bit Integers
Jun 12, 2026Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago +5LLM Quantization Weight-Only Quantization
AMD
ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scaling
Jun 11, 2026Sihwa Lee, Janghwan Lee, Donghoon Yoo +4LLM Quantization Language Model Decoding
Hanyang University, Seoul, Republic of Korea · Xenoscube Korean Inc., Seoul, Republic of Korea
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
Jun 11, 2026Zhixiong Zhao, Zukang Xu, Zhixuan Chen +3Mixed-Precision Quantization LLM Quantization
Houmo AI, China · 2Southeast University, Nanjing, China
Multi-Bitwidth Quantization for LLMs Using Additive Codebooks
Jun 11, 2026Liza Babaoglu, Shuangyi Chen, Ashish KhistiLLM Quantization Weight-Only Quantization
University of Toronto
DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics
Jun 10, 2026Zimo Zhao, Maolin Wang, Bowen Yu +3Mixed-Precision Quantization LLM Quantization
City University of Hong Kong · Hong Kong, China · Zhejiang University of Technology +1
TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs
Jun 9, 2026Wesley Pang, Gregory Hyegang Jun, Feiyang Liu +1Mixed-Precision Quantization LLM Quantization
University of Illinois Urbana-Champaign Department of Electrical and Computer Engineering USA
Optimal Post-Training Quantization Scales and Where to Find Them
Jun 9, 2026Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert +2LLM Quantization Weight-Only Quantization
AMD
UniSVQ: 2-bit Unified Scalar-Vector Quantization
Jun 9, 2026Haoyu Wang, Haiyan Zhao, Xingyu Yu +4LLM Quantization Vector Quantization
Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · School of Physics, Beihang University, Beijing, China
Quality Is Not a Safety Proxy Under Quantization
Jun 8, 2026Sahil KadadekarLLM Quantization LLM Refusal Behavior
Independent Researcher
APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancing
Jun 7, 2026Hong Guo, Nianhui Guo, Weixing Wang +3LLM Quantization GPU Kernel Optimization
Hasso Plattner Institute · German University of Digital Science
Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantization
Jun 7, 2026Patrik Czakó, Gábor Kertész, Sándor SzénásiLLM Quantization 4-Bit Quantization
Doctoral School of Applied Informatics and Applied Mathematics, Obuda University Budapest, Hungary · John von Neumann Faculty of Informatics, Obuda University Budapest, Hungary
Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression
Jun 5, 2026Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi +1Mixed-Precision Quantization LLM Quantization
UiT The Arctic University of Norway · University of Oslo, Norway
OffQ: Taming Structured Outliers in LLM Quantization by Offsetting
Jun 5, 2026Haoqi Wang, Lorenz K. Mueller, Jiawei Zhuang +2LLM Quantization 4-Bit Quantization
School of Computer and Communication Sciences, EPFL, Switzerland · 2Huawei, Switzerland · 3Swiss Data Science Center, ETHZ & EPFL, Switzerland
SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Serving
Jun 4, 2026Hongyuan Liu, Yawei Li, Zhiqiang Que +3LLM Quantization LLM Serving
University of Electronic Science and Technology of China · University of Bristol · 3ETH Zurich
LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models
Jun 4, 2026Rui Wang, Yan Zhao, Li Song +1Model Compression LLM Quantization
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University. Shanghai, China
FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models
Jun 4, 2026Haoyu Huang, Linlin Yang, Sheng Xu +5LLM Quantization Diffusion Model Quantization
National College for Excellent Engineers, Beihang University, Beijing, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China · Independent researcher +4
Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models
Jun 4, 2026Hancheol Park, Geonho Lee, Tairen Piao +1LLM Quantization Mixture-of-Experts Quantization
Nota Inc., South Korea.
Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation
Jun 4, 2026Fangbo Tu, Junhua Zhao, Chi Liu +4LLM Quantization Knowledge Distillation
PayPal, Inc.
Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models
Jun 3, 2026Rayyan Abdalla, Amir Hussein, Min Wu +1LLM Quantization LLM Inference Acceleration
STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models
Jun 3, 2026Xin Yan, Aqiang Wang, Zhenglin Wan +2LLM Quantization Diffusion Model Quantization
School of Artificial Intelligence, Beijing Normal University, Beijing, China · Department of Computer Science, National University of Singapore, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore
QuBLAST: A Framework for Quantizing Large Language Models with Block-Level Compression Approach and Activation Scaling Strategy
Jun 3, 2026Pasindu Wickramasinghe, Achyuta Muthuvelan, Rachmad Vidya Wicaksana Putra +2Mixed-Precision Quantization LLM Quantization
eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates
MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models
Jun 3, 2026Yue Wu, Changyuan Wang, Zixuan Wang +2LLM Quantization 4-Bit Quantization
Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
Jun 2, 2026Devleena Das, Rajeev Patwari, Elliott Delaye +1Mixed-Precision Quantization LLM Quantization
Advanced Micro Devices, Inc. (AMD)
dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats
Jun 2, 2026Giuseppe Franco, Ian Colbert, Pablo Monteagudo-Lago +2Mixed-Precision Quantization LLM Quantization
AMD
LLM Compression with Jointly Optimizing Architectural and Quantization choices
Jun 2, 2026Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi +1Mixed-Precision Quantization LLM Quantization
UiT The Arctic University of Norway · University of Oslo, Norway
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
Jun 2, 2026Lorenz K. Muller, Philippe Bich, Chiara Boretti +3LLM Quantization KV Caching
Huawei
LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection
Jun 2, 2026Liulu He, XuanAng Liu, Juntao Liu +8LLM Quantization LLM Compression
Nanjing University, China · China Mobile Research Institute, China · Xiaomi Corporation, China +2

Jun 15, 2026, cs.LGJ/K move · Enter open · S save

MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs

Yuanteng Chen, Peisong Wang, Zhilei Liu, Nanxin Zeng+8

Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Zhongguancun Academy

Mixed-Precision Quantization LLM Quantization VLM Quantization Multimodal Large Language Models Mixture-of-Experts Quantization Multimodal Foundation Models+3

Open paper AskPDF arXiv

Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer remarkable performance but incur prohibitive GPU memory costs, making compression essential. Among PTQ methods, expert-level mixed-precision quantization has proven effective for MoE-LLMs, yet suffers notable degradation on MoE-MLLMs due to two overlooked biases in expert importance estimation. (1) At the cross-modal level, the numerical dominance of vision tokens causes expert selection frequency to be dominated by vision tokens, masking experts that are critical to the text modality; (2) at the intra-vision level, the large proportion of redundant vision tokens further skew frequency statistics, obscuring experts critical for informative visual content. To bridge gaps, we propose MODE, a modality-decomposed expert-level mixed-precision quantization framework for MoE-MLLMs that decomposes expert selection frequency by modality, filters redundant vision tokens to obtain denoised visual frequency, and further evaluates quantization sensitivity per modality as a complementary signal to frequency-based estimation. These signals are integrated into an Integer Linear Programming formulation to assign per-expert bit-widths under a given budget. Extensive experiments show that MODE is particularly well-suited for MoE-MLLMs, limiting average performance loss to within 2.9% at W3A16, with larger gains at the extreme 2-bit setting.

Momentum

Related topics

Latest papers 251