Quantization

Latest papers

48 papers12 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
Oct 15, 2026Johannes Knittel, Alec Helbling, Hanspeter PfisterLLM Quantization Neural Network Quantization
Harvard University · Georgia Tech
PLEDGE: Progressive Lossy Error-Bounded Compression of 3D Gaussian Splatting
Oct 13, 2026Mingze Xia, Yuxiao Li, Baixi Sun +6Quantization 3DGS Compression
Oregon State University · The Ohio State University · Argonne National Laboratory +1
Fast Quantized Johnson $-$ Lindenstrauss Transforms
Oct 12, 2026Angelos Korakitis, Vasileios NakosQuantization Randomized Sketching
National and Kapodistrian University of Athens
OnlineQAT: On-Policy Distillation for Ultra-Low-Bit Large Language Models
Oct 7, 2026Wenjun Wang, Heng Li, Yanggan Gu +1LLM Quantization Quantization
The Hong Kong Polytechnic University · Sun Yat-sen University · PolyU-Daya Bay Technology and Innovation Research Institute
Few Bits, One Law: Toward W2A4KV2
Oct 6, 2026Kai Yi, Tarek Elgamal, Sruthikesh Surineni +3Neural Network Quantization LLM Quantization
Meta AI
Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Oct 6, 2026Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2LLM Quantization Quantization-Aware Training
ISTA · EPFL, ISTA
ApexQuant: Data-Free Elastic Quantization by Residual Re-Isotropization
Oct 6, 2026Aksel Fristrup, Sumit Pandey, Ankit KariryaaLLM Quantization Rotation-Based Quantization
Henselian (henselian.com) · Department of Computer Science University of Copenhagen Denmark · Towards Deep Learning
Shared Stopping Decisions Change Answers in HQQ Cache Quantization
Oct 5, 2026Seunghui Jwa, Minsu Oh, Chanjun Park +1LLM Inference KV-Cache Quantization
Department of Artificial Intelligence, Jeju National University · School of Software, Soongsil University
ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
Sep 29, 2026Jonathan Mei, Sang Hyub Kim, Oliver Knitter +2LLM Quantization Weight-Only Quantization
Applications R&D IonQ College Park, MD 20740, USA
TORQUE: Optimizing What (not) to Quantize Before and After Rotation
Sep 28, 2026Ran Ben Basat, Michael Mitzenmacher, Shay VargaftikRotation-Based Quantization Vector Quantization
University College London · Harvard University · VMware Research by Broadcom
From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers
Sep 28, 2026Nafiseh HosseinpourFardi, Negar Alihadi, Mahmoudreza Babaei +2Mixed-Precision Quantization LLM Quantization
Sharif University of Technology, Tehran, Iran · GISMA University of Applied Sciences, Potsdam, Germany · BRAINS, Brandenburg Research Center for Applied Intelligent Systems, Potsdam, Germany +3
QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization
Sep 28, 2026Qiulin Shang, Zhoutong Wu, Jie Hu +1LLM Quantization Large Language Model-Guided Optimization
Peking University
JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization
Sep 27, 2026Kaicheng Yang, Kaisen Yang, Chunyu Liu +8Neural Network Quantization Diffusion Model Quantization
Shanghai Jiao Tong University · Tsinghua University · The Hong Kong Polytechnic University
Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
Sep 22, 2026Yuanteng Chen, Zhilei Liu, Peisong Wang +7Low-Bit Quantization LLM Quantization
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy +2
The Undetected Damage of Quantization on Retrieval and How to Fix It
Sep 21, 2026Luca Zhou, Alessandro Zirilli, Daniele Solombrino +2Mixed-Precision Quantization Information Retrieval
Sapienza University of Rome · Not Diamond · Paradigma
Structured Transforms for Low-Overhead Quantization of Language Models
Sep 10, 2026Daria Cherniuk, Alexander Rudikov, Boris Kashin +1LLM Quantization Weight-Only Quantization
Institute of Numerical Mathematics · Steklov Mathematical Institute
Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions
Sep 4, 2026Mahadev Sunil Kumar, Bhavika Gondi, Desaisetty Venkata Satya Sai Swapnith +6Quantization Model Compression
Department of Computer Science and Engineering, Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, India · Amrita School of Agricultural Sciences, Amrita Vishwa Vidyapeetham, Coimbatore, 642109, Tamil Nadu, India · Intel Corporation, Santa Clara, California, USA +2
Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs
Aug 21, 2026Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5Quantization-Aware Distillation LLM Quantization
Multiverse Computing
ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
Aug 7, 2026Yongge Ma, Guoan Wang, Feiyu Wang +5LLM Quantization Post-Training Quantization
School of Computer Science, Peking University · School of Software and Microelectronics, Peking University · School of Physics, Peking University +2
Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
Aug 6, 2026Zekun Wu, Swati Dhiman, Adriano KoshiyamaPost-Training Quantization Low-Bit Quantization
Holistic AI · UCL Centre for Artificial Intelligence, University College London
One Qubit Can Beat One Bit: Quantum Advantage for Post-Training Quantization
Aug 5, 2026Yuma Ichikawa, Moeto MishimaBinary Quantization Post-Training Quantization
Fujitsu Limited, RIKEN for AIP · RIKEN for AIP · Fujitsu Limited
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
Aug 4, 2026Seokho Han, Dongwei Wang, Jinhee Kim +4Weight-Only Quantization Mixed-Precision Quantization
Department of Electrical and Computer Engineering, Sungkyunkwan University, Korea · Department of Electrical and Computer Engineering, University of Arizona, USA · Department of Electrical and Computer Engineering, Duke University, USA +1
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
Jul 29, 2026Jiwon Jang, Kisu Yang, Heuiseok Lim +1Post-Training Quantization AI Agent Reliability
VAIV Company · Seoul National University · Korea University
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Jul 22, 2026Miguel P. Bento, João F. SeabraLLM Quantization Rotation-Based Quantization
Independent Researcher · Lisbon, Portugal
Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models
Jul 3, 2026Hamish OgilvyLLM Inference Acceleration Mixed-Precision Quantization
Algolia Research
Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers
Jul 1, 2026O. M. KiselevWeight-Only Quantization Post-Training Quantization
Innopolis University, Innopolis, Republic of Tatarstan, Russian Federation
Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformers
Jul 1, 2026Jaeah Lee, Hyunjin Kim, Jaewoong Cho +1Quantization Model Compression
KRAFTON AI, Republic of Korea · Amazon, Australia
An Embedded Real-Time License Plate Recognition System for Complex Traffic Scenes
Jun 26, 2026Anuki Pasqual, Dulan Lokugeegana, Manimohan Thiriloganathan +3Quantization Automatic License Plate Recognition
The authors are with the Dept. of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
Jun 22, 2026Yuval Domb, Hadar Sackstein, Tomer SolbergLLM Quantization Diffusion Model Quantization
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
Jun 21, 2026Yitong Li, Junsong Chen, Haopeng Li +6Quantization Diffusion Model Inference Acceleration
NVIDIA Research, Efficient AI Team & Singapore Lab.

Oct 15, 2026J/K move · Enter open · S save

RoleQ: Role-Aligned Scales for Converting Language Models to Ternary

Johannes Knittel, Alec Helbling, Hanspeter Pfister

Harvard University · Georgia Tech

LLM Quantization Neural Network Quantization Quantization Ternary Quantization Post-Training Quantization Weight-Only Quantization

Open paper AskPDF arXiv

Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy. Ternary weights have mostly been obtained by training from scratch or long fine-tuning. We introduce RoleQ, a quantization framework that places scales by the functional roles of Transformer units rather than by the layout of the weight matrix, as conventional formats do. Vectors that read from the residual stream take one scale per output unit; those that write to it, one per input unit. The rule is structural, needs no search, calibration or stored state, and holds on integer and ternary grids. In a rotated basis it matches a per-layer axis search at half the solver work, and at ternary it cuts perplexity by up to half against layout-based placement in two families at every size from 0.6B to 14B, where per-row scales leave MMLU at chance. Our lightweight post-training pipeline converts an existing checkpoint to 1.59 bits per weight in a few GPU-hours, a fraction of the cost of learned ternarization, and leads every post-training ternary method at matched or lower memory-resident bits on perplexity, zero-shot accuracy and MMLU, most widely on knowledge. A sparse second ternary plane placed by a per-unit Fisher saliency keeps the matmul multiplication-free and recovers what ternary loses first: at 8B, a fraction of a bit returns about half the MMLU that conversion costs, well ahead of random or Hessian-based placement at equal bits.