Rotation-Based Quantization

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 36

All topics

ApexQuant: Data-Free Elastic Quantization by Residual Re-Isotropization
Oct 6, 2026Aksel Fristrup, Sumit Pandey, Ankit KariryaaLLM Quantization Rotation-Based Quantization
Henselian (henselian.com) · Department of Computer Science University of Copenhagen Denmark · Towards Deep Learning
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Sep 28, 2026Mehdi Makni, Ryan Lucas, Rahul MazumderLLM Quantization Rotation-Based Quantization
Operations Research Center Massachusetts Institute of Technology
TORQUE: Optimizing What (not) to Quantize Before and After Rotation
Sep 28, 2026Ran Ben Basat, Michael Mitzenmacher, Shay VargaftikRotation-Based Quantization Vector Quantization
University College London · Harvard University · VMware Research by Broadcom
SubRot: Signed Gradient Subspace Calibration for VLM Rotation Quantization
Sep 28, 2026Zhenhao Shang, Haizhao Jing, Haokui Zhang +4VLM Quantization Rotation-Based Quantization
Northwest Polytechnical University · Nanjing University of Science and Technology · Intellifusion +1
A Note on Scaling in Randomly Rotated Quantization and Its Connection to the CDEF +1 Pythagorean Relation
Sep 8, 2026Uri ErezRotation-Based Quantization
School of Electrical Engineering Tel Aviv University Tel Aviv, Israel
RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search
Aug 31, 2026Rastislav Lenhardt, Teodora Dobos, Thomas Vecchiato +2Rotation-Based Quantization Vector Quantization
Google Zurich, Switzerland · Technical University of Munich Munich, Germany · University of Copenhagen Copenhagen, Denmark +1
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Aug 13, 2026Shuhan Wang, Yilin Luo, Nan Xu +1LLM Quantization 4-Bit Quantization
RotaryQuant: Fitting 120B MoE Models on Consumer Hardware via Fused Compressed-Space Attention
Aug 8, 2026Anthony. Lui, Mohamed. Elsaied, N. P. SavaniExpert Offloading Mixture-of-Experts Language Models
Cognizant AI & Analytics, London, UB8 3PH, UK · Southern Methodist University (SMU), Engineering Management, Lyle School of Engineering, Dallas, TX, USA · Cognizant AI Lab, Responsible AI Office, San Francisco, USA +1
Beyond Rotations: AuroOFT for Expressive Quantized Orthogonal Fine-Tuning
Aug 5, 2026Yue Han, Dianlin WangLLM Quantization Fine-Tuning
College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology
Output-Aware Rotation for INT2 KV-Cache Quantization
Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4LLM Inference KV Caching
University of Southern California · Seoul National University · Inha University
LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference
Jul 30, 2026Sangjin Kim, Yuseon Choi, Jungjun Oh +2LLM Quantization AI Accelerator Inference
PIM Semiconductor Design Research Center (AI-PIM), Daejeon 34141, South Korea
GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference
Jul 30, 2026Sangjin Kim, Yuseon Choi, Byeongcheol Kim +2AI Accelerator Inference Rotation-Based Quantization
School of Electrical Engineering, Korea Advanced Institue of Science and Technology (KAIST)
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Jul 22, 2026Miguel P. Bento, João F. SeabraLLM Quantization Rotation-Based Quantization
Independent Researcher · Lisbon, Portugal
KronQ: LLM Quantization via Kronecker-Factored Hessian
Jul 8, 2026Donghyun Lee, Yuhang Li, Ruokai Yin +1Mixed-Precision Quantization LLM Quantization
University of Southern California · Yale University
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
Jul 2, 2026Donghyun Lee, Jitesh Chavan, Duy Nguyen +5Video Diffusion Models Diffusion Transformer
Cantina Labs · University of Southern California · *Work done during an internship at Cantina Labs. +1
RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generation
Jul 1, 2026Yaofu Liu, Wanli Lan, Jinxi Li +2Mixed-Precision Quantization Rotary Positional Embeddings
The Hong Kong University of Science and Technology, Hong Kong SAR, China · Independent Researcher, China
Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach
Jun 19, 2026Pedro M. R. Pereira, Felipe A. P. de Figueiredo, Rausley A. A. de SouzaLLM Quantization Rotation-Based Quantization
National Institute of Telecommunications (Inatel), Santa Rita do Sapucaí, MG 37536-001, Brazil
Ablation, Statistical Inference, and Validation for KV-Cache Compression
Jun 14, 2026Paolo D'Alberto, Ashish Siarasao, Elliott Delaye +1KV Caching Rotation-Based Quantization
Advanced Micro Devices, Inc.
Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantization
Jun 7, 2026Patrik Czakó, Gábor Kertész, Sándor SzénásiLLM Quantization 4-Bit Quantization
Doctoral School of Applied Informatics and Applied Mathematics, Obuda University Budapest, Hungary · John von Neumann Faculty of Informatics, Obuda University Budapest, Hungary
Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference
Jun 1, 2026Chi-Wei Huang, Chia-Chi TsaiLLM Quantization Rotation-Based Quantization
National Cheng Kung University
HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
May 28, 2026Artur Zagitov, Gleb Molodtsov, Aleksandr BeznosikovLLM Quantization Rotation-Based Quantization
1MIRAI · 2BRAIn Lab · 3Innopolis University
Ω-QVLA: Robust Quantization for Vision-Language-Action Models via Composite Rotation and Per-step Scaling
May 27, 2026Xinyu Wang, Mingze Li, Sicheng Lyu +6VLM Quantization Efficient VLA Model Inference
McGill University · Université de Montréal · Mila – Quebec AI Institute +2
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
May 25, 2026Preetam Sharma, Kacper DobekLLM Quantization Rotation-Based Quantization
Independent Research · Institute of Computing Science Poznan University of Technology, Poznan, Poland
Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
May 24, 2026Gorgi PavlovLLM Quantization Rotation-Based Quantization
Lehigh University
OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
May 20, 2026Mark Boss, Vikram Voleti, Simon Donné +1KV Caching Memory-Efficient Optimization
Stability AI
Block-Sphere Vector Quantization
May 19, 2026Heesang Ann, Joongkyu Lee, Min-hwan OhRotation-Based Quantization Vector Quantization
Seoul National University
OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
May 19, 2026Zunhai Su, Rui Yang, Chao Zhang +11KV Caching Rotation-Based Quantization
Tsinghua University · Meituan LongCat Team · The University of Hong Kong +3
TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
May 19, 2026Zukang Xu, Xing Hu, Dawei YangLLM Quantization Rotation-Based Quantization
1Houmo AI.
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
May 18, 2026Zhongzhu Zhou, Donglin Zhuang, Jisen Li +4Rotation-Based Quantization LLM Inference Acceleration
Together AI · University of Sydney · University of Illinois Urbana-Champaign
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
May 16, 2026Sayeh Sharify, Mahsa Salmani, Hesham MostafaDiffusion Transformer Diffusion Model Quantization
d-Matrix Santa Clara, CA 95054

Oct 6, 2026, cs.LGJ/K move · Enter open · S save

ApexQuant: Data-Free Elastic Quantization by Residual Re-Isotropization

Aksel Fristrup, Sumit Pandey, Ankit Kariryaa

Henselian (henselian.com) · Department of Computer Science University of Copenhagen Denmark · Towards Deep Learning

LLM Quantization Rotation-Based Quantization Low-Bit Quantization Neural Network Quantization

Open paper AskPDF arXiv

We introduce ApexQuant, a calibration-free quantization method that recursively re-quantizes the residual error, serving as a refinement layer on top of existing quantizers. We establish that a fresh random rotation returns each residual to the uniform distribution on the hypersphere, which characterizes the rate of progressive error decay across successive passes. This result lets us determine, before any weight is read, how many passes a layer needs for a target weight-space error. Every prefix is itself a valid lower-rate model, so one artifact serves several precisions. We instantiate ApexQuant with three interchangeable stages, scalar,

E_8

and trellis, and validate it on four open-weight LLMs and on Earth-observation and medical domains where in-distribution data is often unattainable as imagery arrives under restrictive licences or due to patient material under privacy constraints. Progressive re-isotropization comes within a few percent of full precision at four bits and gives the best two-bit arm we measure, in a completely data-free setting.

Momentum

Related topics

Latest papers 36