SubRot: Signed Gradient Subspace Calibration for VLM Rotation Quantization

Zhenhao Shang; Haizhao Jing; Haokui Zhang; Guoting Wei; Rong Xiao; Jianqing Gao; Peng Wang

cs.CVSep 28, 2026

SubRot: Signed Gradient Subspace Calibration for VLM Rotation Quantization

Authors: Zhenhao Shang, Haizhao Jing, Haokui Zhang, Guoting Wei, Rong Xiao, Jianqing Gao, Peng Wang

Organizations: Northwest Polytechnical University · Nanjing University of Science and Technology · Intellifusion · iFLYTEK CO., LTD

Abstract

Post-training quantization reduces the deployment cost of vision-language models (VLMs), but preserving multimodal capabilities at low bit widths remains challenging. Existing methods rely on modality- or token-level gradient statistics, which are susceptible to cross-sample variations in visual-to-textual token ratios and the positions of visual information, limiting statistical stability. Moreover, overly coarse aggregation through absolute values and averaging discards gradient signs and channel-wise differences, limiting the separation of modality-specific sensitivities. In contrast, the channel space provides a shared coordinate system across samples, making it a more natural basis for capturing stable task-sensitive structures. We therefore propose SubRot, a signed gradient subspace calibration method for VLM rotation quantization. Through eigendecomposition of the empirical Fisher matrix of activation gradients, SubRot identifies a sensitive channel subspace with three properties: cross-sample stability, clear sensitivity separation, and consistent signed effects on the autoregressive loss along certain directions. Guided by a local Taylor expansion, SubRot combines signed first-order guidance along sign-stable directions with second-order constraints along the remaining sensitive directions, while retaining MSE for overall reconstruction quality. This objective steers quantization errors toward loss-decreasing directions while controlling their magnitude. Experiments on five VLMs across five benchmarks show consistent average-score improvements over FlatQuant under W4A6 and W4A4, reaching 1.4 percentage points on LLaVA-NeXT-7B. Under W4A4, average accuracy degradation from FP16 remains within 1.4 percentage points across all evaluated models, while LLaVA-v1.5-13B exceeds its FP16 average score by 0.4 percentage points.

Figures & tables

Figure 1: Overview of the proposed Fisher-informed subspace calibration. We eigendecompose the empirical Fisher matrix and select the top- $r$ eigen-directions as the sensitive subspace. Within this subspace, sign-stable directions are guided by the signed first-order loss $\mathcal{L}^{(1)}$ , while sign-unstable directions are constrained by the second-order loss $\mathcal{L}^{(2)}$ to suppress quantization errors.

Model	Bitwidth	Method	MMMU	OCRBench	VizWiz	ChartQA	SEED-2+	Average
LLaVA-Next-7b	fp16	-	36.7	51.2	58.7	50.4	50.6	49.5
	W4A6	RTN	14.8	23.7	37.5	27.6	3.9	21.5
		MBQ	28.8	40.6	52.9	41.7	41.1	41.0
		Quarot	32.5	47.6	53.7	50.8	48.0	46.5
		FlatQuant	33.1	50.5	57.2	52.5	51.3	48.9
		SubRot	35.0	51.8	58.1	52.7	51.4	49.8

Table 1: Quantization results of LLaVA-NeXT-7B, Qwen2-VL-7B, and LLaVA-v1.5-7B on five benchmarks under W4A6 and W4A4 settings, with FP16 performance as a reference.

Model	Bitwidth	Method	MMMU	OCRBench	VizWiz	ChartQA	SEED-2+	Average
Qwen2VL-2b	fp16	-	42.1	75.3	66.3	72.3	62.1	63.6
	W4A6	RTN	36.8	68.2	59.4	60.0	56.2	56.1
		MBQ	33.2	63.6	57.6	59.2	54.5	53.6
		Quarot	36.7	71.4	57.1	66.5	59.3	58.2
		FlatQuant	39.3	75.3	65.0	71.5	61.9	62.6
		SubRot	42.5	75.8	65.3	71.8	61.4	63.4

Table 2: Quantization results of Qwen2-VL-2B and LLaVA-v1.5-13B on five benchmarks under W4A6 and W4A4 settings, with FP16 performance as a reference.

$\mathcal{L}^{(1)}$ guide	$\mathcal{L}^{(2)}$ suppress	MMMU	OCRBench	SEED-2+	Average( $\uparrow$ )
-	-	33.1	50.5	51.3	45.0
✓	-	34.4	50.8	50.6	45.3
-	✓	34.8	50.1	51.5	45.5
✓	✓	35.0	51.8	51.4	46.1

Table 3: Ablation study of the first-order guidance loss for sign-stable directions and the second-order suppression loss for sensitive directions. Experiments are conducted on LLaVA-NeXT-7B under the W4A6 quantization setting.

Figure 2: Left: Stability of importance partitions under distribution shift. The $y$ -axis reports the Wasserstein-1 distance between the two log-ratio distributions, where lower values indicate greater stability. Shaded regions denote 95% bootstrap confidence intervals. Right: Layer-wise ratios of mean absolute autoregressive loss changes.

Figure 3: Cross-sample stability of the signed first-order effect. The solid line represents the sample mean, and the shaded region indicates the minimum–maximum range; negative values indicate a decrease in loss.

Bitwidth	Method	End-to-end average time(ms)	Peak Memory(GB)
fp16	-	2146.7	14.9
W4A4	Quarot	1588.9	8.5
	FlatQuant	1303.8	9.7
	SubRot	1303.8	9.7

Table 4: End-to-end inference speed and peak GPU memory usage measured after actual quantized deployment of LLaVA-Next-7B on 4,319 samples from the VizWiz dataset with single RTX 4090.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Subspace Statistics	Search/Training	Total
FlatQuant	-	39:43	39:43
SubRot	6:59	39:47	46:46

Appendix

Table 5: Calibration time breakdown of FlatQuant and SubRot (mm:ss).

Panel of Figure 4 — Figure 4: Eigenvalue spectra of the empirical Fisher matrices(excluding layers 8–19).

Explore similar work

RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models
Sep 21, 2026Zhiping Wu, Dongdong Ren, Yangchengyu Zhou +4Large Language Model Quantization Post-Training Quantization
State Key Laboratory for Novel Software Technology and the School of Intelligence Science and Technology, Nanjing University, Suzhou Campus, Suzhou 215163, China
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Sep 28, 2026Mehdi Makni, Ryan Lucas, Rahul MazumderLarge Language Model Quantization Qantis
Operations Research Center Massachusetts Institute of Technology
SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models
Aug 5, 2025Yufei Xue, Yushi Huang, Lunjie Zhu +2Recent Vision-Language Models Quantization-Aware Training
Institute of Artificial Intelligence (TeleAI), China Telecom · The Hong Kong University of Science and Technology

Sep 21, 2026, cs.CVJ/K move · Enter open · S save

RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models

Zhiping Wu, Dongdong Ren, Yangchengyu Zhou, Zhengjie Zhang+3

State Key Laboratory for Novel Software Technology and the School of Intelligence Science and Technology, Nanjing University, Suzhou Campus, Suzhou 215163, China

Large Language Model Quantization Post-Training Quantization Quantizer Multimodal Large Language Models Riemannian Manifolds

Open paper AskPDF arXiv

Large vision-language models (VLMs) can be efficiently deployed under stringent memory and latency constraints through post training quantization (PTQ). However, most PTQ methods are designed for unimodal large language models (LLMs). These methods treat quantization errors as isotropic perturbations under the Euclidean assumption, which provides weak guidance on directions most sensitive to quantization in VLMs. Consequently, directly adapting unimodal PTQ approaches or solely employing modality-specific scaling often leads to uneven bit-width distribution and inconsistent performance in low-bit settings. To address these challenges, we propose Riemannian Geometry-Sensitive Quantization (RGSQ), which formulates quantization as a reconstruction problem under a unified Fisher-Riemannian metric. RGSQ identifies modality-specific sensitive directions via Riemannian manifold mappings built from modality-partitioned empirical Fisher factors and fused into a modality-aware Kronecker-structured metric. We then apply geometry-aligned rotations to reorient the local tangent frame, steering low-bit perturbations toward loss-insensitive axes. Finally, we apply a whitening transformation that maps the Riemannian objective to an equivalent Euclidean form, enabling standard unimodal PTQ methods to evaluate multimodal quantization error under their original assumptions. Across an extensive and diverse set of mainstream VLM benchmarks, RGSQ achieves the highest accuracy and stability under extremely low-bit settings (W2A8 and W3A8). It outperforms VLM-aware baselines, such as MBQ and MQuant, by up to 5.9% and surpasses single-modality improvements by up to 8.6%.

Abstract

Explore similar work

RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models

ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs

SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models