DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

Xiaoming Yu; Shize Tang; Guanghua Yu; Linchuan Xie; Song Liu; Jianchen Zhu; Feng Li

cs.LGMar 20, 2026

DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

Authors: Xiaoming Yu, Shize Tang, Guanghua Yu, Linchuan Xie, Song Liu, Jianchen Zhu, Feng Li

Organizations: Yuanbao & Hunyuan AI Infra Team

Abstract

We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization objectives minimize reconstruction error but are agnostic to the base model, allowing quantization noise to disproportionately corrupt the small-magnitude parameter deltas ( $ΔW$ ) that encode post-training behavior -- an effect we analyze through the lens of quantization as implicit regularization. DAQ replaces reconstruction-based objectives with two delta-aware metrics -- Sign Preservation Rate and Cosine Similarity -- that directly optimize for directional fidelity of $ΔW$ , requiring only the base and post-trained weight matrices. In a pilot FP8 study, DAQ recovers style-specific capabilities lost under standard quantization while maintaining general performance.

Figures & tables

Metric	Range	Delta-Aware	Complexity
MSE $\dagger$	$[0,+\infty)$	No	Low
SignRate	$[0,1]$	Yes	Low
CosSim	$[-1,1]$	Yes	Medium

Table 1: Comparison of quantization metrics.

Model	$\Delta$ W L2	SignRate (%)	CosSim	Style	General
Base (BF16)	–	–	–	0.215	1.501
Post-trained (BF16)	0	100%	1.000	1.709	1.438
AbsMax (FP8 block)	48641	54.54%	0.239	1.081	1.491
AbsMax (FP8 channel)	51790	62.48%	0.261	1.323	1.501
SmoothQuant (FP8 channel)	–	–	–	1.378	1.491
AWQ-W8A8 (FP8 channel)	–	–	–	1.399	1.479

Table 2: Baseline comparison. Standard FP8 quantization significantly degrades the SFT-specific style metric while general capabilities remain relatively stable.

\ddagger

Type	Range	$\Delta$ W L2	SignRate (%)	CosSim	Style	General
Block	$[0.5,2]$	36625	54.09%	0.228	0.516	1.528
Block	$[0.8,1.25]$	36938	53.39%	0.216	0.322	1.539
Block	$[0.9,1.11]$	28634	52.29%	0.235	0.260	1.571
Channel	$[0.5,2]$	33584	57.12%	0.231	0.642	1.555
Channel	$[0.8,1.25]$	36939	58.43%	0.236	0.569	1.499
Channel	$[0.9,1.11]$	33460	57.00%	0.223	0.440	1.493

Table 3: Scale search with MSE metric.

Type	Range	$\Delta$ W L2	SignRate (%)	CosSim	Style	General
Block	$[0.5,2]$	75028	76.96%	0.324	1.607	1.404
Block	$[0.8,1.25]$	66939	77.31%	0.363	1.718	1.380
Block	$[0.9,1.11]$	69786	82.30%	0.408	1.571	1.377
Channel	$[0.5,2]$	93524	77.23%	0.278	1.505	1.408
Channel	$[0.8,1.25]$	66691	77.62%	0.361	1.761	1.438
Channel	$[0.9,1.11]$	68953	80.38%	0.391	1.639	1.353

Table 4: DAQ with Sign metric. Best Style result per quantization type in bold .

Type	Range	$\Delta$ W L2	SignRate (%)	CosSim	Style	General
Block	$[0.5,2]$	48728	59.51%	0.295	1.554	1.515
Block	$[0.8,1.25]$	59263	64.46%	0.339	1.604	1.480
Block	$[0.9,1.11]$	57445	65.52%	0.342	1.647	1.463
Channel	$[0.5,2]$	53219	65.51%	0.315	1.545	1.454
Channel	$[0.8,1.25]$	60088	68.84%	0.346	1.706	1.442
Channel	$[0.9,1.11]$	63039	72.69%	0.369	1.726	1.471

Table 5: DAQ with Cosine metric. Best Style result per quantization type in bold .

Quantization

See all 49 papers

Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate
Apr 15, 2026Jaemin Kim, Sungkyun Kim, Junyeol Lee +1LLM Quantization Language Model Calibration
Seoul National University · Hanyang University
D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
Apr 18, 2026Junlin Li, Shuangyong Song, Guodong Du +6LLM Quantization LLM Compression
Harbin Institute of Technology, Shenzhen, China · TeleAI of China Telecom · The Hong Kong Polytechnic University +1
Why Does Post-Training Quantization Work?
Sep 10, 2026Yuxiang Chen, Michael Beyer, Jun Zhu +1LLM Quantization Quantization Robustness
Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University · College of AI, Tsinghua University · Bosch AI Research, Renningen, Germany

Apr 15, 2026J/K move · Enter open · S save

Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate

Jaemin Kim, Sungkyun Kim, Junyeol Lee, Jiwon Seo

Seoul National University · Hanyang University

LLM Quantization Language Model Calibration Post-Training Quantization Low-Bit Quantization Neural Network Quantization

Open paper AskPDF arXiv

Large Language Models (LLMs) are widely used across many domains, but their scale makes deployment challenging. Post-Training Quantization (PTQ) reduces memory footprint without retraining by leveraging a small calibration set. Recent Hessian-based PTQ methods compensate quantization error via cross-channel dependencies, but such approaches degrade at low bit-widths due to noisy curvature estimates from limited calibration data. We propose DASH-Q, a robust PTQ framework using diagonal Hessian approximation and iterative weighted least squares. By discarding noise-prone dependencies, DASH-Q filters sampling noise while prioritizing the preservation of salient feature power. We outperform other PTQ baselines in ultra low-bit regime, improving zero-shot accuracy by 7.01% on average and up to 14.01% over the strongest baselines across five baseline LLM models, while showing robust and stable performance with very small calibration data.

Abstract

Quantization

Explore similar work

Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate

D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation

Why Does Post-Training Quantization Work?