cs.AINov 9, 2026

AutoQS: Automated Quantization Strategy Design with Large Language Models

Authors: Zhen Liu, Zhiqin Yang, Yonggang Zhang, Jianyi Liu, Jinjun Wang, Jingwen Fu

Organizations: Xi’an Jiaotong University · The Hong Kong University of Science and Technology · Zhongguancun Academy

Abstract

Quantization reduces LLM deployment costs, but effective strategies must balance model quality and storage. LLM-driven search can automate strategy design through repeated proposal and evaluation. This process must identify useful strategies within a limited evaluation budget. The key challenge is that actual quantization is expensive, while inexpensive proxies may not reflect quantized-model performance. To address this challenge, we present AutoQS, an Automated Quantization Strategy Design with LLMs. AutoQS keeps weight precision fixed and searches group-size allocations across layers and operators under deployment constraints. Evaluation history guides coordinated LLM proposals, while deterministic validation ensures compatible allocations. An inexpensive quality proxy evaluates candidates, and an candidate repository retains alternative allocations to inform subsequent proposals. Periodic actual quantization complements these proxy evaluations with measured quality and checkpoint storage, providing feedback for further search within the evaluation budget. Experiments on Qwen2.5-7B-Instruct show that AutoQS improves seven-task average accuracy over uniform-group GPTQ and AWQ by 0.70 and 1.12 percentage points, respectively, at four-bit weight precision.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training

    Oct 6, 2026Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2LLM QuantizationQuantization-Aware Training

  2. Disaggregated Quantization: Specializing LLM Prefill and Decode

    Sep 22, 2026Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2LLM QuantizationLLM Inference

  3. SliderQuant: Accurate Post-Training Quantization for LLMs

    Mar 26, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +3LLM QuantizationPost-Training Quantization