cs.LGNov 16, 2026

JointQ: Joint Scale-and-Integer Quantization for Large Language Models

Authors: Keiji Kimura, Yuma Ichikawa

Organizations: Fujitsu Limited, Kanagawa, Japan

Abstract

GPTQ is the de facto standard solver for layer-wise post-training quantization (PTQ) of large language models, yet the solver itself has received less attention than weight preprocessing and cross-layer objectives. We target a bottleneck of GPTQ: its greedy procedure optimizes continuous group scales separately from the assignment of discrete integers. Evaluating each integer candidate with its closed-form optimal scale has been used for per-channel quantization; JointQ extends it to group-wise quantization, where a row's scales are coupled through the Hessian, and applies it to refine the GPTQ solution. Each local-search move changes one integer and recomputes its group's optimal scale in closed form, accounting for the other groups through the full Hessian; all scales are periodically re-solved jointly. The search starts from a row-wise selection among scale-refitted candidates including the GPTQ solution, so unregularized JointQ never has a larger layer-wise calibration error than GPTQ. JointQ preserves the GPTQ output format and is thus a drop-in replacement. Across four models and four settings, JointQ improves six-task average zero-shot accuracy over vanilla GPTQ in 14 of 16 comparisons and perplexity in 12 of 16. The mean accuracy gain increases from 0.0044 in W4G128 to 0.1559 in W3 per-channel, and JointQ also improves average accuracy over GPTQ with MSE-based scale search and activation ordering in all eight per-channel comparisons, though rarely in G128. An ablation shows that JointQ's diagonal Tikhonov regularization mitigates severe W3 per-channel perplexity degradation, so the largest gains are not due to the local search alone. Overall, JointQ's primary empirical benefit is downstream accuracy preservation in challenging low-bit and large-group regimes. Code is available in the OneComp library at https://github.com/FujitsuResearch/OneCompression.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. JARQ: Joint Alternating Refinement for Quantization

    Sep 29, 2026Xinyu Wang, Sicheng Lyu, Xiao-Wen ChangLLM QuantizationWeight-Only Quantization

  2. MPQ: A Message-Passing View of Post-Training Quantization

    Oct 23, 2026Yanshuo Chen, Chenxi Liu, Reza Shirkavand +1QuantizationLLM Quantization

  3. G2^2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

    Sep 25, 2026Ruikang Liu, Haoli Bai, Yuxuan Sun +8LLM QuantizationPost-Training Quantization