RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models

Chengzhu Bao; Xianglong Yan; Tianao Zhang; Jiaqi Chen; Shaoqiu Zhang; Yulun Zhang

cs.LGSep 30, 2026

RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models

Authors: Chengzhu Bao, Xianglong Yan, Tianao Zhang, Jiaqi Chen, Shaoqiu Zhang, Yulun Zhang

Organizations: Shanghai Jiao Tong University

Abstract

Post-training quantization (PTQ) has become a widely adopted technique for reducing the memory footprint and inference cost of large language models (LLMs). However, recent studies reveal that when applied to reasoning models, PTQ not only degrades reasoning performance but also exacerbates overthinking, leading to longer reasoning trajectories. These issues may offset the efficiency gains expected from lower-precision inference. Existing approaches mainly rely on complex optimization procedures. More recent lightweight inference strategies instead use predefined overthinking markers, limiting their adaptability across quantized models. To address these issues, we propose Reasoning Analysis and Token-level Inference Optimization (RATIO), a framework that identifies model-specific overthinking tokens and assigns each a tailored penalty. RATIO first introduces Quantization-aware Reasoning Behavior Analysis (QRBA) to identify overthinking tokens by analyzing discrepancies between full-precision and quantized models. It then adopts Token-Specific Penalty Determination (TSPD), which leverages full-precision guidance to derive token-specific penalties without additional training. Extensive experiments show that RATIO achieves a better accuracy-efficiency trade-off than existing token-level interventions. Specifically, RATIO achieves up to 9.8 points accuracy improvement and reduces chain-of-thought (CoT) length by up to 51.3% compared with quantized baselines. The code will be available at https://github.com/steven-bao1/RATIO.

Figures & tables

Figure 1: Performance comparison of different strategies across three quantized reasoning models. RATIO consistently achieves higher accuracy gains and larger CoT length reductions.

Panel of Figure 2 — Figure 2: Target tokens and calibrated penalties for Qwen-1.5B. (a) Word cloud of the 21 selected tokens, repeated for visualization. (b) Token-specific logit penalties in descending order.

Figure 3: Overview of RATIO. The two panels on the left illustrate QRBA, which discovers model-specific quantization-sensitive reasoning tokens by combining token-level discrepancies with reasoning-context information. The rightmost panel presents TSPD, which leverages full-precision guidance to derive token-specific penalties for inference-time calibration.

Figure 4: Numerical evidence construction in RTV. (A) Quantitative behavioral validation compares token probabilities under the same quantized prefixes. (B) Trajectory-level validation examines associations with incorrect answers and repetitive reasoning. Here, $Q$ and $F$ denote the quantized and full-precision models, respectively.

	AIME	MATH	GSM8K	GPQA	HumanEval	Avg.	$\Delta$ Acc	$\Delta$ Len%
	Acc (%) / Len (k)					Acc / Len	vs. baseline
BF16	20.83 / 25.11	85.60 / 6.06	84.69 / 2.81	36.36 / 9.97	73.17 / 6.50	60.13 / 10.09	–	–
GPTQ	4.17 / 51.53	54.00 / 24.27	68.76 / 9.54	31.82 / 23.67	10.37 / 39.31	33.82 / 29.66	–	–
+ Lotfi et al. (2026)	4.17 / 47.53	58.20 / 18.48	70.81 / 6.22	27.27 / 21.14	20.73 / 32.16	36.24 / 25.11	$+2.42$	$-15.34$
+RATIO	5.00 / 42.55	61.40 / 14.06	71.49 / 3.11	30.81 / 20.64	18.29 / 29.90	37.40 / 22.05	$+3.58$	$-25.66$
AWQ	7.50 / 55.53	44.80 / 36.13	61.18 / 22.16	23.23 / 36.56	17.68 / 39.81	30.88 / 38.04	–	–

Table 1: Accuracy (%) and CoT length (k tokens) for BF16 and W3-quantized models using GPTQ or AWQ, with shared-penalty decoding (+ Lotfi et al. (2026) ) or our method (+RATIO). The

\Delta

columns report the accuracy change in percentage points and the relative CoT length change (%) compared with the corresponding quantized model without intervention. For each model, the fixed-penalty baseline uses the penalty strength that achieves the highest mean accuracy across AWQ and GPTQ, averaged over the five benchmarks.

Panel of Figure 5 — Figure 5: Accuracy–efficiency trade-off and selected-token occurrence analysis. (a) Changes in average accuracy and CoT length for fixed-penalty decoding with different penalty strengths and RATIO on Qwen-7B, measured relative to the corresponding uncalibrated AWQ-W3 and GPTQ-W3 baselines. RATIO occupies the upper-left region (shorter CoT and higher accuracy) under both quantizers. (b) Benchmark-wise occurrence rates of QRBA-selected tokens on Qwen-1.5B with AWQ-W3 under RATIO, its component ablations, and the fixed-penalty baseline. RATIO consistently produces lower occurrence rates across benchmarks, showing that it effectively suppresses the selected target tokens during reasoning.

	GPQA	AIME	GSM8K	MATH	HumanEval	Avg.	$\Delta$ Acc	$\Delta$ Len%
	Acc (%) / Len (k)					Acc / Len	vs. baseline
AWQ	23.23 / 36.56	7.50 / 55.53	61.18 / 22.16	44.80 / 36.13	17.68 / 39.81	30.88 / 38.04	–	–
QRBA ( $\lambda=0.5$ )	29.80 / 32.37	4.17 / 48.73	69.29 / 9.17	59.40 / 20.21	34.76 / 29.86	39.48 / 28.07	$+8.60$	$-26.21$
QRBA ( $\lambda=1.0$ )	29.29 / 26.25	4.17 / 42.11	70.36 / 4.08	62.60 / 14.10	31.10 / 26.39	39.50 / 22.59	$+8.62$	$-40.62$
TSPD	20.71 / 31.74	7.50 / 47.74	66.19 / 10.20	59.00 / 20.21	33.54 / 28.25	37.39 / 27.63	$+6.51$	$-27.37$
RATIO	26.26 / 19.87	6.67 / 32.52	72.18 / 3.63	62.80 / 13.86	35.37 / 22.68	40.66 / 18.51	$+9.78$	$-51.34$

Table 2: Effect of QRBA and TSPD. Component-wise ablation on Qwen-1.5B under AWQ-W3 and GPTQ-W3. QRBA applies uniform penalties to the identified tokens, while TSPD applies token-specific penalties to the manually selected markers from prior work. Each entry reports accuracy (%) and CoT length (k tokens), with

\Delta

columns showing changes relative to the corresponding quantized baseline.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

		GPQA	AIME	GSM8K	MATH	HumanEval	Avg.	$\Delta$ Acc	$\Delta$ Len%
		Acc (%) / Len (k)					Acc / Len	vs. baseline
	FlatQuant	34.34 / 25.68	8.33 / 37.20	74.37 / 4.06	65.40 / 16.47	43.90 / 14.64	45.27 / 19.61	–	–
	+ Lotfi et al. (2026)	30.30 / 20.81	11.67 / 39.26	75.66 / 2.83	69.80 / 13.30	48.17 / 11.75	47.12 / 17.59	$+1.85$	$-10.30$
Qwen-1.5B	+RATIO	30.81 / 22.16	11.67 / 37.77	75.51 / 1.69	73.20 / 11.87	51.22 / 11.90	48.48 / 17.08	$\mathbf{+3.21}$	$\mathbf{-12.90}$

Appendix

Table 3: Performance of RATIO on Qwen-1.5B with FlatQuant W4A4KV4. Accuracy (%) and CoT length (k tokens) are reported as Acc/Len. The

\Delta

columns report changes relative to uncalibrated FlatQuant. For each metric, the highest accuracy and shortest CoT length are in bold.

Qwen-1.5B		Qwen-7B		Llama-8B		Qwen-14B
Token	$\bm{\lambda_{k}}$	Token	$\bm{\lambda_{k}}$	Token	$\bm{\lambda_{k}}$	Token	$\bm{\lambda_{k}}$
_but	1.22	_but	1.06	_back	1.00	_but	1.00
_what	1.21	_back	0.58	_try	0.90	_what	1.04
_try	0.73	_check	0.48	_well	0.86	_think	0.71
_again	1.16	_But	1.15	_check	0.93	_But	1.23
_well	0.88	_something	1.09	_But	1.12	_something	1.57

Appendix

Table 4: Model-specific tokens and normalized penalties used by RATIO. A leading underscore denotes a whitespace prefix in the decoded token.

Token	$\bm{\lambda_{k}}$	Token	$\bm{\lambda_{k}}$
_but	1.11	_question	1.11
_what	1.03	But	1.06
_back	0.83	_However	0.94
_think	0.81	but	1.00
_But	1.05	_perhaps	0.84
_something	1.00	Wait	1.02

Appendix

Table 5: Selected tokens and normalized penalties for Qwen3-4B in thinking mode. A leading underscore denotes a whitespace prefix in the decoded token.

Case 1: Repetitive reasoning after reaching the correct result
Prompt	From her science class study, Brady learned that each whale has 40 gallons of blood and that a shark has three times as much blood as a whale. How many gallons of blood do ten sharks have?
GPTQ-W3	Each shark has $3\times 40=120$ gallons, so ten sharks have $10\times 120=1200$ gallons. Wait, let me double-check that. $\ldots$ Wait, that seems correct. Let me think if I interpreted the question correctly. $\ldots$ Alternatively, perhaps I can think of the problem as ten sharks each having three times as much as a whale, so $10\times(3\times W)=3\times(10\times W)$ . Therefore, ten sharks have $3\times 10\times 40=1200$ gallons. Alternatively, perhaps I can think of it as $10\times(3\times W)=3\times(10\times W)$ . $\ldots$ Wait, 3 W. $\ldots$ 3 W3 W3 W.
RATIO	A shark has three times as much blood as a whale. Therefore, each shark has $3\times 40=120$ gallons of blood. Ten such sharks have $10\times 120=1200$ gallons of blood. The final answer is $\boxed{1200}$ .
Case 2: Repetition followed by an incorrect reasoning trajectory
Prompt	Compute $99^{2}+99+1$ in your head.
GPTQ-W3	First, $99^{2}=(100-1)^{2}=10{,}000-200+1=9{,}801$ . Adding 99 gives $9{,}900$ , and adding 1 gives $9{,}901$ . $\ldots$ Let me verify each step again. $99^{2}$ is 9,801, then $9{,}801+99=9{,}900$ , and adding 1 gives 9,901. $\ldots$ Alternatively, maybe I can factor it as $99\times(99+1)+1$ . That gives $99\times 100+1=9{,}901$ . Alternatively, if I think of $99^{2}+99+1$ as $(99+1)^{2}-1$ , that is $100^{2}-1=9{,}999$ . Wait, no, that is incorrect. $\ldots$ So, 9999. $\ldots$ 1099999999999999999999999999999999999999.

Appendix

Table 6: Dialogue examples on Qwen-1.5B with GPTQ-W3. Red marks repetitive or erroneous reasoning; green marks correct RATIO calculations and answers. Ellipses indicate omissions.

Explore similar work

Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
Jun 24, 2026Xinyu Lian, Walid Krichene, Beichen Huang +4Large Language Model Quantization Quantization-Aware Training
University of Illinois Urbana-Champaign · Microsoft · Anyscale +1
Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
May 29, 2026Sanae Lotfi, Polina Kirichenko, Steven Li +1Large Reasoning Models Chain-of-Thought Reasoning
1FAIR at Meta · 2Meta AI
Efficient Reasoning via Constrained Optimization in Latent Space
Sep 28, 2026Zhinan Hou, XingChen Li, Keyou YouLarge Reasoning Models Reasoning Traces
Tsinghua University

Jun 24, 2026, cs.AIJ/K move · Enter open · S save

Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

Xinyu Lian, Walid Krichene, Beichen Huang, Masahiro Tanaka+3

University of Illinois Urbana-Champaign · Microsoft · Anyscale+1

Large Language Model Quantization Quantization-Aware Training Inference Cost Repetition

Open paper AskPDF arXiv

Quantization is widely used to reduce the inference cost of large language models, but its effect on reasoning models is not fully captured by final-answer accuracy or per-token latency. We show that low-bit post-training quantization can introduce a hidden test-time compute cost: quantized reasoning models often generate longer chains of thought even when they still answer correctly. Across mathematical reasoning, code generation, scientific question answering, and agentic tool-use benchmarks, we find that INT4/INT3 quantization can preserve accuracy but increase reasoning-token usage, offsetting the expected per-token speedup. To measure this effect, we introduce the CoT Token Inflation Ratio, which compares reasoning length between quantized and full-precision models averaged across all evaluation benchmarks. We further show that token inflation is accompanied by behavioral changes in the reasoning trace, including more intermediate steps and greater semantic repetition. These changes translate into measurable end-to-end real-world serving penalties. Finally, we evaluate mitigation strategies and find that prompting and decoding-time sampling offer inconsistent accuracy-length trade-offs, while quantization-aware training shows more promise in reducing both accuracy degradation and token inflation. Our results suggest that reasoning-token usage should be reported alongside accuracy when evaluating quantized reasoning models.

Abstract

Explore similar work

Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not

Efficient Reasoning via Constrained Optimization in Latent Space