On the Step Length Confounding in LLM Reasoning Data Selection

Bing Wang; Rui Miao; Chen Shen; Shaotian Yan; Kaiyuan Liu; Ximing Li; Xiaosong Yuan; Sinan Fan

cs.CLApr 8, 2026

On the Step Length Confounding in LLM Reasoning Data Selection

Authors: Bing Wang, Rui Miao, Chen Shen, Shaotian Yan, Kaiyuan Liu, Ximing Li, Xiaosong Yuan, Sinan Fan,

Organizations: College of Computer Science and Technology, Jilin University · Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University · Alibaba Cloud Computing · School of Artificial Intelligence, Jilin University ·

Abstract

Large reasoning models have recently demonstrated strong performance on complex tasks that require long chain-of-thought reasoning, through supervised fine-tuning on large-scale and high-quality datasets. To construct such datasets, existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalness-based selection methods to filter high-quality samples. Despite the proven effectiveness of naturalness-based data selection, which ranks data by the average log probability assigned by LLMs, our analysis shows that, when applied to LLM reasoning datasets, it systematically prefers samples with longer reasoning steps (i.e., more tokens per step) rather than higher-quality ones, a phenomenon we term step length confounding. Through quantitative analysis, we attribute this phenomenon to low-probability first tokens in reasoning steps; longer steps dilute their influence, thereby inflating the average log probabilities. To address this issue, we propose two variant methods: ASLEC-DROP, which drops first-token probabilities when computing average log probability, and ASLEC-CASL, which applies a causal debiasing regression to remove the first tokens' confounding effect. Experiments across four LLMs and five evaluation benchmarks demonstrate the effectiveness of our approach in mitigating the step length confounding problem.

Figures & tables

Figure 1: Step length distribution of data samples selected and unselected by different naturalness-based data selection methods.

Figure 2: Relationship between step - level log probability and step length.

Figure 3: Representative cases illustrating token - level log probabilities for varying step lengths.

Qwen	Method	AIME24		AIME25		MATH500	OlympicB	Avg.
		Accuracy	Pass@4	Accuracy	Pass@4
4B-Base	+ GRACE ( Zhang et al., 2025 )	16.66	30.00	15.83	33.33	59.40	33.33	31.42
	+ Local LP ( Just et al., 2025 )	19.16	36.66	20.83	36.66	71.60	34.11	36.50
	+ Aslec-drop (ours)	30.00 $\uparrow$ 10.84	50.00 $\uparrow$ 13.34	28.33 $\uparrow$ 7.50	43.33 $\uparrow$ 6.67	77.80 $\uparrow$ 6.20	38.38 $\uparrow$ 4.27	44.64
	+ Aslec-casl (ours)	31.66 $\uparrow$ 12.50	53.33 $\uparrow$ 16.67	30.83 $\uparrow$ 10.00	46.66 $\uparrow$ 10.00	80.00 $\uparrow$ 8.40	42.81 $\uparrow$ 8.70	47.54
8B-Base	+ GRACE ( Zhang et al., 2025 )	30.83	53.33	21.66	36.66	72.00	39.70	42.36

Table 1: Experimental results on LIMO-v2 ( Ye et al., 2025 ) . We generate five responses per source LLM, and select five responses from these ones (select 4k responses from 16k data). The bold results represent the best scores.

Method	Acc.	Pass@4	Acc.	Pass@4
	Qwen3-4B-Base		Qwen3-4B-Insruct
GRACE	28.15	60.10	50.37	75.75
Local LP	29.16	61.61	52.14	77.27
Aslec-drop	34.97	65.65	58.83	83.33
Aslec-casl	35.35	66.66	61.23	84.34
Method	Acc.	Pass@4	Acc.	Pass@4

Table 2: Experimental performance on GPQA .

Figure 4: Step length distributions for data selected versus unselected by our two proposed variant methods.

Qwen	Method	AIME24		AIME25		MATH500	OlympicB	Avg.
		Accuracy	Pass@4	Accuracy	Pass@4
4B-Base	+ GRACE ( Zhang et al., 2025 )	30.00	60.00	29.16	36.66	77.20	42.50	43.82
	+ Local LP ( Just et al., 2025 )	32.50	63.33	30.00	40.00	77.80	43.08	45.22
	+ Aslec-drop (ours)	41.66 $\uparrow$ 9.16	73.33 $\uparrow$ 10.00	30.83 $\uparrow$ 0.83	43.33 $\uparrow$ 3.33	84.00 $\uparrow$ 6.20	47.20 $\uparrow$ 4.12	46.48
	+ Aslec-casl (ours)	46.66 $\uparrow$ 14.16	73.33 $\uparrow$ 10.00	30.83 $\uparrow$ 0.83	46.66 $\uparrow$ 6.66	84.60 $\uparrow$ 6.80	48.23 $\uparrow$ 5.15	47.50
8B-Base	+ GRACE ( Zhang et al., 2025 )	40.83	66.66	29.16	43.33	75.00	44.11	49.46

Table 3: Experimental results on AceReason-1.1-SFT ( Chen et al., 2025b ) . We generate one response per source LLM, and select one response from these responses (select 10k responses from 40k data).

Method	AIME24	AIME25	MATH	OlymB.
Qwen3-4B-Base
$\max$ Casl	31.66	30.83	80.00	42.81
$\min$ Casl	29.16	28.33	77.40	39.70
Qwen3-8B-Base
$\max$ Casl	45.00	37.50	85.40	49.03
$\min$ Casl	41.66	36.66	79.60	42.94

Table 4: Comparison of the experimental results for Aslec-casl that selects the highest and lowest

s^{\mathrm{casl}}

Source	$\beta_{1}$	$\beta_{2}$	$\gamma$	$\epsilon$
QwQ-32B	0.077	0.919	-0.284	-0.001
Qwen3-32B	0.068	0.929	-0.529	0.007
DS-Qwen-32B	0.066	0.943	-0.226	0.001
gpt-oss-120b	0.068	0.938	-1.284	0.028
Overall	0.066	0.944	-0.680	-0.054

Table 5: Linear regression parameters of Eq. ( 5 ) fitted on data generated by different source LLMs on LIMO - v2 .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Relationship between response-level log probability and total response length.

Figure 6: Average log probability of tokens at different positions for four source LLMs.

Source	$\beta_{1}$	$\beta_{2}$	$\gamma$	$\gamma_{2}$	$\epsilon$
QwQ-32B	.028	.972	-0.062	1E-8	.004
Qwen3-32B	.018	.979	-0.043	3E-9	.007
DS-Qwen-32B	.014	.983	-0.158	4E-8	-.008
GPT-OSS-120B	.018	.987	-1.166	5E-9	.025

Appendix

Table 6: Linear regression parameters including overall response length

\gamma_{2}|\mathbf{o}_{i}|

Teacher	AIME24	AIME25	MATH	OlymB.
Qwen3-4B-Base
Aslec-casl	31.66	30.83	80.00	42.81
+ $\gamma_{2}\|\mathbf{o}_{i}\|$	30.83	30.83	78.60	42.20
Qwen3-8B-Base
Aslec-casl	45.00	37.50	85.40	49.03
+ $\gamma_{2}\|\mathbf{o}_{i}\|$	43.33	35.83	83.80	48.52

Appendix

Table 7: Comparison of experimental results with and without removing overall response length bias

\gamma_{2}|\mathbf{o}_{i}|

Figure 7: Step length distributions for selected and unselected data and relationship between step-level log probability and step length on AceReason - 1.1 - SFT .

Figure 8: Data selection bias and step length distributions under different splitting methods.

Target LLM	$\beta_{1}$	$\beta_{2}$	$\gamma$	$\epsilon$
Qwen3-4B-Instruct	0.075	0.952	-0.778	0.022
Qwen3-8B-Base	0.027	0.993	-0.660	0.025

Appendix

Table 8: Linear regression parameters of all SFT data on AceReason - 1.1 - SFT for the two target models.

Figure 9: Convergence analysis.

Qwen3-4B-Base	AIME25	MATH500
+ Uniform	24.16	73.20
+ High Difficulty	26.66	77.80
+ Low Difficulty	20.83	71.20
+ Aslec-casl	30.83	80.00

Appendix

Table 9: Results of more data selection methods.

Llama3-3B	AIME25	MATH500
+ GRACE	26.66	76.20
+ Aslec-casl	33.33	84.40

Appendix

Table 10: Results on Llama3-3B.

Large Reasoning Models

Large reasoning models are language models trained or prompted to solve difficult tasks through extended, structured computation.

See all 338 papers

Stabilizing Efficient Reasoning with Step-Level Advantage Selection
Apr 27, 2026Han Wang, Xiaodong Yu, Jialian Wu +4RL for Language Model Reasoning Efficient Language Model Reasoning
UNC Chapel Hill · Advanced Micro Devices, Inc.
ChainPrune: Evaluating and Reducing Redundancy in Long Chain-of-Thought Reasoning
Aug 22, 2026Weihang Pan, Zhengxu Yu, Yuxiang Zhang +5Efficient Language Model Reasoning Overthinking in Language Models
Zhejiang University · Alibaba Group
Unified Data Selection for LLM Reasoning
May 21, 2026Xiaoyuan Li, Yubo Ma, Chengpeng Li +6Training Data Selection LLM Training
University of Science and Technology of China · Alibaba Group · National University of Singapore

Apr 27, 2026J/K move · Enter open · S save

Stabilizing Efficient Reasoning with Step-Level Advantage Selection

Han Wang, Xiaodong Yu, Jialian Wu, Jiang Liu+3

UNC Chapel Hill · Advanced Micro Devices, Inc.

RL for Language Model Reasoning Efficient Language Model Reasoning Efficient Language Model Inference Fine-Grained Credit Assignment Large Reasoning Models

Open paper AskPDF arXiv

Large language models (LLMs) achieve strong reasoning performance by allocating substantial computation at inference time, often generating long and verbose reasoning traces. While recent work on efficient reasoning reduces this overhead through length-based rewards or pruning, many approaches are post-trained under a much shorter context window than base-model training, a factor whose effect has not been systematically isolated. We first show that short-context post-training alone, using standard GRPO without any length-aware objective, already induces substantial reasoning compression-but at the cost of increasingly unstable training dynamics and accuracy degradation. To address this, we propose Step-level Advantage Selection (SAS), which operates at the reasoning-step level and assigns a zero advantage to low-confidence steps in correct rollouts and to high-confidence steps in verifier-failed rollouts, where failures often arise from truncation or verifier issues rather than incorrect reasoning. Across diverse mathematical and general reasoning benchmarks, SAS improves average Pass@1 accuracy by 0.86 points over the strongest length-aware baseline while reducing average reasoning length by 16.3%, yielding a better accuracy-efficiency trade-off.

Abstract

Large Reasoning Models

Explore similar work

Stabilizing Efficient Reasoning with Step-Level Advantage Selection

ChainPrune: Evaluating and Reducing Redundancy in Long Chain-of-Thought Reasoning

Unified Data Selection for LLM Reasoning