Adaptive Stopping of Streamed Reasoning for Language Model Agents

JunHyeok Oh; Byung-Jun Lee

cs.CLOct 20, 2026

Adaptive Stopping of Streamed Reasoning for Language Model Agents

Authors: JunHyeok Oh, Byung-Jun Lee

Organizations: Korea University

Abstract

Reasoning-enabled language model agents can incur substantial computation before acting, although the reasoning needed varies across decisions. Existing stopping signals, such as confidence and intermediate-answer stability, can vary in effectiveness across reasoning models; signals obtained from hidden states or token probabilities also require internal access that may be unavailable or impractical. We introduce StreamStop, which learns model-specific stopping policies offline from prefix-level action agreement. A frozen prefix reader encodes available reasoning text for a lightweight stop head, allowing stopping without access to the reasoning model's internal signals. We construct and publicly release a dataset of agent reasoning trajectories with prefix-level action annotations. These annotations compare each prefix action with the full-reasoning reference action, enabling offline comparison of stopping rules and exact expected-return optimization of a discounted action-agreement objective. Conditioning the head on the discount factor allows the agreement-token trade-off to be adjusted at inference without retraining. In closed-loop evaluations on WebShop and ScienceWorld, StreamStop achieves higher observed success than most validation-selected fixed-budget settings while using fewer reasoning tokens than Full thinking. On WebShop with Qwen3.8-27B, it uses 42% fewer reasoning tokens per episode with similar observed task success.

Figures & tables

Figure 1: Prefix diagnostics on the WebShop validation set. (a) Agreement with the full-reasoning action for the prefix reader scoring valid candidates and the reasoning model answering from each prefix. (b) Cumulative distributions of sufficient-prefix and full-reasoning lengths.

Figure 2: StreamStop overview. (a) A frozen prefix reader and discount-conditioned stop head control the number of reasoning tokens used; the reasoning model generates the action from the selected prefix. (b) Prefix actions and the full-reasoning reference supply agreement labels for exact expected-return training of the stop head.

WebShop
Reasoning model	Episodes	Streams	Prefixes
Qwen3.8-27B-FP8	5,237	40,922	443,611
Nemotron-3-Nano-30B-A3B-FP8	3,437	20,021	374,499
ScienceWorld
Qwen3.8-27B-FP8	1,260	30,885	791,006
Gemini 3.1 Flash-Lite	1,260	31,991	487,700

Table 1: Collected dataset, including successful and unsuccessful episodes. Streams and Prefixes count nonempty reasoning generations and prefix-action annotations, respectively.

	WebShop						ScienceWorld
	Qwen3.8 27B-FP8			Nemotron-3-Nano 30B-A3B-FP8			Qwen3.8 27B-FP8			Gemini 3.1 Flash-Lite
Method	Succ. $\uparrow$	Tok/ep $\downarrow$	Tok/think $\downarrow$	Succ. $\uparrow$	Tok/ep $\downarrow$	Tok/think $\downarrow$	Succ. $\uparrow$	Tok/ep $\downarrow$	Tok/think $\downarrow$	Succ. $\uparrow$	Tok/ep $\downarrow$	Tok/think $\downarrow$
Full thinking	43.0 $\pm$ 2.2	5945 $\pm$ 160	713 $\pm$ 11	34.5 $\pm$ 3.5	5374 $\pm$ 220	871 $\pm$ 25	52.1 $\pm$ 4.8	45380 $\pm$ 2291	1656 $\pm$ 70	47.1 $\pm$ 2.8	10980 $\pm$ 371	359 $\pm$ 4
No thinking	13.2 $\pm$ 1.8	0 $\pm$ 0	0 $\pm$ 0	18.3 $\pm$ 1.5	0 $\pm$ 0	0 $\pm$ 0	32.0 $\pm$ 1.7	0 $\pm$ 0	0 $\pm$ 0	39.9 $\pm$ 2.5	0 $\pm$ 0	0 $\pm$ 0
Const-B 0.98	32.8 $\pm$ 2.1	1257 $\pm$ 25	144 $\pm$ 0.71	29.2 $\pm$ 4.0	1082 $\pm$ 29	247 $\pm$ 1	46.5 $\pm$ 3.3	8017 $\pm$ 328	266 $\pm$ 3	45.1 $\pm$ 2.2	6068 $\pm$ 199	191 $\pm$ 0.05
Const-B 0.99	35.0 $\pm$ 1.5	1519 $\pm$ 7	175 $\pm$ 1	28.2 $\pm$ 3.8	1343 $\pm$ 65	288 $\pm$ 2	46.8 $\pm$ 2.2	8583 $\pm$ 379	289 $\pm$ 4	45.1 $\pm$ 2.2	6068 $\pm$ 199	191 $\pm$ 0.05

Table 2: Test success rate (%) and reasoning tokens used per episode (Tok/ep) and decision attempt (Tok/think), reported as mean

\pm

standard deviation across evaluation seeds. Const-B γ and Const-M γ use nominal- and realized-budget matching on validation data. StreamStop γ uses discount factor

\gamma

. Gemini token counts use prefix-reader tokenizer units for the explicit plan. Appendix A gives matching rules, seed counts, and aggregation details.

Figure 3: Latency and reader timing for local reasoning models. (a–c) Test success versus latency per decision; bars show 95% Wilson intervals. (d) Reader-update times and reasoning-chunk intervals, pooled across configurations; boxes show interquartile ranges, center marks medians, and whiskers the 10th–90th percentiles.

Figure 4: Action agreement on the validation set versus mean reasoning tokens used per decision across four environment–model configurations. Curves vary the stopping settings. Gemini uses plan tokens in prefix-reader tokenizer units.

Figure 5: Selected WebShop–Qwen3.8 validation examples. Shading shows stop probability (log scale); markers show method cutoffs; checks/crosses show agreement/disagreement with the full-reasoning action.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value	Search candidates
Prefix reader	Qwen3-1.7B, FP32	–
Hidden units (main / discount)	128 / 32, tanh	Linear; 64; 128; 256; $128\times 2$ (main)
Fourier order	4	4; 8
Fourier period	2	–
Feature scale	4	1; 4
Optimizer	AdamW, CPU	–

Appendix

Table 3: Prefix reader and stop-head settings. Search candidates are evaluated on WebShop–Qwen3.8 validation decisions; – denotes a fixed setting.

Configuration	Const-B budgets	Const-M budgets	EAT-proxy $(\alpha,m,\delta)$
WebShop–Qwen3.8	$(160,208,272)$	$(192,352,928)$	$(0.3,1,1.69\times 10^{-15})$
WebShop–Nemotron	$(256,304,336)$	$(288,352,416)$	$(0.1,3,3.27\times 10^{-13})$
ScienceWorld–Qwen3.8	$(304,336,352)$	$(512,576,672)$	$(0.3,3,1.69\times 10^{-5})$
ScienceWorld–Gemini	$(192,192,208)$	$(192,208,208)$	$(0.3,3,3.67\times 10^{-6})$

Appendix

Table 4: Baseline settings selected on validation data. Fixed budgets are ordered by

\gamma=(0.98,0.99,0.995)

; EAT-proxy parameters control smoothing, minimum observation count, and variance threshold.

Configuration	$T$	Full inputs	Metadata-only	Context-only	Raw-prefix	Const- $k$
WebShop–Qwen3.8	236	$\mathbf{98.0}\,{\scriptscriptstyle\pm 0.2}$	$91.7\,{\scriptscriptstyle\pm 0.0}$	$92.5\,{\scriptscriptstyle\pm 0.2}$	$96.2\,{\scriptscriptstyle\pm 0.4}$	95.8
WebShop–Nemotron	285	$89.2\,{\scriptscriptstyle\pm 0.5}$	$83.5\,{\scriptscriptstyle\pm 0.0}$	$83.5\,{\scriptscriptstyle\pm 0.0}$	$\mathbf{90.9}\,{\scriptscriptstyle\pm 0.4}$	77.5
ScienceWorld–Qwen3.8	369	$\mathbf{91.6}\,{\scriptscriptstyle\pm 0.8}$	$90.8\,{\scriptscriptstyle\pm 0.3}$	$91.0\,{\scriptscriptstyle\pm 0.9}$	$90.3\,{\scriptscriptstyle\pm 0.9}$	90.9
ScienceWorld–Gemini	229	$\mathbf{97.8}\,{\scriptscriptstyle\pm 0.7}$	$92.4\,{\scriptscriptstyle\pm 0.0}$	$93.5\,{\scriptscriptstyle\pm 1.1}$	$94.2\,{\scriptscriptstyle\pm 0.6}$	89.9
Mean	–	94.1	89.6	90.1	92.9	–

Appendix

Table 5: Interpolated validation action agreement (%) at the Full-input head’s mean token budget

T

for

\gamma=0.99

. Values are mean

\pm

standard deviation across training seeds;

T

is rounded for display.

Figure 6: Validation agreement–token trade-offs for stop-head input variants. Points average three training seeds across discounts; all variants receive progress metadata. Gemini uses explicit-plan prefixes; vertical-axis ranges vary by panel.

Early Stopping for Language Model Reasoning

See all 21 papers

Jun 13, 2026

Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models

By incorporating test-time compute scaling, large reasoning models (LRMs) can solve complex problems through explicit chain-of-thought (CoT) reasoning processes. However, they often suffer from overthinking, resulting in redundant token outputs and degraded accuracy. Current methods to mitigate this issue remain limited: training-based approaches require substantial computational resources, while training-free methods rely on well-crafted prompts or unreliable confidence signals. In this work, we investigate early stopping from the perspective of attention distributions and propose a simple method, ASAG, which infers the model's reasoning state and adaptively adjusts the generation strategy. The proposed framework is training-free and plug-and-play, enabling seamless integration into existing LRMs. Extensive experiments on nine benchmarks demonstrate consistent improvements across mainstream LRMs with varying parameter scales, including the DeepSeek-R1-Distill and Qwen3 series. Specifically, ASAG improves average accuracy by 3.2% while reducing the number of generated tokens by nearly 40% across all reasoning tasks on Qwen3-8B.

Jiakai Li, Ke Qin, Rongzheng Wang +4

University of Electronic Science and Technology of China, Chengdu, China · Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province, Chengdu, China

Early Stopping for Language Model Reasoning Efficient Language Model Inference LLM Reasoning Large Reasoning Models

Ask PDF

Oct 18, 2026

Calibration-Free Early Stopping for Chain-of-Thought Reasoning via E-processes

Reasoning large language models (LLMs) benefit from test-time scaling, motivating adaptive policies that spend tokens when they help and stop early otherwise. Existing rules halt when a confidence or stability signal crosses a threshold, but such thresholds rarely transfer across models and datasets, and validation-calibrated rules lose their guarantee under shifts in test-time horizon or signal dynamics. We reframe adaptive stopping as sequential evidence accumulation along the reasoning trajectory rather than threshold calibration on held-out data. Our framework runs two e-processes on a shared per-chunk signal: a commit stream tracks the signal's position relative to its slow trend, and an abstain stream tracks its signed velocity, making the two anti-correlated by construction. Under signal-level nulls on the per-chunk increments, Ville's inequality yields an anytime-valid signal-level false-alarm bound at user-specified significance levels, with no held-out calibration. Across various model--dataset combinations, the framework matches or exceeds calibration-based stopping on the token--accuracy Pareto front using zero calibration data, and the same significance level transfers unchanged across cells where calibration becomes infeasible.

Dong-Jae Lee, Gyojin Han, Junmo Kim

KAIST

Early Stopping for Language Model Reasoning Adaptive Reasoning in Language Models Efficient Language Model Reasoning Sequential Hypothesis Testing

Ask PDF

Sep 30, 2026

Settle: Learning When to Stop Reasoning

Reasoning models often continue generating after their answers have settled. Settle learns when to stop from answer stability in completed traces. It trains the existing end-of-reasoning token while keeping other predictions close to the base model, and requires only ordinary decoding at inference. On MATH-500 with Qwen3-4B, Settle reduces token count by 40% with a 0.5-percentage-point decrease in accuracy. It gains 6.16 percentage points over supervised fine-tuning on the same traces shortened at their first stable answer, at nearly identical token counts. Its stopping score predicts whether a correct answer will remain correct. Settle extends the accuracy-token-count Pareto frontier of the evaluated stopping methods.

Ryan Brown, Zihao Fu, Chris Russell

Oxford Internet Institute, University of Oxford · Department of Linguistics and Modern Languages, The Chinese University of Hong Kong

Early Stopping for Language Model Reasoning Efficient Language Model Reasoning Overthinking in Language Models

Ask PDF

Abstract

Early Stopping for Language Model Reasoning

Explore similar work

Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models

Calibration-Free Early Stopping for Chain-of-Thought Reasoning via E-processes

Settle: Learning When to Stop Reasoning