Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models

Zhizhuo Kou; Sizai Hou; Long Chen; Jian Yang; Junyu Luo; Jingshu Peng; Wenkai Zhang; Sirui Han

cs.LGOct 22, 2026

Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models

Authors: Zhizhuo Kou, Sizai Hou, Long Chen, Jian Yang, Junyu Luo, Jingshu Peng, Wenkai Zhang, Sirui Han,

Organizations: The Hong Kong University of Science and Technology · Huawei Noah’s Ark Lab · Peking University

Abstract

Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data. We trace the word through open models' training histories, reading it with label-free indices of frequency, position, form and usage and an injected-error test, all computed from sampled outputs. We find that zero-RL almost never coins the marker: on bases that lack the word, RL raises MATH-500 accuracy substantially while {wait} stays near absent. It arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it. Where the base already uses it, both released RL-Zero recipes (OLMo-3, MiMo) multiply it, whereas KL-free GRPO from the same base does not; OLMo-3's recipe wears down its form and makes it a more sensitive but more liberal alarm; RL after supervised fine-tuning barely changes it, and our GRPO recipe erases a marker seeded by fine-tuning. Finally, we show that marker frequency alone does not indicate RL-induced reasoning and caution against LLM-as-a-judge revision metrics, as automated judges can reverse human-labelled trends.

Figures & tables

Figure 1: RL rarely coins wait : the marker enters with mid-training or fine-tuning on traces.

Figure 2: Our methodology extracts a marker’s evolutionary history from its occurrences, one checkpoint at a time. The pipeline processes ❶ lineages and sampled traces into ❷ matched occurrences, from which we construct ❸ label-free indices and validate the ❹ injected-error alarm test, with statistics at every checkpoint (bottom).

Figure 3: OLMo-3 RL-Zero multiplies wait and erodes its form; RL after fine-tuning changes its rate little and restores its comma. (top) OLMo-3 RL-Zero; (bottom) OLMo-3 Think; (a)–(e) rate, sentence-initial share, Wait, share, chaining, sentence length; hollow: base; bands: 95% cluster-bootstrap CIs.

Figure 4: Under RL-Zero, revision markers become a more sensitive but more liberal alarm. Sensitivity

d^{\prime}

and criterion

c

of any revision marker after clean versus error-injected prefixes, per checkpoint, with 95% cluster-bootstrap bands.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Series (Hugging Face)	Base ( $t_{0}$ )	Training	Checkpoints used	Role
S1	allenai/Olmo-3-7B-RL-Zero-Math	Olmo-3-1025-7B	RL from the base	base, 100, 200, 400, 700, 1000, 1400, 1900, final	primary
S1-pre	allenai/Olmo-3-1025-7B (branches)	–	pretraining, mid-training	stage 1: 141k, 707k, 1.41M; stage 2: 47.7k; stage 3: 6k, 11.9k	source
S2	allenai/Olmo-3-7B-Think	Olmo-3-7B-Think-DPO	RL after SFT and DPO	DPO, 25, 100, 250, 500, 750, 1000, 1375	primary
S2-ctl	allenai/Olmo-3-7B-Think-SFT	Olmo-3-1025-7B	SFT only	5k, 15k, 25k, 35k, 43k	lineage control
S9	allenai/Olmo-3.1-32B-Think	Olmo-3-32B-Think-DPO	RL after SFT and DPO	DPO, 100, 300, 700, 1200, 1800, 2300	32B replication
S8	Qwen2.5-32B $\rightarrow$ ORZ / DAPO / QwQ / R1-Distill	Qwen2.5-32B	RL / distillation	endpoints	same-base comparison

Appendix

Table 5: Checkpoint series. Released trajectories (top) and our own runs (bottom). Every released endpoint is listed in Table 13 ; licences are in Section A.2 . openPangu-v2.1-Pro (API) in Table 4 : vendor API queried 2026-10-13, weights not public, thinking cannot be disabled via the API; the API is rate-limited, so

n=745

samples over 190 of 200 problems; rate per 1k output tokens (thinking and answer), 0.75 on the 182 problems with all four samples against 0.77 on all.

Licence	Models, datasets and tools
Apache-2.0	OLMo-2 and OLMo-3 (all series, branches and Think models); Qwen2.5, Qwen2.5-Math, Qwen3, Qwen3.5, Qwen3.8 and QwQ models; SmolLM3; Granite 3.3; Mistral-7B-v0.1 and Mistral-Small-24B; Yi-1.5; OpenThinker-7B; Bespoke-Stratos-7B; Light-R1-14B-DS; DAPO-Qwen-32B; SimpleRL-Zoo (all but the 14B release); Oat-Zero; General-Reasoner; One-Shot-RLVR; gpt-oss; AIME 2024 and 2025; vLLM; verl; Math-Verify
MIT	Phi-4 family; DeepSeek-R1-Distill (Qwen and Llama); DeepScaleR-1.5B; Open-Reasoner-Zero; MiMo-7B; GLM-4 and GLM-Z1; GSM8K; MMLU-Pro; MATH (training split); MATH-500 (its card states no licence; both of its sources are MIT)
Llama 3.1 Community License	Llama-3.1-Tülu-3 8B models
NVIDIA Open Model License	AceReason-Nemotron-7B
CC-BY-4.0	OpenReasoning-Nemotron-7B; reasoning-cues rollouts
Qwen License Agreement	Qwen2.5-Math-PRM-7B

Appendix

Table 6: Licences of all artefacts. One row per licence.per-repository identifiers and checks are at https://anonymous.4open.science/r/wait-provenance/ .

Figure 5: All 120 endpoints of the provenance map. One dot per model after training, adding instruct models without trace training (light squares), pretrained bases not shown elsewhere (MiMo-7B-Base and SmolLM3-3B-Base already write wait ) and gpt-oss-120b at three reasoning efforts (training undisclosed).

Figure 6: Across major model families, wait follows the training route. Rate before (

\circ

) and after training, 95% CI; shape marks the route, colour the family (Qwen2.5, Qwen3, Qwen3.5/3.8, MiMo, DeepSeek/Phi, openPangu).

Figure 7: In OLMo-3, wait enters with mid-training; Qwen2.5-32B’s descendants use it only after trace training. (a) Rate with 95% bands and comma share across OLMo-3 training stages and RL-Zero steps. (b) Qwen2.5-32B and its descendants: rate with 95% CIs, and accuracy.

Parameter (change)	Measurement on checkpoints	Hyp.
Frequency of use (context)	rate of wait and its paradigm per 1k tokens and per 100 sentences; known to rise ( Guo et al., 2025 )	–
Syntagmatic variability ( $\downarrow$ , fixation)	sentence-initial share $\mathrm{SI}$ , canonical-form share $\mathrm{ORTH}$ , position entropy $H_{\mathrm{pos}}$ of wait	Section D.2
Bondedness ( $\uparrow$ ), scope ( $\downarrow$ )	next-word entropy $H_{\mathrm{next}}$ ; tokens from marker to next action	Section D.2
Integrity ( $\downarrow$ , bleaching)	hit and false-alarm rates, $d^{\prime}$ and $c$ on prefixes with vs. without an injected error; usage-embedding change (APD, PRT, JSD); substitutability of synonyms	Section D.2
Paradigmatic variability ( $\downarrow$ ), paradigmaticity ( $\uparrow$ )	entropy $H_{\mathrm{para}}$ and effective size $N_{\mathrm{eff}}$ of the forms opening revisions; policy-side entropy $H_{\mathrm{sub}}$ ; obligatoriness $\mathrm{OBL}$	Section D.2
Procedural function (operator)	risk difference and odds ratio of revising a correct step after inserted Wait, vs. Next, and synonyms, late vs. base policy	Section D.2

Appendix

Table 11: Each grammaticalization parameter becomes a quantity measured on RL checkpoints. Parameters follow Lehmann (2015) ; arrows give the direction predicted under grammaticalization. Definitions are in Section 3 .

Prediction	Pre-registered test	Result	Verdict
H1 Fixation	sentence-initial and Wait, shares rise with step ( $\rho\geq 0.5$ on 2 of 3 series)	S1: SI 89.6 $\rightarrow$ 89.4% ( $\rho=0.55$ ); Wait, 88.9 $\rightarrow$ 37.5% ( $\rho=-0.33$ ); S2: Wait, $\rho=0.88$	falsified: position inherited, form erodes (S1) or is restored (S2)
H2 Bleaching	false alarms rise, $d^{\prime}$ does not (falsified if $\Delta d^{\prime}$ lower bound $>0.25$ )	S1: $d^{\prime}$ 0.57 $\rightarrow$ 1.09 [0.97, 1.24]; $c$ 0.11 $\rightarrow$ $-$ 0.24; FA 34.7 $\rightarrow$ 37.8%	falsified : a more sensitive, more liberal alarm
H3 Paradigm loss, obligatoriness	paradigm entropy $\rho\leq-0.5$ ; $\geq 60\%$ of revisions carry a marker	$\rho(H_{\mathrm{para}})$ : S1 $-$ 0.9, S2 $-$ 0.48, 32B $-$ 0.75; obligatoriness at the end of S1 46.1%	paradigm narrows on S1 only; obligatoriness falsified
H4 Operator	Wait, $-$ Next, $\geq 10$ pts, OR $\geq 2$ , $\geq 1.5\times$ the base model; same trend under both prefix sources	judge: 15.8 pts, $R_{\mathrm{amp}}=2.36$ ; second judge 5.4 pts; label-free value change $-$ 2.3 [ $-$ 5.7, 0.8], but its detector fails its check (precision 10.0%)	inconclusive: judge result exploratory (human $\kappa=0.3$ ), label-free outcome invalid
H5 Drivers	KL and length pressure set the speed (controlled GRPO)	controlled runs from a marker-rich base not run	untested
H6 Monitors	literal readers lose precision with RL; a calibrated monitor does not	not run	untested

Appendix

Table 12: Of six pre-registered hypotheses, three fail, one is unsupported and two were not run; further registered tests fail too. Tests and thresholds as fixed before the main run ( Section D.3 ). S1: OLMo-3 RL-Zero; S2: OLMo-3 Think; 32B: OLMo-3.1 32B Think; SI and Wait, shares in %; FA: false-alarm rate.

Figure 8: By the judges’ labels, an inserted Wait, pushes every checkpoint toward revision, the base included. Judged probability of revising a correct step per inserted form along (a) OLMo-3 RL-Zero and (b) OLMo-3 7B Think, with 95% cluster-bootstrap bands. Exploratory: the judge’s sensitivity against the human audit is low ( Section D.3 ).

RL for Language Model Reasoning

Reinforcement learning for language model reasoning uses reward-driven training to improve multi-step problem solving, tool use, and reliable answers.

See all 1,103 papers

Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently
Jun 22, 2026Stanley Wei, Juno KimRL for Language Model Reasoning Efficient Language Model Reasoning
1Princeton University · University of California, Berkeley.
Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning
May 7, 2026Ömer Faruk Akgül, Rajgopal Kannan, Willie Neiswanger +1RL for Language Model Reasoning Efficient Language Model Reasoning
University of Southern California · 2DEVCOM ARL
What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make
Oct 17, 2026Siyu Zhang, Yuquan Lu, Yoshua Y. Li +7RL for Language Model Reasoning LLM Reasoning
University of California, San Diego · Meituan, LongCat Team · The Chinese University of Hong Kong, Shenzhen

Jun 22, 2026J/K move · Enter open · S save

Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently

Stanley Wei, Juno Kim

1Princeton University · University of California, Berkeley.

RL for Language Model Reasoning Efficient Language Model Reasoning Reinforcement Learning with Verifiable Rewards RL Fine-Tuning LLM Reasoning

Open paper AskPDF arXiv

Recent advances in large language models (LLMs) have demonstrated that reinforcement fine-tuning of pretrained base models can lead to significant gains in reasoning performance at inference time. In this work, we theoretically analyze why reinforcement fine-tuning induces better reasoning ability than purely supervised fine-tuning (SFT) methods. We model chain-of-thought (CoT) reasoning as a pathfinding problem on graphs and compare the popular method of reinforcement learning with verifiable rewards (RLVR) against traditional SFT. We prove that SFT, when trained on golden shortest paths without negative examples, fails to learn how to efficiently backtrack. In contrast, an RLVR-trained model can learn how to efficiently backtrack from dead ends using only outcome reward. This leads to an exponential separation in inference-time compute between the two methods, and demonstrates that RLVR leads the model to learn the location of difficult decisions in a reasoning chain, ultimately allowing for better allocation of inference-time compute. Finally, we show that the reasoning traces of an RLVR model can be distilled to train a base model to backtrack efficiently as well.

Abstract

RL for Language Model Reasoning

Explore similar work

Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently

Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning

What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make