cs.CLOct 15, 2026

ARBOR: Locating Reasoning Failures in Long Chains of Thought through Reasoning State Tracking

Authors: Huihan Li, Siyuan Wang, Naga Vamsi Ramana Dinavahi, Xiaohui Liu, Xiang Ren

Organizations: University of Southern California · The Chinese University of Hong Kong · New York University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large reasoning models solve complex problems through long chains of thought, in which prior work has documented failure modes such as overthinking, underthinking, and unsupported deductions. Existing methods detect these failures from local signals: textual similarity to nearby steps, switch tokens, or an LLM's judgment of a step's correctness. However, what a step contributes to the solution depends on the state that earlier steps have established, so identifying failures requires a global view that tracks that state across the trajectory. Therefore, we propose ARBOR, a reasoning-failure attribution framework that represents each trajectory as a tree of reasoning states, built as each step creates, verifies, prunes, or revisits a state, and attributes each failure to a relation between a step and the state preceding it. Using ARBOR, we refine the three failure modes into distinct subtypes that harm reasoning through different mechanisms and at different costs. We find that the tree state ARBOR maintains before a step separates failure steps from the at-risk steps at which the failure did not occur. We discuss how ARBOR integrates with existing mitigations at inference time and in training data. Code, data and interactive examples are available at https://inklab.usc.edu/ARBOR.

Figures & tables

Figure 1: How a step relates to the states already established defines a reasoning failure. (a) An excerpt of a Qwen3-8B trajectory on a ZebraLogic puzzle; numbers are step indices, and colored steps are labeled by function. (b) Its tree of reasoning states: nodes are intermediate states, tagged with the steps that created or revisited them, edges link each state to the states derived from it, and the case split at step 66 opens two branches. Steps 87–88 are a repetition : they re-derive the state of step 68 in new words. Step 86 is underthinking : it abandons Case 2 while that branch is still viable. Step 90 is an unsupported deduction : it contradicts the Case 1 assumption. Step 113 is a justified switch : it follows the pruning of Case 1 at step 112.
Figure 2: Local signals miss or misattribute the failures that the reasoning state identifies. Columns are steps of the trajectory in Figure 1 . For each failure mode, local signals (gray, dotted) are compared with Arbor (colored, solid), whose arrow points to the earlier step whose state determines the label. Repetition: step 88 shares no nn -gram with step 68, and embedding similarity links it to the clue recap at step 53 rather than to the state it re-derives. Underthinking: a switch token fires at both step 86 and step 113, but only step 86 leaves a viable branch. Unsupported deduction: the process reward model misses step 90, which violates the Case 1 assumption of step 66, and flags the correct retraction at step 112 instead.
Figure 3: Arbor ’s three-stage framework. An LLM parses each step into a canonical operation, then a deterministic engine replays it against the accumulated reasoning state, and deterministic rules map the resulting state relations to functional labels.
RowDefinition: the model …
Repetition
R1 Re-Derivationderives again a result it has already derived, after deriving or verifying others.
R2 Derive-after-Prunederives from a hypothesis it has already rejected.
R3 Redundant Re-Verificationverifies a result again although nothing it depends on has changed.
R4 Subgoal Churnposts a subgoal again after working on others, without progress on it.
Underthinking
Table 1: The failure rows. Each row is one relation between a step and the results the model has established before it; § 2.3 gives the predicate on the reasoning-state tree. A row is undefined, rather than approximated, on a task whose structure has no state for it (Appendix H ).
OperationPayloadEffect on the tree
Derivea result ff , or alternatives {fi}\{f_{i}\}adds a child holding ff under the node ff depends on; one child per fif_{i}
Assumea hypothesis hhadds a child holding hh under the current leaf
Verifya result ff , verdict oorecords oo and s(u)⊨fs(u)\models f on the node uu holding ff
Prunea result or hypothesis ffcloses the node holding ff and its subtree
Confirman answer aacommits the node holding aa ; records accept(a)\mathrm{accept}(a)
Restatea result ffrecords the restatement on the node holding ff
Table 2: The seven operations: the parser assigns each step one operation and its payload, and the engine applies the effect. A result is a fact, value, action or subgoal.
Figure 4: Three tasks, three shapes of reasoning, three failure patterns. Schematic Arbor trees of a typical failing trajectory for each task (statistics in Appendix H.1 ).
CategoryMethodPrec.Rec.
RepetitionArbor64%37%
n-gram49%31%
similarity62%45%
UnderthinkingArbor32%71%
switch tokens27%62%
Post-Answer AttemptArbor77%87%
Table 3: Expert review confirms Arbor and rejects local signals. Precision: share of a method’s flagged steps the annotator confirms as the stated failure. Recall: share of the confirmed steps of the failure mode (all methods’ steps pooled) that the method also flags. For underthinking we also showed the annotators the lineage Arbor tracked, since the text rarely settles whether the abandoned branch was still viable (Appendix G.2 ).
Figure 5: Arbor ’s subtype counts predict the outcome better than all local signals combined. Gain in AUROC for separating incorrect from correct trajectories over a baseline of trajectory length and task (solid line, 0.86), pooled over the three tasks. Error bars: 95% confidence intervals.
Figure 6: The evidence behind § 5 , per task (Qwen3-8B; exact values in Tables 21 to 25 ). In (d), the label above each bar is the share of failing trajectories that held a correct solution; in (h), PlanBench has no branch lifecycle.
Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

TaskModelCorrectTruncatedInvalid
ZebraLogicQwen3-1.7B49.039.60.0
Qwen3-8B70.227.00.0
Qwen3-32B75.422.90.0
24 GameQwen3-1.7B80.86.412.1
Qwen3-8B78.07.010.8
Qwen3-32B78.85.714.8
Appendix
Table 6: Outcome shares (%) per task and model size over the first three samples of every question (ZebraLogic 3,000, 24 Game 4,086, PlanBench 4,992 trajectories per size). PlanBench “correct” is plan validity; “invalid” is no plan emitted.
Figure 7: Subtype rates by model size. For each task and subtype, the number of events per 100 steps (log scale) at 1.7B, 8B and 32B, left to right within each group, with 95% intervals over problems, on the same 86, 88 and 80 questions per task at every size. A group whose three dots descend is a subtype that becomes rarer with size; most groups are flat. Derive-after-Prune (R2, shaded) is the only subtype that falls at both size increases. Subtypes a task does not define are absent from its panel.
Table 8: Prompt Template for 24 Game
Table 9: Prompt Template for Zebra Logic
Table 10: Prompt Template for PlanBench (the benchmark’s one-shot query for a Blocksworld instance, followed by our instruction line)
Figure 8: The annotation interface. One card as the annotators saw it: the flagged step in its surrounding steps, the instruction stating the failure to judge, and the response options including “undecided”.
Figure 9: What the local measures flag.
 Median tokens opened / share of the trajectory (%)Pooled share by outcome (%)
SubtypeZebraLogic24 GamePlanBenchPooledcorrectincorrecttruncated
Repetition
R1 Re-Derivation17 / 0.315 / 0.416 / 0.216 / 0.30.40.30.2
R2 Derive-after-Prune25 / 0.417 / 0.4220 / 2.720 / 0.40.50.50.3
R3 Redundant Re-Verification11 / 0.224 / 0.812 / 0.216 / 0.30.60.20.2
R4 Subgoal Churn–14 / 0.317 / 0.215 / 0.30.40.20.2
Appendix
Table 19: Extended cost: the tokens an event opens (Qwen3-8B). For each event, the region it opens is, for Branch and Subgoal Abandonment, the steps from the event until the model returns to the left branch, else to the end; for Post-Answer Attempt, until the commit; for Premature Finalization, Infeasible Derivation and False Verdict, every later step resolved to the event’s node or a descendant; for Derive-after-Prune, the later steps in the pruned subtree; for Re-Derivation, Redundant Re-Verification and Subgoal Churn, the steps of the burst the event starts; and for False Rejection, the steps until the rejected subtree is next entered, else to the end. –: undefined on the task.
SubtypeStateLabelsPos.Prev. op
24 Game
R1 Re-Derivation0.730.700.500.69
R2 Derive-after-Prune †0.590.510.500.57
R3 Redundant Re-Verification0.830.810.580.77
R4 Subgoal Churn0.770.640.510.60
R5 Branch Abandonment0.910.810.560.63
Appendix
Table 27: The state before a step separates a failure from the steps where it could have happened. Held-out AUROC of a classifier on the tree state over the preceding steps (State) against each subtype’s matched opportunity steps, and the same classifier restricted to operation labels (Labels), to position in the trajectory (Pos.), and to the previous operation (Prev. op). Bold: state above the previous-operation baseline by more than 0.04; † : within 0.04, inconclusive. –: no label-only feature survived screening.
SubtypeStateLabelsPos.Prev. op
Pooled
R1 Re-Derivation †0.570.500.500.60
R2 Derive-after-Prune0.710.510.500.56
R3 Redundant Re-Verification †0.740.730.580.70
R9 Infeasible Derivation0.730.530.530.54
Appendix
Table 28: Pooled cells (the three tasks together, with task indicators as features), as in Table 27 .
PairTaskJaccardStatePos.Prev. op
R1 vs R2ZebraLogic0.000.630.540.57
R1 vs R224 Game0.000.650.540.62
R1 vs R2Pooled0.140.830.600.60
R1 vs R3ZebraLogic0.000.840.740.69
R1 vs R324 Game0.000.880.780.77
R1 vs R3Pooled0.130.810.810.72
Appendix
Table 29: Sibling subtypes arise after different states. Jaccard: overlap of the two subtypes’ stable precursor sets. State: AUROC of a classifier on the union of the two sets at telling one sibling’s occurrences from the other’s, with the two baselines.
FeatureCondition the stand-in readsMitigation it gates, and what the gate changes
Branch contentthe constraints fixed on the branch within the last ten stepsa thinking intervention ( Wu et al., 2025 ) , a chain-of-verification ( Dhuliawala et al., 2023 ) or a process reward ( Lightman et al., 2024 ; Wang et al., 2024 ) : the inserted text or the verification question names those constraints, so the check lands on the constraint an Infeasible Derivation violates
Frontier statuswhether the current frontier leaf has a verdictthe thought-switch penalty ( Wang et al., 2025 ) : applied only while the frontier leaf has no verdict, so it stops Branch Abandonment and spares the switches that leave a ruled-out branch, which an unconditional penalty suppresses as well
Frontier statuswhether a checked complete solution standsbudget forcing ( Muennighoff et al., 2025 ) and confidence-gated early exit ( Yang et al., 2025 ; Zhang et al., 2025a ) : generation stops when a checked complete solution stands rather than at a length or confidence threshold, which ends the Post-Answer Attempt without cutting off unfinished search
Recent operationsthe branch’s last operation and its pruned statussteering vectors for verification and backtracking ( Venhoff et al., 2025 ; Ward et al., 2025 ) : the verification vector is damped right after a check, which removes Redundant Re-Verification, and the backtracking vector is damped on a pruned branch, which removes Derive-after-Prune
Offlinethe return rows and the dead-end subtrees opened at a departurespans to remove when compressing trajectories for fine-tuning ( Xia et al., 2025 ; Zhao et al., 2025 ; Munkhbat et al., 2025 ) , replacing token-importance or length criteria
Offlinethe per-step failure rowsa structure-aware penalty for length-harmonizing reinforcement learning ( Luo et al., 2025a ; Chen et al., 2025b ) that charges a repeated operation rather than every token past the answer
Appendix
Table 30: The two routes, feature by feature. Route 1 (first four rows): a stand-in trained on Arbor ’s labels reads the condition that precedes the failure and gates an existing inference-time mitigation on it. Route 2 (last three rows): Arbor ’s labels, computed offline, replace length and token criteria in training-side methods.
More in this topic

Reasoning Trace Analysis

Reasoning Trace Analysis studies the structure, reliability, and causal role of language models’ reasoning traces.

See all 125 papers

Explore similar work

CardsList
Oct 15, 2026

Effectiveness of Hidden-State Probes for Localizing Reasoning Failures

Verifying the final answer of a long reasoning trace may show that the trace is wrong but it does not distinguish three cases. The problem may lie beyond the model's capability, another attempt would succeed, or one step in an otherwise recoverable trace made the trace fail. Standard hidden-state probes are trained on final-answer correctness, a label that conflates these cases. Monte Carlo rollouts distinguish the cases and locate the failing step at additional inference cost. We test the hidden states of 4B and 8B reasoning models for the failure type and the failing step and compare each probe with the strongest baseline that uses no hidden states. On incorrect 8B traces, the confidence probe, a calibrated correctness probe at a midpoint cutoff, misses 81% of cases in which a single step makes the trace unrecoverable and 47% of cases in which another attempt would succeed. A probe with a different training design over the same states reverses the ordering of these two kinds of failure, so either ordering can be read from those states. At both scales, no single tested probe locates the failing step more accurately than a rule based only on trace length, and at 4B a probe trained to predict that step fails to outperform this rule in any of 10 runs on identical traces. The confidence probe therefore indicates that an answer is wrong, but its failure-type ordering depends on its training design.

Shubham Subhnil, Yannis Belkhiter, Erik Miehling +2

Trinity College Dublin, Ireland · IBM Research, Dublin, Ireland · Umeå University, Sweden

Hidden-State ProbingReasoning Trace AnalysisConfidence Estimation in Language Models
AskPDF
Jun 1, 2026

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models

Large Reasoning Models (LRMs) improve performance by generating explicit intermediate reasoning traces through increased test-time compute, yet the assumption that longer reasoning is consistently beneficial remains under-examined. While recent evidence shows that additional reasoning can lead models to overthink, we ask: "Once a model has reached the correct answer, does further reasoning refine the solution, or deviate from it?" To study the dynamics after correctness, we introduce a prefix-level trajectory evaluation protocol grounded in reasoning sufficiency, defining the minimum reasoning budget required for a model to first generate the correct answer. This allows us to disentangle verbose overthinking, where additional reasoning is redundant but harmless, from harmful overthinking, where continued reasoning destabilizes an already-correct trajectory. Starting from multimodal benchmarks, we find that many instances considered reasoning-intensive require surprisingly little reasoning. Moreover, stopping at the first correct prefix improves accuracy over standard reasoning up to 21%, revealing that current models are limited not only by their ability to reason, but also by their inability to stop at the right time. Furthermore, while common efficiency strategies like early stopping substantially reduce verbose overthinking (up to 50%), they fail to mitigate harmful overthinking. Failure analysis reveals that correctness deviations are mainly driven by logical drift and visual reinterpretation. Finally, we show that our findings generalize to language-only reasoning benchmarks, highlighting harmful overthinking as a broader reliability risk. Code available at https://simonecaldarella.github.io/thinking-past-the-answer.

Simone Caldarella, Davide Talon, Rahaf Aljundi +2

University of Trento · 3Fondazione Bruno Kessler · 2Toyota Motor Europe

LLM EvaluationOverthinking in Language ModelsReasoning EvaluationLLM ReliabilityLarge Reasoning ModelsReasoning Trace Analysis
AskPDF
Sep 29, 2025

Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models

Current answering paradigms for Large Reasoning Models (LRMs) often fail to account for the fact that some questions may lie beyond the model's operational capability boundary, leading to long but unproductive reasoning. In this paper, we study whether LRMs expose early signals predictive of such cases, and whether these signals can be used to mitigate unproductive reasoning. In black-box settings, we find that reasoning expressions contain failure-predictive signals. In white-box settings, we show that the hidden states of the last input token contain information that is predictive of whether a question will not be solved correctly under our evaluation setup. Building on these observations, we propose two test-time monitoring strategies: reasoning expression monitoring and hidden states monitoring, that reduce token usage by 62.7-93.6%, substantially improving efficiency and reliability while largely preserving accuracy.

Qingjie Zhang, Yujia Fu, Yang Wang +5

Tsinghua University · 2Ant Group

Efficient Language Model ReasoningEfficient Language Model InferenceLarge Reasoning Models
AskPDF