cs.LGAug 18, 2026

Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth

Authors: Ivan Viakhirev, Kirill Borodin, Amirah Almutairi, Serguei Barannikov, Maxim Abramov, Grach Mkrtchian

Organizations: FRC RAS, Saint-Petersburg, Russia · ITMO, Saint-Petersburg, Russia · lab260, Moscow, Russia · BitmanagerAI, Dubai, UAE · MTUCI, Moscow, Russia · KAU, Jeddah, Saudi Arabia · CNRS, IMJ-PRG, Paris, France

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recurrent-depth reasoners aim to solve harder problems by iterating their update longer at test time, but additional iterations can improve, preserve, or degrade an answer. We show that a measurable property of the trained operator, its finite-time dynamical regime (estimated as settling, marginal, or drifting), indicates which of these occurs. We give a sufficient condition for depth-safety: once an operator's per-step displacement is small relative to the decoder margin, the decoded answer cannot change under further iterations. Empirically, on algorithmic tasks trained from 800800 unaugmented examples per difficulty tier, settling operators do not degrade with added depth, and on some tasks convert it into higher accuracy on harder unseen instances (Sudoku, 0.190.19 to 0.340.34 past the training horizon). A single terminal fixed-point objective moves the regime and the depth behavior together: removing it induces drift and removes the gains, and adding it to a generic recurrence yields depth-safe extrapolation on carry propagation. We give four operational criteria for useful test-time depth, use them to catalogue failure modes, and, as a consistency check, apply the same measurements to Huginn-3.5B, which falls in the non-settling family.

Figures & tables

Figure 1: Two notions of “harder,” one task family. Left : larger graphs, competent on trained sizes (dashed), transfer nothing. Right : same-size, deeper instances, where settling models generalize and plateau.
Figure 2: Depth curves behind Table 1 : EM on the first unseen tier; one line == one run, colored by measured regime (boundary dotted, drift dash-dotted): settling runs rise and plateau, marginal rise and collapse or stay flat, drifting decay here or on deeper tiers (Sec. 5.4 ); which architecture settles differs per ladder.
 Sudoku (hole fraction)Reachability (path length)Addition (carry length)
 learn ↑\uparrowextrapolate ↑\uparrowoverthink ↓\downarrowlearn ↑\uparrowextrapolate ↑\uparrowoverthink ↓\downarrowlearn ↑\uparrowextrapolate ↑\uparrowoverthink ↓\downarrow
CoRe (full recipe)1.000.19 →\to 0.34 ±\pm 0.0400.980.92 →\to 0.92 ±\pm 0.1000.990.13 →\to 0.17 ±\pm 0.150.05
−- structure0.00—n/a0.15 ±\pm 0.02—n/a= CoRe (no spatial structure on 1-D)
−- extra input injection0.29 ±\pm 0.01—n/a0.23—n/a0.04 ±\pm 0.02—n/a
−- fixed-point objective1.000.01 →\to 0.03 ±\pm 0.010.041.000.97 →\to 0.97 ±\pm 0.020.37 †1.000.14 →\to 0.15 ±\pm 0.020.81 †
generic recurrence0.00—n/a0.17 ±\pm 0.01—n/a1.000.11 →\to 0.12 ±\pm 0.020.84 †
Table 1: Three questions, kept separate (mean ±\pm sd): the primary ablations and all architectures (Appendix F , Table 3 for six further ablations, none switching mode). learn ↑\uparrow : in-distribution EM at best depth. extrapolate ↑\uparrow : first-unseen-tier EM at training depth →\to best over hc≤128h_{c}{\leq}128 , selected per run on the evaluation set (R1 conversion). Bold == best value per column, ties included (max for learn/extrapolate, min for overthink; gated cells excluded). overthink ↓\downarrow : worst over tiers of best EM −- EM@128 ( †\dagger == severe ≥0.2{\geq}0.2 loss, undesirable). EM <0.3{<}0.3 fails the competence gate: extrapolate/overthink are n/a (nothing to convert or lose). Reachability’s rounds stay below the training depth, so its middle column is generalization ++ safety, not conversion; “—” marks undefined or gated cells. Naming per Sec. 4 . CoRe , −- fixed-point objective, and TRM-style cells carry n≈20n{\approx}20 seeds, architecture rows n=10n{=}10 , the remaining ablation rows n=3n{=}3 .
Figure 3: One training term, three consequences (mean and seed range, first unseen tier). Removing the fixed-point objective destroys conversion (Sudoku, left) or safety (reachability, middle); adding only this term to the unchanged generic recurrence creates both on addition-carry (right): 0.12 ⁣→ ⁣0.880.12\!\to\!0.88 , damage 0.91 ⁣→ ⁣00.91\!\to\!0 .
Figure 4: Recurrence at scale, one prompt bank. Left : a latent depth knob (Huginn-3.5B) saturates without converting depth. Right three : token-budget knobs on open 7B reasoners (bold == illustrative family, n=40n{=}40 /point); none shows the settling fate.
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

CorpusRunsUsed for
Original ladders (LOO ++ architectures)304 total; 164 competent (17 settle / 129 marginal / 18 drift)Depth-safety separation (main Sec. 5) and Fig. 5 ; anisotropy AUC; non-drift extrapolator count
Fixed-size ladders (primary)246Table 1, depth curves (main Fig. 2), headline conversion numbers
Trajectory topology (original)325 filesσeff ⁣↔ ⁣\sigma_{\mathrm{eff}}\!\leftrightarrow\! settling ( r=0.75r{=}0.75 ); per-instance confidence
Trajectory topology (fixed-size)1,206 filesRank replication (Spearman ρ=0.69\rho{=}0.69 ); Betti-1 shares
Appendix
Table 2: Population and unit of analysis for each corpus. Counts differ by design; the depth-safety statistic and the regime figure below both use the original-ladder competent corpus (164 runs).
Figure 5: Regime →\to outcome on the 164 competent original-ladder runs (dot == run, bar == median; nn per regime in axis labels). Left: depth conversion, EM(best) −- EM at training depth on the first unseen tier. Right: overthinking damage, worst over tiers of EM(best) −- EM@128. Settling runs show no overthinking tail; marginal and drifting runs do.
Figure 6: The Sudoku ablation landscape (all arms overlaid; colored == narrative-central operators, light blue == recipe variants, gray == others). (a) Lyapunov spectra: learners are anisotropic ; every failure family is flat. (b) Logit-lens rank of the correct answer (hardest training tier): CoRe crystallizes to ≈0{\approx}0 , generic bounces, the identity ablation is frozen. Depth curves for these arms are in the main text; the training-time formation view is in Sec. O .
 Sudoku (hole fraction)Reachability (path length)Addition (carry length)
 learnextrapolateoverthinklearnextrapolateoverthinklearnextrapolateoverthink
CoRe (full recipe)1.000.19 →\to 0.34 ±\pm 0.0400.980.92 →\to 0.92 ±\pm 0.1000.990.13 →\to 0.17 ±\pm 0.150.05
−- structure0.000.00 →\to 0.0000.15 ±\pm 0.020.08 →\to 0.08 ±\pm 0.030= CoRe (no spatial structure on 1-D)
−- extra input injection0.29 ±\pm 0.010.00 →\to 0.0000.230.17 →\to 0.1700.04 ±\pm 0.020.00 →\to 0.000
−- fixed-point objective1.000.01 →\to 0.03 ±\pm 0.010.041.000.97 →\to 0.97 ±\pm 0.020.371.000.14 →\to 0.15 ±\pm 0.020.81
−- orthogonal init1.000.19 →\to 0.34 ±\pm 0.0301.000.96 →\to 0.96 ±\pm 0.010.051.000.18 →\to 0.32 ±\pm 0.280
Appendix
Table 3: Full leave-one-out and architecture table (mean ±\pm sd; architecture rows n=10n{=}10 seeds, CoRe / −- fixed-point/TRM-style n≈20n{\approx}20 , other ablations and SE-RRM 33 ); column definitions as in main text Table 1. learn : in-distribution EM at best depth. extrapolate : first-unseen-tier EM at training depth →\to best over hc≤128h_{c}{\leq}128 . overthink : worst over tiers of best EM −- EM@128 (bold ≥0.2{\geq}0.2 ); “—” marks arm undefined or not run.
SignalDiagnosisFix
σmax⁡→1.0000\sigma_{\max}\to 1.0000 , solve steps flatidentity fixed point (“do nothing”)extra input injection (recall)
Henrici →0\to 0 , σmax⁡=ρ=1\sigma_{\max}{=}\rho{=}1 , KY saturatednormal marginal wander (coverage learner)structured step; fixed-point objective
λmax⁡>0\lambda_{\max}{>}0 , trajectory length explodingdrift; depth will hurt at test timefixed-point objective
settles but EM =0=0 ; flat Lyapunov spectrum ( λ1−λ8<0.1\lambda_{1}{-}\lambda_{8}{<}0.1 )settled isotropically: converged to a non-algorithminductive bias (structure) / less augmentation, not more dynamics control
all eight estimated leading exponents <0<0 , rank curve monotone ↓\downarrow , EM rises with depthlearned recursionnone needed
seed-to-seed EM variance ≫\ggmissing step-size adaptation / clippingadaptive updates; clip
Appendix
Table 4: Signal →\to diagnosis →\to fix: what to monitor while training a recursive model, the pathology each pattern indicates, and the recipe component that treats it.
Figure 7: Hidden-feature rank across training checkpoints (Sudoku; colors as in the main text).
taskdifficultytrainevalsteps
Fixed-size ladders (depth axis; primary)
sudokuhole fraction0.2,0.35,0.50.2–0.78k
reach_pathlenBFS depth2,32,3,5,8,124k
addition_carrycarry-run len2,42,4,8,16,284k
maze_pathlenpath length6,126,12,24,48,806k
sort_disorderdisplacement2,42,4,8,16,244k
Appendix
Table 5: Per-task difficulty parameter, training/evaluation tiers (parameter values), and training-step budget. All other hyperparameters are shared (text above).

Explore similar work

CardsList

  1. Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization

    Mar 23, 2026Hung-Hsuan ChenEfficient Transformer InferenceRecurrent Transformers

    Computer Science and Information Engineering National Central University Taoyuan, Taiwan

  2. Recursive Agentic Reasoning

    Aug 25, 2026Shengxin Zhang, Xiaomin Wu, Xiyang Wu +1Agentic ReasoningReasoning Evaluation

    Google · University of Maryland

  3. Input-Conditioned Search for Efficient Test-Time Scaling of Recursive Reasoners

    Oct 13, 2026Haochen Shi, Tianwen Fu, Qianggang Ding +7Inference-Time SearchInference-Time Scaling

    Mila – Quebec AI Institute · Université de Montréal · University of Southern California +2