Input-Conditioned Search for Efficient Test-Time Scaling of Recursive Reasoners

Haochen Shi; Tianwen Fu; Qianggang Ding; Yinhan Lu; Yizhan Li; Huan Zhang; Yajie Zhao; Xingdi Yuan

cs.AIOct 13, 2026

Input-Conditioned Search for Efficient Test-Time Scaling of Recursive Reasoners

Authors: Haochen Shi, Tianwen Fu, Qianggang Ding, Yinhan Lu, Yizhan Li, Huan Zhang, Yajie Zhao, Xingdi Yuan, +2 more

Organizations: Mila – Quebec AI Institute · Université de Montréal · University of Southern California · Microsoft Research · Ambarella Inc.

Abstract

People often tackle difficult problems by thinking more deeply or exploring more possibilities. Similarly, recursive reasoners, which reason by repeatedly applying a learned update rule to latent states, can also scale inference through more updates (depth) or additional trajectories generated by perturbing latent states (width). When people hit a dead end in their thinking, they may step back and approach the problem from a fresh angle. However, neither depth scaling nor perturbation-based width scaling of recursive reasoners can reliably achieve this: both may converge to incorrect attractors. To address this, we propose to explore diverse reasoning traces by applying different update maps to the same latent state, without any additional training. We realize this idea through Input-Conditioned Search (ICS), which searches over input conditions to induce alternative update maps from the pretrained reasoner. Across six reasoning tasks, ICS achieves the highest mean accuracy, while requiring up to two orders of magnitude fewer model FLOPs than the evaluated scaling baselines. Controlled experiments confirm that ICS can redirect trajectories from the same state. Our theoretical analysis and rigorous numerical verification also confirm that ICS can locally eliminate incorrect attracting fixed points. Together, these findings establish ICS as an effective axis for efficient test-time scaling and motivate further research on shaping recursive reasoning during training and at test time.

Figures & tables

Figure 1: Sudoku test-time scaling without a task certificate at selection. All curves are the average of three trained seeds. The Attractor point uses the

3{,}072

-puzzle subset and is marked separately.

Figure 2: ICS lifts efficient test-time scaling of recursive reasoners by shaping the update rules.

Task certificate available at selection
Protocol	Sudoku	Maze	Light-Up	Nurikabe	Tapa	Heyawake
Standard TRM	$86.0\pm 0.6$	$91.6\pm 0.3$	$99.7\pm 0.5$	$83.8\pm 2.8$	$72.8\pm 5.4$	$86.4\pm 3.3$
+ depth scaling	$98.2\pm 0.1$	$91.6\pm 0.3$	$\mathbf{100.0\pm 0.0}$	$84.9\pm 2.7$	$75.4\pm 5.5$	$87.8\pm 1.5$
PTRM	$98.3\pm 0.1$	$92.4\pm 0.3$	$\mathbf{100.0\pm 0.0}$	$86.1\pm 2.3$	$76.4\pm 6.9$	$88.7\pm 1.5$
GRAM	$97.5\pm 0.4$	$90.8\pm 0.6$	$99.7\pm 0.5$	$91.9\pm 1.8$	$52.3\pm 5.5$	$97.1\pm 1.3$
EqR	$98.8\pm 0.1$	$95.0\pm 1.9$	$91.4\pm 0.5$	$77.1\pm 0.5$	$26.7\pm 0.9$	$31.6\pm 2.7$

Table 1: Answer accuracy (%) on six tasks.

\dagger

: Attractor Models is evaluated on a fixed

3{,}072

-instance Sudoku-Extreme subset. Appendix A.5 compares all methods on the same subset.

Figure 3: Six-task efficiency and PPBench golden-set accuracy. Both radar panels use logarithmic radii, farther inward indicates lower cost.Appendix A.8 gives complete LLMs results.

	Certificate selection		Model selection
Components	Accuracy (%)	TFLOPs	Accuracy (%)	TFLOPs
A: TRM + depth scaling	87.85	0.38	87.85	0.38
AB: + hypotheses	99.92	1.39	99.75	1.10
AC: + restatements	98.83	1.19	98.82	1.19
ABC: full ICS	99.94	1.39	99.81	1.12

Table 2: Sudoku component ablation on the same eight test windows and trained model. Appendix C.4 provides more details for the complete ablation study.

Figure 4: State interventions and ICS recover different failures. Oracle direction requires label access and more computation(Appendix C ).

Figure 5: Recovery depends on when the state is perturbed. Shading shows pointwise 95% instance-bootstrap intervals. The time axis uses

\log(1+t)

spacing.

t=0

denotes fresh initialization.

Figure 6: The landscapes show the relative one-step update residuals. Smaller residuals indicate less change in the latent state after one recursion step.

Appendix figures & tables42 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Certificate selection	Model selection
Standard TRM	$84.80\pm 1.18$	$84.80\pm 1.18$
TRM + depth scaling †	$97.48\pm 0.63$	$97.49\pm 0.75$
PTRM	$97.47\pm 0.96$	$97.47\pm 0.96$
GRAM, LPRM	$94.99\pm 0.30$	$94.69\pm 0.52$
GRAM, majority	$94.99\pm 0.30$	$94.09\pm 0.56$
EqR	$97.48\pm 0.16$	$97.46\pm 0.17$

Appendix

Table 3: Solution accuracy (%) on the same first

3{,}072

Sudoku-Extreme test instances. Values are means and sample standard deviations across three training seeds. Each method retains its corresponding inference budget; this is not an equal-compute comparison. Standard TRM evaluates the same native 16-step output in both columns. The two GRAM rows share the same certificate coverage.

\dagger

: depth scaling uses a

6{,}400

-step budget. Its two columns come from separate archived numerical executions with different batch configurations, rather than two selectors applied to an identical trajectory pool.

Task	Hypothesis assignment
Sudoku-Extreme	A digit from $1$ to $9$
Maze-Hard	A traversable cell belongs to the path
Light-Up	A lamp is placed in an empty cell
Nurikabe	A cell is shaded or unshaded
Tapa	A cell is shaded or unshaded
Heyawake	A cell is shaded or unshaded

Appendix

Table 4: Task-specific assignments used for ICS.

Task	Original-input depths	Levels	Beam	Proposals	Nodes	Views	Selection
Certificate-based selection
Sudoku-Extreme	20	6	4	48	96	32	Certificate
Maze-Hard	20	6	4	48	96	8	Certificate
All four PPBench tasks	20	6	4	48	96	8	Certificate
Model-based selection
Sudoku-Extreme	20	8	3	6	24	32	$q$ threshold

Appendix

Table 5: ICS search configurations. Levels, beam width, proposals per parent, and evaluated hypothesis nodes specify upper limits. Views denotes the size of the final restatement set, including the identity representation.

Model / configuration	Light-Up	Nurikabe	Tapa	Heyawake	Overall (%)
gpt-5.6-sol, xhigh	13	9	13	12	78.3
gpt-5.6-sol, high	12	7	11	11	68.3
gpt-5.6-sol, medium	9	2	6	3	33.3
Claude Opus 5, high	11	11	14	9	75.0
Kimi K3, max	13	4	10	4	51.7
Qwen3.8 Max, max	13	5	8	2	46.7

Appendix

Table 6: Results on the PPBench golden set. Task columns give correctly solved counts out of 15 instances; overall accuracy uses all 60 instances. All rows use answer selection without the task certificate. ICS rows correspond to separate training seeds.

Figure 7: Different latent behaviors can accompany either correct or incorrect answers. Top: incorrect endpoints. Bottom: correct endpoints. Columns show near-stationary, periodic, regular moving, and irregular moving examples. Each panel uses an independent PCA basis fitted to the final 256 saved random projections. Colored paths show this window; gray paths show the earlier approach for the near-stationary examples. Black markers indicate endpoints, and

r(T)

reports the final recorded relative update residual. Axis scales differ across panels.

Figure 8: Near-fixed points with different answer outcomes. Both Sudoku MLP trajectories approach numerical stationarity within 512 updates. The incorrect raw prediction satisfies the row, column, and box constraints but violates an original given; restoring that given produces an answer that fails the task rules. The other endpoint is correct. Columns show update residuals, the projected trajectory, the final-window residual distribution, and a centered scalar return map. The two residual curves normalize the update norm by

\|z_{t}\|_{2}

and

\|z_{t+1}\|_{2}

, respectively.

Figure 9: Periodic returns with different answer outcomes. The Tapa trajectory has an incorrect endpoint and a detected period of seven after 512 updates. The Light-Up trajectory has a correct endpoint and a detected period of three after 2,048 updates. Both continue moving between successive states. Period detection and local stability are evaluated separately: the Light-Up example does not pass the conservative spectral-radius criterion. Correctness is checked at the saved endpoint. Readouts follow Figure 8 .

Figure 10: Regular-motion examples with different answer outcomes. The Heyawake endpoint is incorrect and the Nurikabe endpoint is correct, both after 512 updates. Both satisfy the regular-motion diagnostic criteria, despite their different projected appearances. The Nurikabe trajectory exhibits a looping projection while maintaining nonzero update residuals. Its correct endpoint therefore does not require latent stationarity. Readouts follow Figure 8 .

Figure 11: Irregular-motion examples with different answer outcomes. The Nurikabe endpoint is incorrect after 512 updates; the Heyawake endpoint is correct after 2,048 updates. Their small/large-motion distinction uses the relative projected diameter in Equation 12 . The Heyawake example satisfies the irregularity criterion through its 0–1 statistic despite a negative finite-time perturbation-growth estimate. Readouts follow Figure 8 .

$\sigma$	Answer retention (%)	Latent return (%)
0.001	100.0	75.0
0.003	100.0	64.3
0.010	100.0	43.8
0.030	100.0	25.5
0.100	100.0	16.8
0.300	100.0	11.8

Appendix

Table 7: Perturbation robustness of 239 incorrect near-fixed points from one Sudoku-Extreme TRM checkpoint. Each scale includes 16 perturbations per instance (3,824 trials). Answer retention compares the raw endpoint decode with the original decode after 64 updates. Solution discovery records whether any rule-valid prediction appears during those updates, with the original givens restored. Latent return uses the full-state neighborhood criterion defined in the text. All entries are descriptive percentages over perturbation trials.

Figure 12: Sudoku MLP: failed-reference diagnostics. Left: first-eight strict recovery coverage; HIS uses up to eight cold model hypotheses. Shading and bars are pointwise Wilson 95% intervals across instances. Middle/right: per-trial decode preservation and latent return.

Figure 13: Sudoku MLP: solved-reference diagnostics. Top: per-trial state-edit outcomes under the original input. Bottom: separate answer-assisted input hypotheses agreeing or conflicting with the stored answer, evaluated from cold initialization; these are not HIS performance estimates. Bars are pointwise Wilson 95% intervals across instances; counts follow legend order.

Figure 14: Maze attention: failed-reference diagnostics. Left: first-eight strict recovery coverage; HIS uses up to eight cold model hypotheses. Shading and bars are pointwise Wilson 95% intervals across instances. Middle/right: per-trial decode preservation and latent return.

Figure 15: Maze attention: solved-reference diagnostics. Top: per-trial state-edit outcomes under the original input. Bottom: separate answer-assisted input hypotheses agreeing or conflicting with the stored answer, evaluated from cold initialization; these are not HIS performance estimates. Bars are pointwise Wilson 95% intervals across instances; counts follow legend order.

Figure 16: Sudoku GRAM (mean path): failed-reference diagnostics. Left: first-eight strict recovery coverage; HIS uses up to eight cold model hypotheses. Shading and bars are pointwise Wilson 95% intervals across instances. Middle/right: per-trial decode preservation and latent return.

Figure 17: Sudoku GRAM (mean path): failed-reference diagnostics. Left: first-eight strict recovery coverage; HIS uses up to eight cold model hypotheses. Shading and bars are pointwise Wilson 95% intervals across instances. Middle/right: per-trial decode preservation and latent return.

Figure 18: Sudoku GRAM (mean path): solved-reference diagnostics. Top: per-trial state-edit outcomes under the original input. Bottom: separate answer-assisted input hypotheses agreeing or conflicting with the stored answer, evaluated from cold initialization; these are not HIS performance estimates. Bars are pointwise Wilson 95% intervals across instances; counts follow legend order.

Figure 19: Tapa: failed-reference diagnostics. Left: first-eight strict recovery coverage; HIS uses up to eight cold model hypotheses. Shading and bars are pointwise Wilson 95% intervals across instances. Middle/right: per-trial decode preservation and latent return.

Figure 20: Tapa: solved-reference diagnostics. Top: per-trial state-edit outcomes under the original input. Bottom: separate answer-assisted input hypotheses agreeing or conflicting with the stored answer, evaluated from cold initialization; these are not HIS performance estimates. Bars are pointwise Wilson 95% intervals across instances; counts follow legend order.

Figure 21: Heyawake: failed-reference diagnostics. Left: first-eight strict recovery coverage; HIS uses up to eight cold model hypotheses. Shading and bars are pointwise Wilson 95% intervals across instances. Middle/right: per-trial decode preservation and latent return.

Figure 22: Heyawake: solved-reference diagnostics. Top: per-trial state-edit outcomes under the original input. Bottom: separate answer-assisted input hypotheses agreeing or conflicting with the stored answer, evaluated from cold initialization; these are not HIS performance estimates. Bars are pointwise Wilson 95% intervals across instances; counts follow legend order.

Figure 23: Nurikabe: failed-reference diagnostics. Left: first-eight strict recovery coverage; HIS uses up to eight cold model hypotheses. Shading and bars are pointwise Wilson 95% intervals across instances. Middle/right: per-trial decode preservation and latent return.

Figure 24: Nurikabe: solved-reference diagnostics. Top: per-trial state-edit outcomes under the original input. Bottom: separate answer-assisted input hypotheses agreeing or conflicting with the stored answer, evaluated from cold initialization; these are not HIS performance estimates. Bars are pointwise Wilson 95% intervals across instances; counts follow legend order.

Figure 25: Light-Up: solved-reference diagnostics. Top: per-trial state-edit outcomes under the original input. Bottom: separate answer-assisted input hypotheses agreeing or conflicting with the stored answer, evaluated from cold initialization; these are not HIS performance estimates. Bars are pointwise Wilson 95% intervals across instances; counts follow legend order.

Model and task	Failed references	Correct references	Total
Sudoku TRM	202	153	355
Maze TRM	82	32	114
Sudoku GRAM	209	120	329
Tapa TRM	11	32	43
Heyawake TRM	10	32	42
Nurikabe TRM	14	32	46

Appendix

Table 8: Diagnostic cohorts in the extended perturbation study. Correct references include late-solution references. Counts describe the sampled model–instance pairs, not the frequency of each outcome in the full test sets.

Figure 26: Recovery depends on initialization and hypothesis content. All panels use the frozen Sudoku-Extreme TRM-MLP checkpoint (training seed 0). (a,b) Warm and cold starts use the same instance–hypothesis pairs under

F_{x_{h}}

. Compatible hypotheses agree with the reference solution at the edited cell; this label is assigned after proposal generation. (c) Cold-start, label-informed controls inject correct or wrong values at unproposed erroneous cells or already-correct mutable cells. All rates measure validity of the final answer after 512 updates and restoration of original givens, pooled over instance–hypothesis pairs; they do not measure whether any of several hypotheses succeeds per instance. Injected values are not forced into the returned answer, so incompatible hypotheses can still yield valid outputs. Error bars show 95% confidence intervals from 1,000 bootstrap resamples of instances. Sample counts are given in Tables 9 and 10 .

Failure cohort	Hypothesis	Pairs	Warm (%)	Cold (%)
Near-fixed point (239)	Compatible	602	0.00	73.92
	Incompatible	3,222	0.00	10.09
Moving (4,073)	Compatible	17,381	50.43	61.26
	Incompatible	47,787	1.43	0.09
Periodic (2)	Compatible	8	100.00	100.00
	Incompatible	24	12.50	0.00

Appendix

Table 9: Final-answer recovery under paired warm and cold starts. Each pair uses the same instance and injected hypothesis for both initializations, with 512 updates under the conditioned map. Parentheses in the first column give instance counts; the denominator of each percentage is the number of instance–hypothesis pairs. The periodic and other/unresolved groups are reported descriptively because they contain very few instances.

Injected assignment	Near-fixed-point failures	Moving failures
Model proposal, compatible	73.9% (445/602)	61.3% (10,648/17,381)
Model proposal, incompatible	10.1% (325/3,222)	0.09% (45/47,787)
Correct value at unproposed error cell	74.9% (715/955)	73.3% (11,950/16,292)
Correct value at already-correct cell	49.1% (469/956)	26.2% (4,262/16,292)
Wrong value at unproposed error cell	14.6% (139/955)	6.6% (1,082/16,292)
Wrong value at already-correct cell	14.2% (136/956)	2.9% (480/16,292)

Appendix

Table 10: Cold-start recovery after 512 updates, stratified by hypothesis content. Counts are successful/attempted instance–assignment pairs. An already-correct cell is an editable cell correctly predicted at the saved state, not an original given. Compatible model proposals occur on 225 of 239 near-fixed-point instances and 4,060 of 4,073 moving instances. One near-fixed-point instance has only three eligible unproposed error cells, yielding 955 rather than 956 error-cell controls.

State	Map	$n$	Median $\hat{\rho}$ [Q1, Q3]	$\hat{\rho}<1$	Batch check
Original incorrect state	$F_{x}$	239	0.9185 [0.8926, 0.9429]	239	128
Retained incorrect, compatible	$F_{x_{h}}$	585	0.9085 [0.8846, 0.9366]	585	84
Retained incorrect, incompatible	$F_{x_{h}}$	3,089	0.9076 [0.8829, 0.9339]	3,087	449
Recovered correct, matched pairs	$F_{x_{h}}$	424	0.9113 [0.8849, 0.9320]	424	47
After restoring $x$ , same pairs	$F_{x}$	424	0.9175 [0.8986, 0.9380]	424	9
Retained correct, solved controls	$F_{x_{h}}$	6,534	0.9147 [0.8882, 0.9371]	6,524	604

Appendix

Table 11: Spectral-radius estimates at near-fixed states. The first row has one state per instance; subsequent rows count instance–hypothesis pairs. Retained endpoints preserve the original decoded answer and lie within

0.05\lVert z_{\star}\rVert_{2}

of the starting state. The two matched rows use exactly the same 424 compatible interventions on 203 initially failed instances, before and after restoring

x

. The last row uses the separate 431-instance solved cohort described below. The final two columns count estimates below one and endpoints belonging to batches that pass the ratio-stability check, respectively; the latter is not an individual-state convergence rate. Quartiles describe the distribution of estimates, not uncertainty in an individual spectral radius.

	Certificate selection			Model selection
Configuration	Acc. (%)	Steps	TFLOPs	Acc. (%)	Steps	TFLOPs
A: TRM + depth scaling ( $D=20$ )	87.85	20.0	0.38	87.85	20.0	0.38
AB: + hypotheses	99.92	74.0	1.39	99.75	58.8	1.10
AC: + restatements	98.83	63.7	1.19	98.82	63.6	1.19
ABC: full HIS	99.94	74.4	1.39	99.81	59.8	1.12
ABC: random locations and values	99.85	75.5	1.42	98.81	58.8	1.10

Appendix

Table 12: HIS ablation on Sudoku-extreme (with 49,152 instances). A uses the same best-

q

output in both columns; its exact cost is

0.375

TFLOPs. Search configurations retain adaptive original-input attempts. Both accuracy columns evaluate delivered answers with the original task rules; only certificate-selected search consults those rules during selection. Steps and TFLOPs are mean executed model computation per instance, charging every launched evaluation. One step costs

1.875\times 10^{10}

FLOPs. Search accuracies pool reconstructed counts; search costs use rounded window means. No multi-seed uncertainty is implied.

Figure 27: Numerical continuation in the trained Maze-Hard and Sudoku-Extreme models. (a) The Maze example exhibits a fold near

\lambda=0.79650

. (b) A Sudoku example has two folds, revealing an intervening unstable branch. (c) Its full continuation and a persistent Sudoku control reach the fully conditioned input. Points are connected in continuation order; vertical coordinates report

100\|z-s_{w}\|_{2}/\|s_{w}\|_{2}

. Panel (b) magnifies the fold region. Stability styles use numerical multiplier estimates, not interval certificates.

		Solution-consistent hypotheses				Inconsistent hypotheses
Width	Seed	Tested	Folds	Roots	Complete	Tested	Complete
8	0	71	21	19	19	9	0
8	1	56	1	1	1	24	3
8	2	61	13	12	12	19	4
16	0	23	6	6	3	17	0
16	1	45	7	7	7	35	2

Appendix

Table 13: Fold detection and certification in six trained small models. Counts concern selected checkpoint–instance–hypothesis pairs, not independent puzzles. All nine detected folds under inconsistent hypotheses have complete certificates. Failure to complete a certificate does not establish absence of a fold.

Figure 28: Certified local folds and subsequent warm-start outcomes in trained small models. (a) Fold locations and certified bounds on the noncritical spectral radius for all 51 complete certificates; markers distinguish solution-consistent and inconsistent hypotheses, and colors indicate the 512-update endpoint category after crossing the fold. Each bound concerns the noncritical spectrum; the critical multiplier itself equals one. (b) Endpoint categories at

\lambda=\min\{1,\lambda_{c}+0.02\}

: 42 consistent and nine inconsistent hypotheses. Bar widths show percentages and labels give case counts. “Same wrong answer” denotes output agreement with the initial failure, not return to the same latent state.

Figure 29: Recovery under spatial restatements depends on initialization and the duration of conditioning. Recovery is the fraction of instance–restatement pairs with a correct prediction by the recorded rollout step; original-map controls average over instances. Bands show pointwise 95% instance-bootstrap intervals. Each restated cold/warm rollout uses the same task and transformation, differing only in its initial state. The one-step control first applies

\widetilde{F}_{\Phi}

once, then records

t

updates under

F_{x}

: its total cost is

t+1

steps, and the preparatory state is not included in its recovery window. Panel

y

-axis ranges differ.

	Sudoku-Extreme			Maze-Hard
Condition	All failures	Near-fixed	Moving	Near-fixed
Instances	4,320	239	4,073	42
Original map, cold	1.13	5.86	0.83	0.00
Original map, warm	0.05	0.00	0.00	0.00
Restated map, cold	7.60	35.41	5.92	72.45
Restated map, warm	1.85	0.00	1.88	43.88

Appendix

Table 14: Recovery and coverage (%) on the archived failure cohorts. The upper rows report recovery within 512 recorded updates; restated conditions average over instance–restatement pairs, and original-map controls average over instances. The final two rows count instances recovered by at least one of

K=8

Sudoku or

K=7

Maze restatements.

\dagger

: this control uses one preparatory restated update followed by 512 original-map updates, for 513 total steps.

Condition	Sudoku-Extreme	Maze-Hard
Original map, cold	0.45% (2/444)	0.00% (0/38)
Restated map, cold	0.53% (19/3,552)	4.89% (13/266)
Restated map, warm	0.00% (0/3,552)	0.38% (1/266)

Appendix

Table 15: Invalid final predictions after 512 updates on the separately screened solved-control cohorts. Counts are invalid/total rollouts. Restated rows contain eight transformations per Sudoku instance and seven per Maze instance.

Figure 30: Input retention has model-dependent effects. (a) Strict correctness after 512 updates from a saved incorrect state. Solid curves: hypotheses successful after 64 cold-start updates; dashed curves: all 128 hypotheses from the 16-instance screening cohort. Bands show nondegenerate 95% instance-bootstrap intervals. (b) Endpoint decode categories among cold-successful hypotheses when conditioning is retained throughout (

m=T

Model	Hypothesis pool	$m=1$	$m=2$	$m=16$	$m=64$	$m=512$
Sudoku MLP	All	$0/128$	$0/128$	$0/128$	$0/128$	$0/128$
	Cold-successful	$0/37$	$0/37$	$0/37$	$0/37$	$0/37$
Maze attention	All	$9/128$	$23/128$	$27/128$	$27/128$	$20/128$
	Cold-successful	$7/25$	$18/25$	$20/25$	$20/25$	$20/25$
Sudoku attention	All	$0/128$	$1/128$	$4/128$	$7/128$	$15/128$
	Cold-successful	$0/26$	$1/26$	$4/26$	$7/26$	$14/26$

Appendix

Table 16: Strictly correct warm-start endpoints after 512 updates. The hypothesis is retained for the first

m

updates. Each model’s full pool contains eight hypotheses for each of 16 screened instances. Cold-successful subsets are selected by correctness after 64 updates from fresh initialization. Denominators count instance–hypothesis pairs, not distinct instances.

Model	Strictly correct	Same decode	Latent return
Sudoku MLP	$16/16$	$16/16$	$16/16$
Maze attention	$15/15$	$15/15$	$15/15$
Sudoku attention	$7/7$	$7/7$	$5/7$

Appendix

Table 17: Solution round trips. The selected pairs have strictly correct, numerically near-fixed endpoints under both input conditions before the final phase. Results describe

\widetilde{s}_{H}

after reapplying the hypothesis for 512 updates. Same-decode and latent-return measurements use

s_{H}

as their reference. Each instance contributes one trial.

Model	Original input	Hypothesis input
	$r_{x}(w)$	$r_{x_{H}}(w)$
Sudoku MLP	$1.92\times 10^{-7}$	$2.80\times 10^{-2}$
Maze attention	$2.37\times 10^{-7}$	$6.67\times 10^{-2}$
Sudoku attention	$4.75\times 10^{-7}$	$1.65\times 10^{-1}$

Appendix

Table 18: Median relative one-step displacement from saved incorrect states. Each model contributes 20 instances. The original-input column uses one measurement per instance; the hypothesis-input column pools eight hypotheses per instance, giving 160 state–hypothesis pairs without filtering by cold-start success or halting score. Both quantities are dimensionless and normalized by the same starting state’s norm.

Figure 31: Training supervision changes decoded behavior. Each point summarizes one trained model’s reference-mismatch cohort after 16 recursion steps. The horizontal axis measures subsequent decoded changes; the vertical axis measures return to the original decoded prediction after perturbation. Dashed arrows compare separately trained conditions, whereas solid arrows connect a self-imitation initialization to its third fine-tuning round. The Sudoku inset magnifies the same observations. Standard Sudoku models are probed on 3,072 test instances and retrained models on 422,786; Maze models use the same 1,000-instance test split. Maze reference mismatches can include valid alternative solutions.

Figure 32: Task performance under training interventions. (a) Additional inference on the 469 initially rule-invalid instances of the Maze Sample

K=1

model. Continuation measures correctness at the budget endpoint; restart measures coverage across trial endpoints. Noise scales are standard deviations in raw state units. (b) Sudoku rule-valid accuracy under changes to training inputs and target conventions. Original-test bars show three-seed means and standard deviations; clue-removed results use seed 0.

Explore similar work

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
May 20, 2026Benhao Huang, Zhengyang Geng, Zico KolterDynamical Systems Test-Time Scaling
CMU
ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling
Aug 11, 2026Vaibhav Singh, Soumya Suvra Ghosal, Sarvesh Gharat +3Numerical Reasoning in Language Models Test-Time Scaling
IIT Bombay · University of Maryland, College Park · Adobe Research
Planned Test-Time Scaling with Coordinated Reasoning Paths
Sep 23, 2026Xueqing Wu, Langxing Bai, Hritik Bansal +5RL for Language Model Reasoning Test-Time Scaling
University of California, Los Angeles · Amazon

May 20, 2026J/K move · Enter open · S save

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning

Benhao Huang, Zhengyang Geng, Zico Kolter

CMU

Dynamical Systems Test-Time Scaling Algorithmic Reasoning Latent Reasoning

Open paper AskPDF arXiv

Scaling test-time compute by iteratively updating a latent state has emerged as a powerful paradigm for reasoning. Yet the internal mechanisms that enable these iterative models to generalize beyond memorized patterns remain unclear. We hypothesize that generalizable reasoning arises from learning task-conditioned attractors: latent dynamical systems whose stable fixed points correspond to valid solutions. We formalize this process through Equilibrium Reasoners (EqR), which enable test-time scaling without external verifiers or task-specific priors. EqR scales internal dynamics along two axes: depth, by running more iterations, and breadth, by aggregating stochastic trajectories from multiple initializations. Empirically, gains from test-time scaling are tightly coupled with stronger convergence toward solution-aligned attractors. This attractor perspective allows neural networks to adaptively allocate test-time compute based on task difficulty. While simple cases converge within 1 to 5 iteration steps, harder cases benefit from massive test-time scaling. By unrolling up to the equivalent of 40,000 layers, scalable latent reasoning boosts accuracy from 2.6% for feedforward models to over 99% on Sudoku-Extreme. These results suggest that learned attractor landscapes provide a useful mechanistic lens for understanding scalable reasoning in iterative latent models.

Abstract

Explore similar work

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning

ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling

Planned Test-Time Scaling with Coordinated Reasoning Paths