Resample the Answer: Where to Spend Test-Time Compute in Recursive Reasoning

Maxime Illouz

cs.LGOct 12, 2026

Resample the Answer: Where to Spend Test-Time Compute in Recursive Reasoning

Authors: Maxime Illouz

Organizations: Concordia University · Mila – Québec AI Institute · Montréal, Canada

Abstract

Recursive reasoning models such as the Tiny Recursive Model (TRM) iteratively refine two latent states: an answer state y, decoded into the output, and a reasoning state z, which is never decoded. Many of their failures have been attributed to bad basins, regions of latent space that trap the recursion on wrong answers. Recent test-time approaches perturb y, z or both to recover these failures, but the impact of this choice on performance, test-time compute efficiency and scaling properties remains unclear. Through controlled experiments on TRM, we find that a failed trajectory carries its failure in its answer state. At equal compute, in depth and in width, resampling y recovers more failures than any perturbation of z we tested, and the performance gap widens as the test-time compute budget grows. Our approach, Gated Answer Resampling (GAR), pairs this resampling of the answer state with the model's own halting head and restarts only the puzzles that the head rejects. With at most 64 restarts per puzzle and no training or hyperparameter search, GAR takes TRM from 85.9% to 99.4% on Sudoku-Extreme and from 68.8% to 85.6% on Pencil Puzzle Bench validation. On the same checkpoints, GAR also outperforms Probabilistic TRM (PTRM), which perturbs z at each step, even when PTRM uses 42 to 197 times as much compute.

Figures & tables

Panel of Figure 1 — Figure 1: Gated Answer Resampling (GAR). The recursion runs in blocks of $16$ supervision steps over a fast reasoning state $\mathbf{z}$ (blue) and a slow answer state $\mathbf{y}$ (orange). The first block is the standard pass. After each block, the grid $\hat{\mathbf{y}}$ is committed if the halting logit $\hat{q}$ , read from $\mathbf{y}$ , is non-negative. Otherwise a new block (a retry ) runs, at most $R$ times. A retry restarts from the initial states, except on the fillable cells of $\mathbf{y}$ , which receive fresh isotropic Gaussian vectors rescaled to the norm of $\mathbf{y}_{\mathrm{init}}$ . If no grid is committed, the fallback returns the grid of the block with the highest $\hat{q}$ .

Figure 2: Resampling

\mathbf{y}

recovers the most failures, whether retries continue or restart. Fraction of the standard-pass failures that end up correct after

16

retries, out of

141

on Sudoku-Extreme and

96

on PPBench validation. In each panel, retries continue the failed trajectory on the left (depth) and restart from the initial state on the right (width). Each side has four bars, one per kind of retry: no perturbation, PTRM’s noise on

\mathbf{z}

, resampling

\mathbf{z}

, and resampling

\mathbf{y}

, which on the right is GAR. A restart without perturbation replays the standard pass and recovers nothing. Bars show the mean over three seeds

\pm

one standard deviation (values in Table 9 ).

Figure 3: When continuing a failed trajectory, resampling

\mathbf{y}

lets it escape, but resampling

\mathbf{z}

does not. The states

\mathbf{y}

and

\mathbf{z}

of one sudoku from the PPBench validation split after each supervision step, coloured by block. Each state is projected onto two principal axes common to the three runs. With depth alone,

\mathbf{y}

stays near the region visited by the standard pass. Resampled at each block boundary,

\mathbf{y}

returns there until block

14

, where it escapes,

\mathbf{z}

follows, and the block ends on the correct grid. Resampled instead,

\mathbf{z}

is back near its region one step later, and

\mathbf{y}

stays near its own, as with depth alone.

Figure 4: Resampling

\mathbf{y}

scales better with retries than perturbing

\mathbf{z}

, at a lower cost. For each benchmark, the figure shows the accuracy after

r

retries and the fraction of re-opened puzzles still active (no grid accepted) before the

r

-th retry, against

r

on a logarithmic scale (Table 11 gives accuracies and costs at seven of the ten plotted values of

r

). Whether

\mathbf{z}

is resampled once (“

\mathbf{z}

instead of

\mathbf{y}

”) or perturbed before every step (PTRM), GAR’s lead grows with

r

, and its searches end sooner. On PPBench, each doubling of

r

adds about two points to GAR, versus about one when

\mathbf{z}

is perturbed. On Sudoku-Extreme, “continue instead of restart” almost coincides with GAR. Curves show the mean over three seeds

\pm

one standard deviation. TRM continues without perturbation under the harness (depth alone). PTRM scales width: each retry is one rollout of

D=16

steps.

	Sudoku-Extreme		PPBench validation
	TRM, standard pass: $85.90$		TRM, standard pass: $68.83$
	$16$ retries	$64$ retries	$16$ retries	$64$ retries
TRM, under the harness	$96.30$ $\,\pm 0.00$ $(1.97)$	$97.40$ $\,\pm 0.00$ $(3.33)$	$73.70$ $\,\pm 0.00$ $(4.79)$	$75.00$ $\,\pm 0.00$ $(15.20)$
PTRM	$96.93$ $\,\pm 0.15$ $(1.82)$	$97.87$ $\,\pm 0.06$ $(2.90)$	$76.52$ $\,\pm 0.68$ $(4.25)$	$79.00$ $\,\pm 0.19$ $(11.86)$
GAR	$98.27$ $\,\pm 0.21$ $(1.68)$	$99.43$ $\,\pm 0.15$ $(2.03)$	$81.28$ $\,\pm 0.68$ $(3.48)$	$85.61$ $\,\pm 0.37$ 0 $(7.08)$
$\mathbf{z}$ instead of $\mathbf{y}$	$96.97$ $\,\pm 0.35$ $(1.80)$	$97.90$ $\,\pm 0.00$ $(2.84)$	$75.43$ $\,\pm 0.50$ $(4.42)$	$77.16$ $\,\pm 0.19$ $(12.92)$

Table 1: With

16

retries, GAR already outperforms TRM and both perturbations of

\mathbf{z}

64

retries. Accuracies (%,

\pm

standard deviation) and costs in blocks per puzzle (in parentheses) are means over three seeds. All methods, PTRM included, share our checkpoints, instances and harness, which begins with the standard pass of TRM, given in the header. Under the harness, TRM continues without perturbation (depth alone). Each indented row changes one thing in GAR. PTRM runs one rollout of

D=16

steps per retry and differs from “

\mathbf{z}

instead of

\mathbf{y}

” only in how

\mathbf{z}

is perturbed.

			Sudoku-Extreme			PPBench validation
state	method	runs on	cost	accuracy	lost	cost	accuracy	lost
Deterministic TRM
—	TRM, standard pass	all	$1$	$85.90$	—	$1$	$68.83$	—
—	TRM, uniform unroll ( $D{=}64$ / $48$ )	all	$4$	$92.80$	$0.0$	$3$	$68.18$	$10.0$
PTRM with its own protocol: $K^{\prime}$ rollouts (width) of $D$ steps (depth)
$\mathbf{z}$	PTRM ( $K^{\prime}{=}1$ , $D{=}16$ )	all	$1$	$87.70$	$29.0$	$1$	$68.29$	$12.0$

Table 2: With

R=64

, GAR outperforms every PTRM configuration we ran and loses no solved puzzle. We rerun PTRM on our checkpoints, with

\sigma=0.3

on Sudoku-Extreme and

\sigma^{\star}=1.5

on PPBench. The first column gives the perturbed state, and rejected means the puzzles the halting head rejects. Cost is in blocks per puzzle. Lost counts the puzzles that the standard pass solves and a method fails, out of

859

and

212

. Stochastic rows are averaged over three seeds, and the standard deviation of their accuracy is at most

0.46

points on Sudoku-Extreme and

0.75

on PPBench. The uniform unroll runs the recursion for

D

steps on every puzzle. The dagger marks PTRM’s headline budget of

100

rollouts of

64

and

48

steps. It costs

197

and

42

times as much as GAR at

R=64

, and

4.3

and

6.0

times as much under the harness. Tables 7 and 8 list ten configurations per benchmark.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

benchmark	split	$n$	standard pass (%)	recall (%)	false-positive rate (%)
Sudoku-Extreme	$1{,}000$ test puzzles	$1{,}000$	$85.90$	$99.3$	$0.0$
PPBench (5 types)	validation	$308$	$68.83$	$87.5$	$0.0$
PPBench (5 types)	golden	$49$	$67.35$	$93.8$	$3.0$

Appendix

Table 3: The halting head at the last step of the standard pass, against ground truth. Recall is the fraction of the standard-pass failures that the gate re-opens (

140

141

84

96

and

15

16

), and the false-positive rate is the fraction of the standard-pass successes that it re-opens (

0

859

0

212

and

1

33

	training		validation		golden
type	ours	PTRM	ours	PTRM	both
sudoku	$708$	$710$	$99$	$97$	$15$
lightup	$874$	$864$	$55$	$65$	$8$
nurikabe	$1{,}446$	$1{,}380$	$59$	$55$	$9$
heyawake	$4{,}086$	$3{,}828$	$69$	$70$	$7$
tapa	$333$	$333$	$26$	$26$	$10$

Appendix

Table 4: Our PPBench splits against those of Sghaier et al. (2026) . Entries count puzzles per type after the size filter. PTRM’s Table 4 counts training examples, which we divide by

11

. Each build draws its own validation set before the size filter, hence the different validation counts. The asterisk marks shakashaka, which is used in training but left out of the aggregates.

model
	Sudoku-Extreme	PPBench
variant	TRM-MLP (MLP across the $97$ positions)	TRM-Att (self-attention, $8$ heads)
positional encoding	none	RoPE
layers, width	$2$ , $512$ ; channel SwiGLU width $1{,}536$
$T$ , $n$ , $N_{\mathrm{sup}}$	$3$ , $6$ , $16$
prefix $P$	$16$

Appendix

Table 5: Training configuration of the two checkpoints. Centred entries apply to both. The loss uses the stablemax of Prieto et al. (2025) . The

97

positions of TRM-MLP are the

P=16

prefix positions and the

81

cells. Each checkpoint is the last of its training run.

$\sigma$	pass@ $25$	best- $Q$ @ $25$
$0.0$ (deterministic)	$68.83$	$68.83$
$0.1$	$76.19$	$76.19$
$0.2$ (PTRM’s value)	$75.97$	$75.87$
$0.3$	$76.19$	$76.08$
$0.5$	$76.30$	$76.19$
$0.8$	$77.16$	$76.84$

Appendix

Table 6: The noise scale of PTRM on our PPBench checkpoint. Pass@

25

and best-

Q

25

(%) on the

308

validation puzzles, with

K=25

rollouts of

D=16

steps, averaged over three seeds. The grid extends PTRM’s window

[0,1]

, beyond which this checkpoint still improves. Bold entries mark the plateau of best-

Q

25

inside which we fix

\sigma^{\star}=1.5

	own protocol				$+$ fallback
configuration	cost	accuracy (%)	pass@ $K^{\prime}$	lost	accuracy (%)	lost
TRM, standard pass	$1$	$85.90$	—	—	—	—
TRM, uniform unroll ( $D{=}64$ )	$4$	$92.80$	—	$0.0$	—	—
$K^{\prime}{=}1$ , $D{=}16$	$1$	$87.70$ $\,\pm 0.44$	$87.70$ $\,\pm 0.44$	$29.0$	$90.57$ $\,\pm 0.25$	$0.0$
$K^{\prime}{=}5$ , $D{=}16$	$5$	$94.70$ $\,\pm 0.26$	$94.70$ $\,\pm 0.26$	$3.7$	$95.03$ $\,\pm 0.15$	$0.0$
$K^{\prime}{=}10$ , $D{=}16$	$10$	$96.30$ $\,\pm 0.10$	$96.33$ $\,\pm 0.15$	$1.3$	$96.40$ $\,\pm 0.10$	$0.0$

Appendix

Table 7: PTRM on Sudoku-Extreme, with its own protocol and under the harness. Each configuration uses the first

K^{\prime}

100

rollouts of

D

steps at

\sigma=0.3

, with three seeds, on our

1{,}000

test puzzles. Costs are in blocks per puzzle. Pass@

K^{\prime}

counts a puzzle as solved when any of its

K^{\prime}

rollouts is correct. Lost counts the standard-pass successes a configuration loses, out of

859

, and is zero on every row of the bottom part, under the gate. The last columns are PTRM under the harness.

	own protocol				$+$ fallback
configuration	cost	accuracy (%)	pass@ $K^{\prime}$	lost	accuracy (%)	lost
TRM, standard pass	$1$	$68.83$	—	—	—	—
TRM, uniform unroll ( $D{=}48$ )	$3$	$68.18$	—	$10.0$	—	—
$K^{\prime}{=}1$ , $D{=}16$	$1$	$68.29$ $\,\pm 0.75$	$68.29$ $\,\pm 0.75$	$12.0$	$72.19$ $\,\pm 0.75$	$0.0$
$K^{\prime}{=}5$ , $D{=}16$	$5$	$74.13$ $\,\pm 0.82$	$74.35$ $\,\pm 0.86$	$3.0$	$75.00$ $\,\pm 0.32$	$0.3$
$K^{\prime}{=}10$ , $D{=}16$	$10$	$76.19$ $\,\pm 0.75$	$76.62$ $\,\pm 0.56$	$1.3$	$76.41$ $\,\pm 0.37$	$0.7$

Appendix

Table 8: The same configurations on PPBench validation. The table follows the conventions of Table 7 on the

308

validation puzzles, with

\sigma^{\star}=1.5

, and omits the standard deviation where the three seeds agree. Lost counts the losses among the

212

standard-pass successes and is zero on every row of the bottom part.

	the retry
	no perturbation	adds PTRM’s noise to $\mathbf{z}$	resamples $\mathbf{z}$	resamples $\mathbf{y}$
continues (depth)	$73.8$ / $15.6$	$70.7$ $\,\pm 2.7$ / $17.7$ $\,\pm 1.0$	$74.2$ $\,\pm 1.5$ / $19.1$ $\,\pm 1.2$	$88.7$ $\,\pm 1.9$ / $34.4$ $\,\pm 2.8$
restarts (width)	$0.0$ / $0.0$	$78.3$ $\,\pm 1.1$ / $24.7$ $\,\pm 2.2$	$78.5$ $\,\pm 2.5$ / $21.2$ $\,\pm 1.6$	$87.7$ $\,\pm 1.5$ / $39.9$ $\,\pm 2.2$

Appendix

Table 9: The values of Figure 2 . In each entry, the first percentage is for Sudoku-Extreme and the second for PPBench validation. The shaded entry is GAR, and the restart entry with PTRM’s noise is PTRM itself, with

K^{\prime}=16

rollouts of

D=16

steps under the harness.

	Sudoku-Extreme ( $141$ failures)		PPBench validation ( $96$ failures)
$\mathbf{z}$	$\mathbf{y}$ kept	$\mathbf{y}$ resampled	$\mathbf{y}$ kept	$\mathbf{y}$ resampled
kept	$104.0$	$\mathbf{125.0\pm 2.6}$	$15.0$	$\mathbf{33.0\pm 2.6}$
resampled, fillable cells	$104.7\pm 2.1$	$123.3\pm 2.5$	$18.3\pm 1.2$	$29.7\pm 2.9$
resampled, all cells	$102.0\pm 3.0$	—	$17.3\pm 0.6$	—
reset, fillable cells	$103.0$	$126.3\pm 2.1$	$15.0$	$31.7\pm 1.5$
reset, all cells	$101.0$	$126.7\pm 1.5$	$17.0$	$29.3\pm 2.1$

Appendix

Table 10: Perturbations of

\mathbf{z}

crossed with the resampling of

\mathbf{y}

, when continuing. Entries count the failures recovered after

16

continued retries, by what each retry does to

\mathbf{z}

(rows) and, at the block boundary, to the fillable cells of

\mathbf{y}

(columns). A reset returns

\mathbf{z}

\mathbf{z}_{\mathrm{init}}

. All cells means every grid cell, without the

P

prefix positions, and all positions includes the prefix. The noise is PTRM’s, at the scales of Table 2 . Bold marks the retry that resamples

\mathbf{y}

alone, and a dash a retry we did not run.

Sudoku-Extreme ( $1{,}000$ puzzles, $141$ failures)
	TRM, under the harness		PTRM		GAR		$\mathbf{z}$ instead of $\mathbf{y}$		continue instead of restart
$r$	accuracy (%)	cost	accuracy (%)	cost	accuracy (%)	cost	accuracy (%)	cost	accuracy (%)	cost
$1$	$89.70$	$1.14$	$90.57\pm 0.25$	$1.14$	$91.43\pm 0.32$	$1.14$	$91.23\pm 0.47$	$1.14$	$90.87\pm 0.15$	$1.14$
$2$	$91.80$	$1.24$	$92.63\pm 0.49$	$1.23$	$93.27\pm 0.74$	$1.22$	$93.10\pm 0.56$	$1.23$	$92.67\pm 0.21$	$1.23$
$5$	$93.70$	$1.46$	$95.00\pm 0.20$	$1.42$	$95.87\pm 0.31$	$1.39$	$95.30\pm 0.46$	$1.40$	$95.83\pm 0.12$	$1.41$
$10$	$95.50$	$1.73$	$96.37\pm 0.15$	$1.63$	$97.33\pm 0.25$	$1.55$	$96.47\pm 0.42$	$1.61$	$97.53\pm 0.06$	$1.57$

Appendix

Table 11: Accuracy and cost at seven retry budgets. The table extends Table 1 and adds the golden split at

16

and

64

retries. Accuracies (%) are means over three seeds, with the standard deviation shown where the seeds disagree, and costs are in blocks per puzzle. The standard pass scores

85.90\%

68.83\%

and

67.35\%

on the three splits.

Sudoku-Extreme ( $1{,}000$ puzzles, $141$ failures)
$r$	$\mathbf{y}$ minus $\mathbf{z}$	once minus noise	GAR minus PTRM	continue minus restart
$1$	$+0.20$ $[-0.60,+1.03]$	$+0.67$ $[-0.03,+1.37]$	$+0.87$ $[+0.07,+1.67]$	$-0.57$ $[-1.23,+0.10]$
$2$	$+0.17$ $[-0.60,+0.97]$	$+0.47$ $[-0.20,+1.13]$	$+0.63$ $[-0.10,+1.37]$	$-0.60$ $[-1.27,+0.07]$
$5$	$+0.57$ $[-0.23,+1.37]$	$+0.30$ $[-0.20,+0.80]$	$+0.87$ $[+0.10,+1.67]$	$-0.03$ $[-0.70,+0.63]$
$10$	$+0.87$ $[+0.13,+1.63]$	$+0.10$ $[-0.33,+0.53]$	$+0.97$ $[+0.30,+1.70]$	$+0.20$ $[-0.30,+0.70]$
$16$	$+1.30$ $[+0.60,+2.07]$	$+0.03$ $[-0.40,+0.47]$	$+1.33$ $[+0.67,+2.10]$	$+0.13$ $[-0.30,+0.57]$

Appendix

Table 12: Paired differences at the budgets of Table 11 . Each column gives the difference in accuracy points between two methods, with a

95\%

bootstrap interval over instances.

\mathbf{y}

minus

\mathbf{z}

is GAR minus the same restart with

\mathbf{z}

resampled instead of

\mathbf{y}

, and once minus noise is that restart minus PTRM, so that their sum gives GAR minus PTRM up to rounding. Continue minus restart is GAR’s continuation variant minus GAR. Differences are computed before rounding.

	resampling $\mathbf{y}$				resampling $\mathbf{z}$ , fillable cells				resampling $\mathbf{z}$ , all cells
step	$\cos\mathbf{y}$	$\cos\mathbf{z}$	cells	solved	$\cos\mathbf{y}$	$\cos\mathbf{z}$	cells	solved	$\cos\mathbf{y}$	$\cos\mathbf{z}$	cells	solved
$0$	$0.300$	$1.000$	—	$0.0$	$1.000$	$0.300$	—	$0.0$	$1.000$	$0.000$	—	$0.0$
$1$	$0.768$	$0.565$	$0.462$	$1.7$	$0.957$	$0.837$	$0.161$	$4.7$	$0.952$	$0.829$	$0.169$	$5.0$
$2$	$0.892$	$0.723$	$0.321$	$5.7$	$0.929$	$0.790$	$0.186$	$11.0$	$0.934$	$0.796$	$0.179$	$11.7$
$4$	$0.876$	$0.718$	$0.269$	$18.7$	$0.901$	$0.756$	$0.209$	$20.0$	$0.907$	$0.762$	$0.210$	$18.0$
$8$	$0.856$	$0.707$	$0.257$	$31.7$	$0.891$	$0.754$	$0.219$	$31.7$	$0.881$	$0.739$	$0.228$	$27.0$

Appendix

Table 13: How long a boundary perturbation survives, by state. For each perturbation, two copies of the continuation are run in lockstep from one standard pass on the

1{,}000

Sudoku-Extreme test puzzles, one perturbed at the block boundary and the other not. The step column counts supervision steps since the boundary, so step

16

ends the first retry. The cosines compare the two copies cell by cell and are averaged over grid cells and re-opened puzzles, and then over seeds. Cells gives the fraction of the fillable cells that the two copies decode differently, and solved counts the re-opened puzzles whose decoded grid is correct at that step, out of

140

				$16$ retries				$64$ retries
type	$n$	failures	standard pass (%)	GAR	$\mathbf{z}$ instead of $\mathbf{y}$	continue instead of restart	PTRM	GAR	$\mathbf{z}$ instead of $\mathbf{y}$	continue instead of restart	PTRM
sudoku	$99$	$56$	$43.43$	$74.07$	$59.93$	$69.36$	$60.94$	$85.86$	$64.98$	$78.11$	$67.00$
lightup	$55$	$4$	$92.73$	$96.97$	$94.55$	$96.36$	$96.36$	$97.58$	$94.55$	$97.58$	$96.36$
nurikabe	$59$	$12$	$79.66$	$81.36$	$81.36$	$82.49$	$82.49$	$81.36$	$81.36$	$82.49$	$83.05$
heyawake	$69$	$11$	$84.06$	$85.99$	$84.06$	$84.54$	$85.02$	$85.99$	$84.06$	$84.54$	$86.47$
tapa	$26$	$13$	$50.00$	$62.82$	$57.69$	$62.82$	$57.69$	$67.95$	$58.97$	$67.95$	$58.97$

Appendix

Table 14: PPBench per type, on the validation split. Accuracy (%) after

16

and

64

retries, averaged over three seeds. Methods are named and run as in Table 1 . Shakashaka is used in training but left out of the aggregate, as in Sghaier et al. (2026) . These results are descriptive, and we run no test per type.

state	method	runs on	cost	accuracy (%)	recovered	lost
—	TRM, standard pass	all	$1$	$67.35$	—	—
Depth: one trajectory, continued
—	TRM, uniform unroll ( $D{=}48$ )	all	$3$	$71.43$	$3.0$	$1.0$
—	no perturbation ( $R{=}16$ )	rejected	$5.16$	$75.51$	$4.0$	$0.0$
$\mathbf{y}$	resampling ( $R{=}16$ )	rejected	$4.14$	$78.23\pm 2.36$	$6.3$	$1.0$
Width: trajectories drawn from the initial state

Appendix

Table 15: The main operating points on the PPBench golden split. The split has

49

puzzles, and the standard pass solves

33

of them (

67.35\%

), leaving

16

failures. The table follows the conventions of Table 2 , and recovered counts the failures that end up correct. The depth row with

\mathbf{y}

resampled is the continuation variant of GAR, and the width row with

\mathbf{z}

resampled is “

\mathbf{z}

instead of

\mathbf{y}

”.

retries	method	sudoku	lightup	nurikabe	heyawake	tapa	aggregate
$16$	GAR	$71.11$	$100.00$	$88.89$	$80.95$	$70.00$	$80.27$
	PTRM	$55.56$	$100.00$	$88.89$	$71.43$	$60.00$	$72.11$
$64$	GAR	$75.56$	$100.00$	$88.89$	$80.95$	$70.00$	$81.63$
	PTRM	$64.44$	$100.00$	$88.89$	$71.43$	$60.00$	$74.83$

Appendix

Table 16: PPBench per type, on the golden split. Accuracy (%), averaged over three seeds, with PTRM run as in Table 1 . Lightup and nurikabe have no standard-pass failures, and both methods lose the nurikabe puzzle re-opened by the gate’s only false positive. These results are descriptive.

Explore similar work

Form Follows Function: Recursive Stem Model
Mar 3, 2026Navid HakimiNeural Algorithmic Reasoning Algorithmic Reasoning
Probabilistic Tiny Recursive Model
May 19, 2026Amin Sghaier, Ali Parviz, Alexia Jolicoeur-MartineauTest-Time Scaling
Mila – Quebec AI Institute ILLS & ETS Montreal · Independent
When Recursive Models Finish Computing
Sep 22, 2026Hare Krishna, Shubham Singh, Stephen Ebert +1Recurrent Neural Networks Algorithmic Reasoning
Weinberg Institute, Department of Physics, University of Texas at Austin · Department of Urban and Environmental Policy Planning, Tufts University · Zyphra Technologies, San Francisco, CA, USA +2

Mar 3, 2026J/K move · Enter open · S save

Form Follows Function: Recursive Stem Model

Navid Hakimi

Neural Algorithmic Reasoning Algorithmic Reasoning Test-Time Scaling

Open paper AskPDF arXiv

Recursive reasoning models such as Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM) show that small, weight-shared networks can solve compute-heavy and NP puzzles by iteratively refining latent states, but their training typically relies on deep supervision and/or long unrolls that increase wall-clock cost and can bias the model toward greedy intermediate behavior. We introduce Recursive Stem Model (RSM), a recursive reasoning approach that keeps the TRM-style backbone while changing the training contract so the network learns a stable, depth-agnostic transition operator. RSM fully detaches the hidden-state history during training, treats early iterations as detached "warm-up" steps, and applies loss only at the final step. We further grow the outer recursion depth

H

and inner compute depth

L

independently and use a stochastic outer-transition scheme (stochastic depth over

H

) to mitigate instability when increasing depth. This yields two key capabilities: (i)

>20\times

faster training than TRM while improving accuracy (

\approx 5\times

reduction in error rate), and (ii) test-time scaling where inference can run for arbitrarily many refinement steps (

\sim 20,000 H_{\text{test}} \gg 20 H_{\text{train}}

), enabling additional "thinking" without retraining. On Sudoku-Extreme, RSM reaches 97.5% exact accuracy with test-time compute (within ~1 hour of training on a single A100), and on Maze-Hard (

30 \times 30

) it reaches ~80% exact accuracy in ~40 minutes using attention-based instantiation. Finally, because RSM implements an iterative settling process, convergence behavior provides a simple, architecture-native reliability signal: non-settling trajectories warn that the model has not reached a viable solution and can be a guard against hallucination, while stable fixed points can be paired with domain verifiers for practical correctness checks.

Abstract

Explore similar work

Form Follows Function: Recursive Stem Model

Probabilistic Tiny Recursive Model

When Recursive Models Finish Computing