Looped refiners such as HRM and TRM apply one small network repeatedly to a draft answer, and deep supervision trains every pass toward one stored answer per puzzle. We prove that when a puzzle has many correct answers this target sets a loss floor that no model capacity removes, and that the loss-minimising pass blends the answers into an invalid grid. We propose the proximal target: train each pass toward the correct answer nearest to its own input, found by scanning a list of answers, by a search solver started at the draft, or within a band of quality set by a knob; correct answers become fixed points, and a pass only has to make fewer errors than its input has for the loop to converge, which its training loss bounds. With the same 1.7M-parameter network and 5 seeds, the proximal target solves 98.8% of Sudoku puzzles with 2-64 answers (standard target: 2.5%), 93.8% of sparse puzzles with more than 10,000 answers without any list of answers (0.0%), and 90.9% of 200-node 3-colourings after training on graphs of at most 50 nodes (36.9%), and different random starts end at different answers. Against GRAM, a stochastic-latent reasoner, a single run is correct far more often (98.8% vs. 57.7%) and twenty runs find more of the answers (53.6% vs. 37.4%), while GRAM is slightly ahead on its own 10-node colouring benchmark; in Scrabble the knob becomes a calibrated strength dial.
Figures & tables
Figure 1: Same network, same data, different target: training each pass toward the nearest correct answer removes blending. Left: our running example, a Sudoku with 37 clues and 9 correct answers. Top: trained with the standard target (one stored answer per puzzle), the refiner sends all 8 random starts to the same grid, which mixes answers; 9 cells break a rule (red), and 0 of 8 starts solve. Bottom: trained toward the correct answer nearest each pass’s input, all 8 starts solve and reach 6 different answers (four shown; blue marks cells that differ from answer 1). All grids are real outputs of the two trained models.
Figure 2: Each pass is trained toward a correct answer chosen from its own input, so different starts travel to different answers. (a) A pass reads the draft and writes a new one, 8 times, carrying a scratchpad. Its target is the nearest listed answer, the answer a solver started at the draft finds, or (with a knob k ) the nearest answer of quality near k . The standard target ignores the input. (b) Real runs on the running example, drawn on a map of cell distances. Stars on the rim: the 9 correct answers; a draft’s distance from the centre shows how close it is to its nearest answer. Our 8 runs start near the centre ( ∘ ) and travel to different answers; the standard model’s run stops between answers ( × ).
Figure 3: The proximal target solves where the standard target fails, on every task. Puzzles solved; dots are 5 training seeds, bars their mean. (a) Sudoku with 2–64 answers. (b) Sparse Sudoku with more than 10,000 answers, trained with the solver target and no answer list. (c) The benchmark of Nandwani et al. (2021) . (d) Planted 3-colouring trained on 20–50 nodes (shaded) and tested up to 200: the gap widens with size.
Method
Each pass is trained toward …
Params
Passes
Solved (%)
Found (%)
Standard target (TRM)
one stored answer
1.73M
8
2.5 ± 0.2
0.8 ± 0.0
4 × parameters
one stored answer
6.9M
8
2.4 ± 0.1
0.8 ± 0.0
+ test-time noise (PTRM-style)
one stored answer
1.73M
8
4.5 †
8.8
Solver’s first answer, fixed per puzzle
one predictable answer
1.73M
8
51.3 ± 0.7
8.3 ± 0.1
MinLoss ( Nandwani et al., 2021 )
answer nearest the output
1.73M
8
98.3 ± 0.3
14.7 ± 0.2
GRAM ( Baek et al., 2026 ) , KL 0.045
all answers, stochastic latent
9.1M
16
57.7 ± 5.7
37.4 ± 1.9
Table 1: Methods for many-answer puzzles on Sudoku with 2–64 answers (1,931 multi-answer test puzzles; mean ± sd over seeds). Solved: one run is correct. Found: share of each puzzle’s answers reached by 20 runs. “Passes” are applications of the network at test time (GRAM: adaptive steps of a nested recursion). † Share of noisy runs that are valid; with 20 runs, 24.4% of puzzles have at least one.
Figure 4: GRAM, run from its official code, against the proximal target (3 seeds each; dots are seeds). (a) On Sudoku with 2–64 answers, GRAM’s samples are diverse but a single one is often wrong. (b) On GRAM’s own 10-node colouring benchmark, where our runs of its code reproduce its published result, GRAM is slightly ahead.
Figure 5: The theory holds in the trained loops. (a) A pass’s errors against its own target, divided by its input’s distance to the nearest answer (mean of 3 seeds). Below 1, Proposition 1 guarantees progress: the nearest-answer loop stays below 1, fixed targets cross it by pass 3. (The solver target is not the nearest answer, so its ratio near 1 is a loose bound, not a stall; it solves 97%.) (b) Loss at pass 8 of standard-target models trained from blank or random starts, with a new random answer at every pass, or at 4 × size; all sit above the floor F of Lemma 1 . (c) Solved by number of answers per puzzle: fixed targets fail more as answers multiply; targets chosen from the input do not.
Figure 6: What matters is that the target depends on the pass’s input. Each point is a target rule (large dot: mean over seeds; small dots: seeds); colour shows what the target is chosen from.
Figure 7: A knob turns the target into a strength dial (Scrabble, 11×11 , 3 seeds). (a) Requested k against the rank of the move played: the knob follows from k≈0.2 up. (b) At equal average strength, the knob plays far fewer bottom-10% moves than temperature. (c) Games won against MAGPIE’s static-equity bot (2,000 per setting; 95% band). (d) One real position: at k=0.7 the knob aims at ranks 0.6–0.85, while temperature of the same mean rank spreads over all ranks, blunders included. The knob changes which moves are targets; temperature only adds noise.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Multi-answer puzzles
Unique
Model
Solved
Any of 20
Q-sel.
Found
Found, 2–4
Found, 17–64
Solved
GRAM, KL 0.045
57.7 ± 5.7
96.6 ± 0.4
65.7 ± 7.4
37.4 ± 1.9
70.6
20.8
64.5
GRAM, KL 0.1
44.6 ± 16.1
67.3 ± 11.0
51.6 ± 15.9
12.6 ± 1.7
39.5
3.1
88.9
Proximal (nearest answer)
98.8 ± 0.3
–
–
53.6 ± 0.4
–
–
98.6 ± 0.3
Appendix
Table 2: GRAM on Sudoku with 2–64 answers (test set, 20 samples, mean ± sd over 3 seeds). Q-selected: the sample chosen by GRAM’s Q head is correct. Distinct: different valid answers among 20 samples.
Variant
Solved (multi)
Solved (unique)
Answers found
Fixed pt.
standard target (blank start)
2.5 ± 0.2
96.3 ± 0.4
0.8 ± 0.0
0.1 ± 0.1
nearest to input (ours)
98.8 ± 0.3
98.6 ± 0.3
53.6 ± 0.4
100.0 ± 0.0
standard, random start
4.3 ± 2.9
94.1 ± 3.9
1.7 ± 1.2
0.1 ± 0.1
standard, new random answer every pass
3.2 ± 1.5
95.5 ± 1.5
1.3 ± 0.4
0.1 ± 0.0
standard, 4 × parameters (6.9M)
2.4 ± 0.1
96.6 ± 0.2
0.8 ± 0.0
0.1 ± 0.0
solver’s first answer, fixed per puzzle
51.3 ± 0.7
96.9 ± 0.2
8.3 ± 0.1
0.3 ± 0.1
Appendix
Table 3: Sudoku with 2–64 answers: all variants. Mean ± sd over 5 seeds (first two rows) or 3. Fixed pt.: one pass with a fresh scratchpad returns a correct draft unchanged. The solver-target model fails that test, and starting 10% of its training runs at a correct answer fixes it.
Target
Solved (multi)
Answers found
Fixed pt.
Unlisted fixed pt.
nearest listed, m=1
6.1 ± 2.2
2.5 ± 0.8
0.0 ± 0.0
–
nearest listed, m=2
27.7 ± 3.0
9.1 ± 0.8
0.8 ± 0.0
0.1 ± 0.1
nearest listed, m=5
75.9 ± 1.5
30.0 ± 4.4
49.8 ± 43.6
44.8 ± 47.3
verified, m=2
77.0 ± 2.5
21.8 ± 0.7
16.6 ± 2.1
10.6 ± 2.3
all answers listed (reference)
98.8 ± 0.3
53.6 ± 0.4
100.0 ± 0.0
–
Appendix
Table 4: Incomplete answer lists bring blending back; a rules checker recovers most of it. m randomly chosen answers kept per training puzzle (all answers count at test time). Verified: the draft itself if it obeys the rules, else the nearest listed answer. Unlisted fixed pt.: answers never used as targets.
Target
Ratio, pass 1 / 4
Loss
Asked
V8
Fixed pt.
Warm
standard
0.30 / 2.29
43.4
13.57
6.68
0.001
0.87
standard, 4 × parameters
0.29 / 2.41
42.7
13.41
6.43
0.000
0.92
solver’s first answer (fixed)
0.32 / 2.95
32.7
8.65
2.08
0.002
0.93
nearest to output (MinLoss)
0.27 / 0.41
1.0
0.36
0.33
0.004
1.00
solver from output
0.27 / 0.39
0.8
0.28
0.25
0.003
1.00
solver from input
0.33 / 0.97
1.6
0.49
0.38
0.026
1.00
Appendix
Table 5: Measured theory quantities (1,000 multi-answer puzzles × 4 starts, mean of 3 seeds). Ratio: the pass’s errors against its own target over the input’s distance V to the nearest answer (below 1 guarantees progress). Loss: bits per puzzle at pass 8 (floor of Lemma 1 : 35.5). Asked: d(st,yt) at pass 8. V8 : distance to the nearest answer before the last pass. Warm: keeps its answer within a run.
Variant
Solved
Distinct (of 20)
Fixed pt. (fresh)
standard target (5 seeds)
0.0 ± 0.0
0.0 ± 0.0
0.0 ± 0.0
solver’s first answer, fixed per puzzle
37.3 ± 4.7
0.6 ± 0.1
0.0 ± 0.0
solver started from output
99.4 ± 0.2
10.0 ± 0.4
0.0 ± 0.0
solver started from input (ours; 5 seeds)
93.8 ± 0.7
15.1 ± 0.1
15.7 ± 1.7
Appendix
Table 6: Sparse Sudoku (more than 10,000 answers): the solver started from the input gives the most distinct answers. Distinct: different correct answers among 20 starts. Fixed pt. (fresh): the fixed-point test on answers never used as targets.
Variant
n=30
n=50
n=100
n=200
n=200 , 64
Distinct
standard (5 seeds)
71.9 ± 1.5
71.9 ± 3.1
57.8 ± 3.9
36.9 ± 5.6
37.7 ± 5.4
0.9 ± 0.3
solver from input (ours; 5 seeds)
99.0 ± 0.4
98.3 ± 0.4
96.5 ± 0.4
90.9 ± 1.0
91.8 ± 1.1
9.2 ± 0.1
ours, separate weights
97.7 ± 0.4
98.4 ± 0.4
95.9 ± 0.6
93.5 ± 0.5
89.6 ± 6.4
8.9 ± 0.5
rule penalty only (no target)
79.1 ± 3.4
75.5 ± 5.1
54.9 ± 8.4
31.6 ± 7.2
35.5 ± 9.9
3.2 ± 1.2
no loop, 10 blocks (8.5M)
48.9 ± 0.9
45.0 ± 1.4
16.8 ± 1.2
3.1 ± 0.2
3.1 ± 0.2
0.3 ± 0.0
Appendix
Table 7: Planted 3-colouring: the solver target generalises from 20–50 to 200 nodes. Solved by graph size n (8 passes; “ n=200 , 64”: 64 passes at test time), and distinct colourings among 20 starts at n=200 .
Mean degree
Pre-coloured
Standard, 8 passes
Ours, 8 passes
Ours, 32 passes
Distinct (std / ours)
3.5
0%
0.0 ± 0.0
84.8 ± 7.2
87.5 ± 3.5
0.0 / 8.8
3.5
5%
2.2 ± 0.3
84.9 ± 1.8
86.1 ± 2.2
0.0 / 8.7
3.5
15%
35.0 ± 0.9
91.9 ± 1.1
92.0 ± 2.3
0.6 / 9.2
4.0
0%
0.0 ± 0.0
79.4 ± 16.6
93.0 ± 5.1
0.0 / 9.3
4.0
5%
15.4 ± 0.5
87.3 ± 8.1
93.8 ± 3.1
0.4 / 9.4
4.0
15%
58.3 ± 1.5
95.8 ± 0.7
96.2 ± 1.0
0.9 / 9.7
Appendix
Table 8: Harder colouring families: the solver target works everywhere; the standard target needs clues. 100-node planted graphs by mean degree and pre-coloured share (models trained at degree 4, 15%); 3 seeds, 500 graphs each. Distinct: valid colourings among 10 starts, counted up to relabelling of colours when nothing is pre-coloured.
Variant
Solved (multi)
Found
Solved (unique)
standard target (5 seeds)
6.3 ± 4.0
2.3 ± 1.0
96.7 ± 3.4
nearest to input (5 seeds)
89.3 ± 1.1
49.4 ± 1.4
99.0 ± 0.2
verified target
99.0 ± 0.1
52.2 ± 1.4
98.9 ± 0.1
Appendix
Table 9: Left: the benchmark of Nandwani et al. (2021) ; the verified target removes most of the loss from its incomplete lists. Right: the N-Queens audit; no method transfers to new solution families. Found: answers found by 20 starts.
Sudoku
T=8
T=100
T=1,000
T=2,000
Unique, T=2 k
solved
0.1 ± 0.1
35.6 ± 0.6
92.5 ± 0.5
97.5 ± 0.2
95.9 ± 0.5
answers found
64.3 ± 0.3
solved, 3 heads
0.2 ± 0.1
35.4 ± 0.7
91.8 ± 1.0
97.3 ± 0.5
94.4 ± 1.4
solved, 8-pass training
0.0 ± 0.0
Colouring
30 nodes, T=8
30, T=2 k
200, T=8
200, T=2 k
solved
5.0 ± 0.2
89.3 ± 1.9
0.0 ± 0.0
74.8 ± 5.0
Appendix
Table 10: ConsFormer needs hundreds to thousands of passes. Solved by passes at test time T ; answers found on Sudoku at T=1,000 .
Figure 8: The loop is a weak repairer, iterated. (a) The running example, one start, pass by pass. Green: cells this pass fixed; red: cells it broke; amber outline: still wrong. (b) Test set (3 seeds): cells fixed vs. broken per pass (log scale). (c) Net share of the remaining wrong cells fixed per pass: shared weights keep repairing; separate weights per pass stop by pass 5; without the scratchpad, repair fades. (d) The nearest answer switches in 61%, 24%, 8% and 3% of runs after passes 1–4.
Figure 9: A search solver replaces the answer list. (a) A sparse puzzle (19 clues, more than 10,000 answers). Amber marks the cells that a depth-first solver, started from the current draft, would change; its answer is the target for the next pass. The corrections shrink from 54 cells to 21, 8 and 0. (b) Test set, solved and distinct answers of 20 starts, by target.
Figure 10: A rule-penalty refiner and a short trained loop work in different regimes. Solved against passes at test time: ConsFormer (orange) and ours at 8 passes (blue). (a) Sudoku; (b) colouring, 30 and 200 nodes.
Figure 11: The loop converges because its targets are fixed points, not because the map shrinks. Inside the trained weights of the nearest-answer model on Sudoku (3 seeds). (a) Removing one attention head at a time: one head (a different one per seed) is essential (solved drops to 19 / 8 / 14%); 5 of 16 heads matter (solved below 90%). (b) 8-, 6- and 4-bit weights keep 97.6–99.6% solved; 3-bit weights and halving every weight matrix’s rank break it. (c) The largest stretch σ1 of one pass’s latent map falls from about 60 at pass 1 to 10–15 at pass 7 but stays far above 1: the loop is not a contraction.
Figure 12: Sampling the early passes raises answers found at no cost in validity. The draft is sampled at temperature τ in a window of passes (3 seeds, mean). Sampling in the first passes raises answers found from 53.9% to 59.6% ( τ=1.5 , passes 1–4); sampling only in passes 2–3 helps little. Solved stays 98.5–99.1% throughout. Diversity is decided where the answer is chosen: in the first passes.