cs.AIOct 19, 2026
SaveSpherical Fixed-Point Reasoners: Adaptive Loop Halting on the Hypersphere
Organizations: EPFL · Lamarr Institute · Fraunhofer IAIS · Amazon AGI
Abstract
Looped (weight-tied) transformers enable adaptive computation by allocating more iterations to harder inputs. Fixed-Point Reasoners (FPRM) achieve this with a damped fixed-point solver, but operate in an unbounded Euclidean residual space with coupled rescaling and relative-norm halting. We introduce Spherical Fixed-Point Reasoners (SFPRM), which perform the same computation on the unit hypersphere of normalized transformers (nGPT). Input injection, damping, and halting become geodesic operations, yielding boundedness by construction and eliminating coefficient coupling. We prove fixed-point existence, local convergence under an explicit step-size condition, and global convergence under strong injection. On Sudoku-Extreme, Maze-Hard, and Mini-ARC, SFPRM achieves 96.6%, 87.9%, and 59.1% test accuracy, respectively, outperforming FPRM and other baselines while allocating test-time compute more effectively. Its converged geometry is also interpretable: token-input angles capture puzzle structure and predict correctness.
Figures & tables
Figure 1: SFPRM overview. (a) Each iteration interpolates the current state toward the normalized input ( ), applies the weight-tied spherical nGPT block to obtain the proposal , and rotates a fraction of the way toward it ( ); the loop halts once the worst-token chordal residual falls below . The eigen-learning-rate retractions inside the block keep every token state on unit sphere . (b) One token’s iterates rotate along a great circle by shrinking angles ; the halting test compares each proposal with the current iterate, so it never needs the fixed point .
| Method | Recurrence | Fix. pt. | State update and input | Halting / output |
| HRM | Fast -loop slow -loop | Appr. | Euclidean ; input drives | ACT head; read |
| TRM | Shared net refines , answer | No | Euclidean; input re-read each step | ACT head; read |
| URM | UT; fixed inner ACT outer loop | No | Euclidean; short conv in MLP | ACT head, steps |
| EqR | Tied loop; stochastic restarts | Appr. | Euclidean; random init, path noise | ACT; lowest-residual run |
| Attractor | Backbone equilibrium loop | Yes | Backbone seeds each step | Solver tol.; read |
| FPRM | Tied Transformer loop | Yes | mix; injection | Rel.- residual |
Table 1: Recent recurrent reasoners: HRM ( Wang et al., 2025 ) , TRM ( Jolicoeur-Martineau, 2025 ) , URM ( Gao et al., 2025 ) , Attractor ( Fein-Ashley and Rashidinejad, 2026 ) , EqR ( Huang et al., 2026 ) and FPRM ( Movahedi et al., 2026 ) . “Appr.” means no numerical fixed-point solve is performed.
Figure 2: Proposals accepted by each halting test around a token state . The chordal test accepts a disk; FPRM’s accepts an axis-aligned section, so two proposals at the same angle can be judged differently.
Figure 3: Exact accuracy and halting on Maze-Hard (Sudoku: fig. 10 ). SFPRM vs. FPRM evaluation on test puzzles. Top: exact accuracy; mean halt and number of examples halting. Bottom: per-sequence residual (thin), median (heavy).
| Model | Architecture | # params. | Single Loop (No Hier.) | Sudoku-Ext. Pass@1 | Maze-Hard Pass@1 | Mini-ARC Pass@1 | ARC-1 Pass@2 | |
| Larger | HRM | rec. Tf | 27M | ✗ | 55.0% | 74.5% | 44.9% (51.7%) | 40.3% |
| Attractor Model | Tf FP loop | 27M | ✗ | 91.4% | 93.1% | — | — | |
| URM | rec. Tf conv | 14M | ✗ | 77.6% | — | — | — | |
| – M | TRM | rec. Tf | 7M | ✗ | 74.7% | 85.3% | 48.4% (36.9%) | 44.6% |
| EqR | rec. Mixer restarts | 5-7M | ✗ | 93.0% | — | 55.3% (40.3%) | — | |
| Attractor Model | FP Tf | 7M | ✗ | 54.3% | 46.7% | — | — |
Table 2: Comparison with published recurrent reasoners. Reported numbers are taken from the respective papers; Mini-ARC numbers are from Huang et al. (2026) (Table 14). Numbers in parentheses are our own runs under the same setting as SFPRM. Tf Transformer, Mixer MLP-Mixer, rec. recurrent refinement, FP fixed-point solve. Within the – M group the best result is bold and the second best underlined.
Figure 4: Global convergence under input injection (SFPRM, Sudoku). (a) Largest angle between the end states of eight initializations, over puzzles, against (teal: all coincide, coral: end states); indigo, right axis: token accuracy. (b) Trajectories from the eight (squares) to (star), coloured by the token accuracy of the decoded state; circles mark the first iteration with , dashed red the exact-match region.
Figure 5: Fixed-point solvers (SFPRM, Maze). (a) Accuracy, halting disabled. (b) Radius is the geodesic angle (log-scaled) to , rings at .
Figure 6: Anchor angles by cell class on Maze : cells (thin lines), class means (heavy), the halt iteration (vertical line), and the final distribution (margin).
Figure 7: SFPRM rollout frames (FPRM in Figures 21 and 22 ). Top row : the board decoded from ; bottom row : each cell’s angle to its own input anchor. left : Sudoku-Extreme; right : Maze-Hard.
Figure 8: Rollout frames on Mini-ARC for FPRM (left) and SFPRM (right). Input (top) and target (bottom), then iterations to : decoded grid ( : wrong cell) and angle to the input anchor, . Only SFPRM solves this puzzle, and its angle map singles out the cells the task rewrites.
Figure 9: Final anchor angle vs. correctness on Sudoku puzzles: angle (SFPRM) and state norm (FPRM) of clue and blank cells, for solved (filled) and unsolved (hatched) puzzles.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Maze-Hard | Sudoku-Extreme | ARC-AGI-1 | Mini-ARC | |
| Data and schedule | ||||
| base puzzles augmentations | ||||
| sequence length / vocabulary | / | / | / | / |
| test set | mazes | puzzles | tasks | tasks |
| epochs | ( ) | |||
| optimizer steps | ( ) | |||
Table 3: Hyperparameters and learned scalars of the SFPRM runs in Section 4 . Sudoku-Extreme values are for the epoch- run, while values in parentheses are for the -epoch constant-schedule run. All convolutions are depthwise and bias-free. For the per-channel step sizes and the per-dimension QK scale , cells give the mean for layer 1 / layer 2, and the range row gives the minimum and maximum over all channels of both layers. On ARC, a linear Q-head on a detached state copy only breaks ties in Pass@ .
| Model | Setting | Sudoku-Ext. |
| FPRM | Sudoku | 78.5% |
| FPRM | Sudoku | 78.4% |
| SFPRM | Sudoku (best checkpoint) | 87.0% |
| SFPRM | Sudoku (final checkpoint) | 12.2% |
| SFPRM | Sudoku | 92.8% |
Table 4: Exact accuracy on Sudoku-Extreme for SFPRM vs. FPRM at matched backbone ( M trainable parameters), data, global batch ( ), and optimizer steps/epochs ( ). “ ” and “ ” are the number of distinct base puzzles times the augmentations per puzzle. The SFPRM run is given at both its best and checkpoint because it collapses between them, which the build fixes. For the run the final checkpoint is also the best. The FPRM rows fall well below the reported for FPRM in Table 2 .
Figure 10: Accuracy and halting on Sudoku-Extreme and Mini-ARC. Open-loop rollouts of FPRM (amber) and SFPRM (blue). (a) Sudoku test sequences of the released FPRM checkpoint and of SFPRM with the learned (left) and with fixed at (right, the run in Table 2 ). (b) All Mini-ARC test rows ( tasks, augmentations each) over iterations, for the runs in Table 2 . Top: exact accuracy of the decoded state (a) and task-level Pass@1 (b). Dotted lines and stars mark each model’s mean halt iteration, labelled with how many sequences halt. Bottom: every sequence’s halting residual (thin) and its median (heavy) against its own threshold ( chordal for SFPRM, for FPRM).
Figure 11: Global convergence under input injection on Maze-Hard (SFPRM, learned ), analogue to Figure 4 . (left) The largest angle between the end states of eight initializations of the raw map (teal: all coincide, coral: end states), and the token accuracy of the end state (indigo, right axis), as a function of . (right) Trajectories at four values of for a single Maze example: the top-2 principal plane of the trajectories from the eight initializations (squares) through (star; circles: first iteration with ). The color shows the token accuracy of the state decoded at each position (dashed red: exact-match region).
Figure 12: Learned parameters of SFPRM during training. Per task, top row: injection fraction , logit scale , and QK scale (mean over its 64 dimensions, per layer). Bottom row: the eigen learning rates (attention) and (MLP) of both layers, as the mean, minimum and maximum over the 512 hidden dimensions.
Figure 13: Final anchor angle on solved and unsolved mazes (Maze-Hard analogue of Figure 9 ). Per-class angle to the anchor at the end of the rollout over test mazes ( solved, unsolved), for (a) SFPRM and (b) FPRM’s normalized state. Filled: solved; hatched: unsolved. Unlike on Sudoku, the distributions overlap for every class, so the final angle does not predict correctness.
Figure 14: One SFPRM trajectory per cell class, on the sphere (Sudoku left, Maze right). For every cell class we draw the token whose -frame captures the most trajectory variance. The frame is spanned by the trajectory’s two leading singular directions and the anchor , which sits at the pole. is nearly orthogonal to the plane of the trajectory ( ). Arcs are great circles between projected iterates. The colour shows the iteration index.
Figure 15: Anchor angles by cell class on Sudoku-Extreme . Four test puzzles (Maze in Figures 6 and 16 ). In each pair FPRM is above and SFPRM below. Thin lines are individual cells, thick lines class means, the vertical line the halt iteration, and the margin the distribution at the end of the rollout. SFPRM rotates the clues to from their anchors within about iterations and leaves the blanks near orthogonal, whereas FPRM’s normalized states settle within the first few iterations with clues and blanks only apart.
Figure 16: Anchor angles by cell class on Maze-Hard . Four test puzzles, FPRM above and SFPRM below in each pair, with the conventions of Figure 15 . On every maze SFPRM settles walls at , free cells that stay empty at , free cells that become path at , and start and goal at , whereas FPRM keeps all classes within – and separates the two free-cell classes by only a few degrees.
| Halting | Exact at halt (%) | Open loop (%) | |||||||
| Solver | % | med. | mean | evals | all | halted | peak ( ) | ||
| SFPRM, Maze ( , ) | |||||||||
| Adaptive (default) | 99.2 | 27 | 96.3 | 104.5 | 87.8 | – | 87.9 | 87.8 (20) | 87.8 |
| Fixed, | 63.1 | 23 | 25.8 | 386.3 | 87.8 | 1.00 | 87.3 | 87.8 (20) | 87.8 |
| Anderson, | 100.0 | 13 | 13.8 | 14.8 | 86.8 | 0.04 | 86.8 | 87.4 (19) | 86.7 |
| Broyden, | 99.4 | 14 | 14.0 | 20.9 | 87.4 | 0.39 | 87.9 | 87.9 (16) | 51.5 |
Table 5: Inference-time fixed-point solvers on the trained models. Every example runs steps from its own random start, shared by the four solvers; each step is one evaluation of , and an example halts at the first iterate with residual below . Halting : share of examples that halt (%), median and mean halting step over those, and mean evaluations over all examples. Exact at halt : accuracy over all examples, exact binomial test against the default solver ( ), and accuracy among halted examples. Open loop : accuracy with halting disabled, at its best step and at . Maze: all test mazes; Sudoku: puzzles evenly spaced over the test set. Best within each block: exact accuracy , share halted , fewest halting steps .
Figure 17: Accuracy and halting residual over the iterations for the remaining settings of Table 5 . SFPRM on Sudoku ( , left), FPRM on Maze (middle) and FPRM on Sudoku (right); SFPRM on Maze is Figure 5 . Same examples and solvers as the table. Top: exact accuracy of the iterate with halting disabled. Bottom: each model’s halting residual, median (solid) and 75th percentile (dotted) over the examples, its threshold (dashed), and the iteration by which half of the examples have halted (circle).
Figure 18: The four solvers in the principal plane through the fixed point , for test example of Maze-Hard (top, ) and of Sudoku-Extreme (bottom, ). Axes and markers as in Figure 4 b, with the end point of Anderson acceleration; every solver runs from the same eight initializations (squares) and six further starts placed on its plane. Each panel uses the top-2 principal directions of that solver’s own iterates, so the background landscapes differ between panels.
Figure 19: SFPRM on Maze-Hard: State trajectories in polar coordinates. The four solvers (rows, with each one’s halting step) on the maze of Figure 5 b, one token per cell class (columns). Radius: geodesic angle to the reference fixed point, log-scaled from at the centre to at the rim, rings at , and ; azimuth: the top-2 principal plane of that token’s component orthogonal to the reference; square: start, dot: end of the -iteration run; colour: iteration. Anderson ends at the centre, the default and fixed steps orbit at a small radius, and Broyden leaves for the rim.
Figure 20: SFPRM on Sudoku-Extreme: State trajectories in polar coordinates. A puzzle on which the default solver halts at its median step ( ); columns : the middle clue cell and the blank cells whose decode settles earliest, at the median and last. Anderson and Broyden reach the tolerance early, the default solver orbits for hundreds of iterations before halting, fixed never halts, and Broyden fills the disc after leaving.
| Variant | Pass@1 | |
| SFPRM (full) | 59.1 | – |
| w/o QK normalization | 26.2 | |
| w/o damping ( ) | 47.7 | |
| Fixed injection fraction ( ) | 49.7 | |
| ACT halting instead of chordal residual | 54.4 | |
| Fixed eigen learning rates ( ) | 55.0 |
Table 6: SFPRM ablations on Mini-ARC. Pass@1 accuracy at an iteration cap of 200. Each row changes one component of the full model. : change relative to the full model.
Figure 21: Filmstrip gallery on Sudoku-Extreme. Test puzzles – . One test puzzle per row, rolled open-loop for iterations and rendered at (protocol in Appendix C.4 ), for FPRM (left) and SFPRM (right): decoded board on top, each cell’s angle to its input anchor below.
Figure 22: Filmstrip gallery on Maze-Hard. Test puzzles – . One test maze per row, rolled open-loop for iterations and rendered at (protocol in Appendix C.4 ), for FPRM (left) and SFPRM (right): decoded path on top, each cell’s angle to its input anchor below.
Figure 23: Filmstrip gallery on Mini-ARC. One test task per row, rolled out open-loop for iterations. Left: FPRM; right: SFPRM. Each row starts with the test input (top) and its target (bottom), followed by five iterations from to . At each iteration the decoded grid is shown on top, with marking cells that differ from the target, and each cell’s angle to its input anchor, , below.
Figure 24: Filmstrip gallery on Mini-ARC (continued). Same caption as in Figure 23 .