Dynamical Systems Theory Behind a Hierarchical Reasoning Model

Vasiliy A. Es'kin; Mikhail E. Smorkalov

cs.AIMar 24, 2026

Dynamical Systems Theory Behind a Hierarchical Reasoning Model

Authors: Vasiliy A. Es'kin, Mikhail E. Smorkalov

Organizations: Department of Radiophysics, University of Nizhny Novgorod, Nizhny Novgorod 603950, Russia · Huawei Nizhny Novgorod Research Center, Nizhny Novgorod 603006, Russia · Skolkovo Institute of Science and Technology, Moscow 121205, Russia

Abstract

Current large language models (LLMs) primarily rely on linear sequence generation and massive parameter counts, yet they severely struggle with complex algorithmic reasoning. While recent reasoning architectures, such as the Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM), demonstrate that compact recursive networks can tackle these tasks, their training dynamics often lack rigorous mathematical guarantees, leading to instability and representational collapse. We propose the Contraction Mapping Model (CMM), a novel architecture that reformulates discrete recursive reasoning into continuous Neural Ordinary and Stochastic Differential Equations (NODEs/NSDEs). By explicitly enforcing the convergence of the latent phase point to a stable equilibrium state and mitigating feature collapse with a hyperspherical repulsion loss, the CMM provides a mathematically grounded and highly stable reasoning engine. On the Sudoku-Extreme benchmark, a 5M-parameter CMM achieves a state-of-the-art accuracy of 93.7 %, outperforming the 27M-parameter HRM (55.0 %) and 5M-parameter TRM (87.4 %). Remarkably, even when aggressively compressed to an ultra-tiny footprint of just 0.26M parameters, the CMM retains robust predictive power, achieving 85.4 % on Sudoku-Extreme and 82.2 % on the Maze benchmark. These results establish a new frontier for extreme parameter efficiency, proving that mathematically rigorous latent dynamics can effectively replace brute-force scaling in artificial reasoning.

Figures & tables

Figure 2: Architecture of hierarchical reasoning model.

Figure 3: Diagram of training of HRM (a) and TRM (b).

Figure 4: Pseudocode of training of the HRM (left) and TRM (right).

Figure 5: Evolution of the phase portrait of the dynamical system ( 20 ), ( 21 ) during neural network training.

Figure 6: Integration of the system of differential equations ( 20 )–( 21 ) under the trained neural network, which determines the landscape of phase space. Green and red arrows correspond to the integration stages using the $L$ - and $H$ -modules, respectively.

Figure 7: Illustration to repulsion loss term. (a) is initial hidden representation, (b) is projection to the surface of a unit hypersphere, (c) is projection after using repulsion loss term, (d) is hidden representation after using repulsion loss term.

Figure 8: (a) represents the integration of the system of differential equations ( 20 ) during training according to the descriptions in [ 1 , 7 ] ; (b), (c), and (d) represent the integration of the system of differential equations ( 20 ) during training according to the official code for these papers. Panels (b), (c), and (d) correspond to batches $i$ , $i+1$ , and $i+2$ , respectively.

Method	Accuracy %	Number of	Features
		Parameters
DeepSeek R1 [ 1 ]	0.0	671B	–
HRM [ 1 ]	55.0	27M	–
TRM [ 7 ]	87.4	5M	MLP-mixer
TRM [ 7 ]	74.7	7M	Attention
HRM (our run)	45.9	27M	AMP

Table 1: Test accuracy on Sudoku-Extreme benchmark. Hidden size of HRM and TRM is 512.

Method	Accuracy %	Number of	Features
		Parameters
DeepSeek R1 [ 1 ]	0.0	671B	–
HRM [ 1 ]	74.5	27M	–
TRM [ 7 ]	85.3	7M	Attention
TRM (our run)	84.2	7M	AMP
(gradient accumulation on 10 segments)

Table 2: Test accuracy on Maze benchmark. Hidden size of HRM and TRM is 512.

Features	Accuracy %
$B=256$	69.6
SiLU to $\tanh$ , $B=256$	81.8
SiLU to $\tanh$ , $B=64$ , $N_{\rm grad}=4$ ,
using the all states of the $H$ -module for gradient calculation	56.8

Table 3: Test accuracy of TRM on Sudoku-Extreme benchmark. General features:

D=512

, AMP,

N_{\rm super}=16

N_{\rm H}=3

N_{\rm L}=6

MLP-mixer, number of parameters is 5M,

N_{\rm accum}=N_{\rm super}

$N_{\rm H}$	$N_{\rm L}$	Accuracy %	Features
1	2	58.9	$\tanh$ is activation function
1	2	63.8
1	4	68.9
1	6	68.7
2	1	64.6
2	2	70.5

Table 4: Dependence of accuracy of

\text{CMM}^{\text{NODE}}_{512}

N_{\rm H}

and

N_{\rm L}

for Sudoku-Extreme benchmark. General features:

D=512

, AMP,

N_{\rm super}=16

N_{\rm grad}=1

B=256

, MLP-mixer, number of parameters is 5M, SiLU is activation function for main part of experiments, StableMax is loss for main part of experiments.

Features	Accuracy %
$N_{\rm accum}=3$ , $N_{\rm grad}=1$	79.2
$N_{\rm accum}=2$ , $N_{\rm grad}=1$ ,
using the all states of the $H$ -module for gradient calculation	74.6
$N_{\rm accum}=2$ , $N_{\rm grad}=1$	78.8
$N_{\rm accum}=2$ , $N_{\rm grad}=3$	81.8
$N_{\rm accum}=2$ , $N_{\rm grad}=4$	81.3

Table 5: Test accuracy of

\text{CMM}^{\text{NODE}}_{512}

on Sudoku-Extreme benchmark. General features:

D=512

B=256

, AMP,

N_{\rm super}=16

N_{rmH}=2

N_{rmH}=4

, StableMax3, equilibrium in

\hat{\bf x}

and

\hat{\bf z}_{\rm H}

, MLP-mixer, number of parameters is 5M.

Features	Accuracy %
$B=256$ , $N_{\rm grad}=3$ , without repulsion for $\hat{\bf x}$	88.0
$B=128$ , $N_{\rm grad}=6$	89.9
$B=256$ , $N_{\rm grad}=3$ , $\tanh$ is activation function	79.4
$B=256$ , $N_{\rm grad}=3$	90.1
$B=384$ , $N_{\rm grad}=3$	89.9
$B=250$ , $N_{\rm grad}=4$	90.5

Table 6: Test accuracy of

\text{CMM}^{\text{NODE}}_{512}

on Sudoku-Extreme benchmark. General features:

D=512

, AMP,

N_{\rm super}=16

N_{\rm accum}=2

N_{\rm H}=3

N_{\rm L}=6

, StableMax3, equilibrium in

\hat{\bf x}

and

\hat{\bf z}_{\rm H}

, repulsion for

\hat{\bf z}_{\rm H}

and

\hat{\bf x}

, MLP-mixer, number of parameters is 5M, learning rate is

10^{-4}

Features	Accuracy %
$N_{\rm super}=8$ , learning rate $3\times 10^{-4}$	87.4
$N_{\rm super}=16$ , learning rate $2\times 10^{-4}$	91.7
$N_{\rm super}=16$ , learning rate $10^{-4}$	91.1
$N_{\rm super}=16$ , $N_{\rm accum}=16$ , $N_{\rm grad}=64$
learning rate $2\times 10^{-4}$	93.7
$B=1000$ , $N_{\rm super}=16$ , $N_{\rm accum}=16$ , $N_{\rm grad}=16$

Table 7: Test accuracy of

\text{CMM}^{\text{NSDE}}_{512}

on Sudoku-Extreme benchmark. General features:

D=512

B=250

N_{\rm grad}=4

, AMP,

N_{\rm accum}=2

N_{\rm H}=3

N_{\rm L}=6

, StableMax3, equilibrium in

\hat{\bf z}_{\rm H}

, repulsion for

\hat{\bf z}_{\rm H}

and

\hat{\bf x}

, MLP-mixer,

\sigma=0.01

Features	Accuracy %
$N_{\rm super}=16$ , $N_{\rm grad}=64$ , $\lambda_{\rm LM}=2$	80.2
$N_{\rm super}=2$ , $N_{\rm grad}=8$ ,
the optimizer Adam: epsilon is $10^{-14}$ , weight decay is 2	89.8
$N_{\rm super}=3$ , $N_{\rm grad}=12$ ,
the optimizer Adam: epsilon is $10^{-14}$ , weight decay is 2	91.7

Table 8: Test accuracy of

\text{CMM}^{\text{NSDE}}_{512}

on Sudoku-Extreme benchmark with gradient normalization (AlgGradNorm). General features:

D=512

B=250

, AMP,

N_{\rm accum}=16

N_{\rm H}=3

N_{\rm L}=6

, StableMax3, equilibrium in

\hat{\bf z}_{\rm H}

, repulsion for

\hat{\bf z}_{\rm H}

and

\hat{\bf x}

, MLP-mixer,

\sigma=0.01

, weight decay is 1, learning rate

2\times 10^{-4}

, .

$N_{\rm H}$	$N_{\rm L}$	Features	Accuracy %	Number of
				Parameters
2	4	$\text{CMM}^{\text{NODE}}_{128}$ , $B=256$	20.3	0.52M
3	6	TRM , $B=250$ , $N_{\rm grad}=4$ , $N_{\rm super}=8$ , Attention	23.8	0.52M
3	6	$\text{CMM}^{\text{NODE}}_{128}$ , $B=250$ , $N_{\rm grad}=4$ ,
		learning rate is $3{\times}10^{-4}$ , Attention	31.4	0.52M
2	4	$\text{CMM}^{\text{NSDE}}_{128}$ , $B=250$

Table 9: Dependence of accuracy of CMM on

N_{\rm H}

and

N_{\rm L}

for Sudoku-Extreme benchmark. General features:

D=128

, AMP,

N_{\rm super}=16

N_{\rm accum}=2

N_{\rm grad}=3

, MLP-mixer, number of parameters is 0.5M, StableMax3 is the loss function for the majority of the experiments, equilibrium in

\hat{\bf x}

and

\hat{\bf z}_{\rm H}

, repulsion for

\hat{\bf z}_{\rm H}

and

\hat{\bf x}

, learning rate is

10^{-4}

, epsilon of the optimizer Adam is

10^{-12}

\hat{\bf z}_{\rm H}

and

\hat{\bf z}_{\rm L}

are initialized by

\hat{\bf x}

and

\hat{\bf 0}

, respectively, at the beginning of the 1st segment of training.

Features	Accuracy %	Number of
		Parameters
initialization of $\hat{\bf z}_{\rm H}$ and $\hat{\bf z}_{\rm L}$ as in baseline, epsAdam = $10^{-8}$	77.8	0.52M
epsAdam = $10^{-8}$	81.8	0.52M
epsAdam = $10^{-12}$	85.0	0.52M
epsAdam = $10^{-14}$	86.1	0.52M
epsAdam = $10^{-14}$ , identical transformer layers	85.4	0.26M

Table 10: Dependence of accuracy of CMM on different features for Sudoku-Extreme benchmark. General features:

D=128

N_{\rm H}=3

N_{\rm L}=6

, AMP,

N_{\rm super}=16

N_{\rm accum}=16

N_{\rm grad}=16

B=1000

, MLP-mixer, number of parameters is 0.5M, StableMax3 is the loss function for the majority of the experiments, equilibrium in

\hat{\bf x}

and

\hat{\bf z}_{\rm H}

, repulsion for

\hat{\bf z}_{\rm H}

and

\hat{\bf x}

, learning rate is

2\times 10^{-3}

, freeze embedding after 2500 epoch, AlgGradNorm, epsilon of the optimizer Adam is epsAdam =

10^{-12}

\hat{\bf z}_{\rm H}

and

\hat{\bf z}_{\rm L}

are initialized by

\hat{\bf x}

and

\hat{\bf 0}

, respectively, at the beginning of the 1st segment of training.

Features	Accuracy %	Number of
		Parameters
$\lambda_{\rm LM}=10$ , epsAdam = $10^{-14}$ , identical transformer layers	82.2	0.26M

Table 11: Dependence of accuracy of CMM on different features for Maze benchmark. General features:

D=128

N_{\rm H}=3

N_{\rm L}=4

, AMP,

N_{\rm super}=16

N_{\rm accum}=4

N_{\rm grad}=4

B=100

, MLP-mixer, StableMax3 is the loss function for the majority of the experiments, equilibrium in

\hat{\bf x}

and

\hat{\bf z}_{\rm H}

, repulsion for

\hat{\bf z}_{\rm H}

and

\hat{\bf x}

, learning rate is

2\times 10^{-4}

, freeze embedding after 5000 epoch, AlgGradNorm, epsilon of the optimizer Adam is epsAdam =

10^{-12}

\hat{\bf z}_{\rm H}

and

\hat{\bf z}_{\rm L}

are initialized by

\hat{\bf x}

and

\hat{\bf 0}

, respectively, at the beginning of the 1st segment of training.

Sudoku Solving

See all 10 papers

Jun 26, 2025

Hierarchical Reasoning Model

Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language models (LLMs) primarily employ Chain-of-Thought (CoT) techniques, which suffer from brittle task decomposition, extensive data requirements, and high latency. Inspired by the hierarchical and multi-timescale processing in the human brain, we propose the Hierarchical Reasoning Model (HRM), a novel recurrent architecture that attains significant computational depth while maintaining both training stability and efficiency. HRM executes sequential reasoning tasks in a single forward pass without explicit supervision of the intermediate process, through two interdependent recurrent modules: a high-level module responsible for slow, abstract planning, and a low-level module handling rapid, detailed computations. With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples. The model operates without pre-training or CoT data, yet achieves nearly perfect performance on challenging tasks including complex Sudoku puzzles and optimal path finding in large mazes. Furthermore, HRM outperforms much larger models with significantly longer context windows on the Abstraction and Reasoning Corpus (ARC), a key benchmark for measuring artificial general intelligence capabilities. These results underscore HRM's potential as a transformative advancement toward universal computation and general-purpose reasoning systems.

Guan Wang, Jin Li, Yuhao Sun +6

Sapient Intelligence, Singapore · Tsinghua University

Recurrent Neural Networks Algorithmic Reasoning

Ask PDF

Sep 30, 2026

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic tasks. Such compact solvers are natural candidates for tools that an LLM can call on narrow algorithmic subproblems. However, existing models such as HRM, TRM and URM differ in architecture, gradient propagation and training procedure simultaneously. This makes it hard to tell what drives their performance, and their optimization is still poorly understood and often unstable. In this work we address both of these gaps. First, we study these questions under a unified experimental pipeline spanning six algorithmic domains. Individual controlled ablations are performed on representative domains, while the resulting recipe is evaluated across the full suite. The study reveals a surprisingly simple recipe for stable and generalizable recursive reasoning: an intermediate gradient horizon, large physical batches and controlled updates of the recurrent state. An explicit hierarchical architecture is not needed. Second, we combine these findings into a stable 13.6M-parameter model that achieves the strongest overall performance among the evaluated recursive baselines, with particularly large gains on out-of-distribution generalization. It raises Arithmetic OOD accuracy to 71.2%, from 36.2% for the strongest baseline, while reaching 98.41% on Sudoku and 59.5% pass@2 on ARC-AGI-1. Our results show that, within the recursive architectures studied here, performance depends strongly on how recurrence is optimized and stabilized. More broadly, it shows how AI systems can be improved by optimizing their components one at a time.

Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov +4

RND NLP, DAIMLD, Russian Federation

Recursive Reasoning Models Neural Algorithmic Reasoning Recurrent Transformers Algorithmic Reasoning Recursive Language Models

Ask PDF

Feb 18, 2026

Training Large Reasoning Models Efficiently via Progressive Thought Encoding

Large reasoning models (LRMs) excel on complex problems but face a critical barrier to efficiency: reinforcement learning (RL) training requires long rollouts for outcome-based rewards, where autoregressive decoding dominates time and memory usage. While sliding-window cache strategies can bound memory, they disrupt long-context reasoning and degrade performance. We introduce Progressive Thought Encoding, a parameter-efficient fine-tuning method that enables LRMs to reason effectively under fixed-size caches. By progressively encoding intermediate reasoning into fixed-size vector representations, our approach eliminates the need to backpropagate through full-cache rollouts, thereby reducing memory usage, while maintaining constant memory during inference. Experiments on three models, including Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and DeepSeek-R1-Distill-Llama-8B, on six widely used challenging mathematical benchmarks show consistent gains: our method achieves +19.3% improvement over LoRA-based fine-tuning and +29.9% over LRMs without fine-tuning on average, with up to +23.4 accuracy improvement on AIME2024/2025 under the same tight cache budgets. These results demonstrate that Progressive Thought Encoding not only improves reasoning accuracy but also makes RL training of LRMs substantially more efficient and scalable under real-world memory constraints.

Zeliang Zhang, Xiaodong Liu, Hao Cheng +3

University of Rochester · Microsoft Research

Efficient Language Model Reasoning LLM Reasoning Large Reasoning Models Parameter-Efficient Fine-Tuning RL for Language Models

Ask PDF

Abstract

Sudoku Solving

Explore similar work

Hierarchical Reasoning Model

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

Training Large Reasoning Models Efficiently via Progressive Thought Encoding