DART: Distributional Adversarial Recurrent Training for Algorithm Learning

Hieu Tran Bao; Phung Thanh Dang; Pham Quang Nhat Minh; Hoang Thanh Tung

cs.LGSep 5, 2026

DART: Distributional Adversarial Recurrent Training for Algorithm Learning

Authors: Hieu Tran Bao, Phung Thanh Dang, Pham Quang Nhat Minh, Hoang Thanh Tung

Organizations: FPT IS AI R&D Center Hanoi, Vietnam · VNU – University of Engineering and Technology Hanoi, Vietnam

Abstract

Recurrent reasoning models (RRMs) can solve structured problems, achieving easy-to-hard generalization through iterative computation in hidden space. These models are typically trained with instance-level supervision, which becomes increasingly problematic as task difficulty grows: valid solutions occupy a tiny region of the solution space, while invalid solutions proliferate rapidly. We propose Distributional Adversarial Recurrent Training (DART), a training framework that replaces single-point supervision with a local target distribution around the ground-truth solution and aligns model outputs with this distribution through an adversarial objective. DART provides a richer learning signal and encourages more stable iterative trajectories toward valid solutions. When evaluated on Maze, Chess, and masked Sudoku with multiple RRMs, including Deep Thinking Systems and Tiny Recursive Models, DART improves solution quality, stability, and robustness under the evaluated distribution shifts. Comparisons with label smoothing, Gaussian softened targets, and progressive training show that DART is not explained by target softening alone and is complementary to training schemes that stabilize long-horizon recurrence. These results identify DART as a promising approach for improving robustness across the evaluated recurrent reasoning models.

Figures & tables

\mathcal{L}_{critic}=-\Big(\mathbb{E}_{y\sim P_{\text{real}}}[C_{\psi}(x,y)]-\mathbb{E}_{\hat{y}\sim P_{\text{gen}}}[C_{\psi}(x,\hat{y})]\Big)

Algorithm 1 Training Recursive Reasoning Models with Adversarial Alignment

Figure 1: Maze accuracy as the evaluation size increases from

13\times 13

31\times 31

. The CE failure mode is mostly absent at the smaller size but appears under the harder spatial extrapolation setting.

Method	$13\times 13$ Peak	$13\times 13$ Final (100)	$31\times 31$ Peak	$31\times 31$ Final (300)
CE	$99.88\pm 0.00$	$99.85\pm 0.03$	$73.92\pm 26.01$	$73.88\pm 25.98$
Progressive CE	$99.17\pm 0.59$	$99.06\pm 0.63$	$75.01\pm 16.69$	$74.96\pm 16.75$
DART	$\mathbf{100.00\pm 0.00}$	$\mathbf{100.00\pm 0.00}$	$\mathbf{99.91\pm 0.13}$	$\mathbf{99.91\pm 0.13}$
Progressive DART	$99.98\pm 0.01$	$99.98\pm 0.01$	$98.75\pm 1.25$	$98.73\pm 1.26$

Table 1: Maze scaling results for standard and progressive training (mean

\pm

std over seeds).

Figure 2: Sudoku accuracy under increasing masking ratios. DART consistently outperforms CE, with larger gains under stronger distribution shift.

			Standard CE			DART
H	L	Depth	30%	40%	50%	30%	40%	50%
3	6	42	$95.5\pm 0.0$	$78.1\pm 0.8$	$32.3\pm 1.7$	$\mathbf{96.0\pm 0.0}$	$\mathbf{81.4\pm 0.6}$	$\mathbf{37.5\pm 0.1}$
4	6	56	$95.5\pm 0.0$	$78.2\pm 0.8$	$32.4\pm 1.7$	$\mathbf{96.0\pm 0.0}$	$\mathbf{81.5\pm 0.6}$	$\mathbf{38.0\pm 0.1}$
5	6	70	$95.5\pm 0.0$	$78.3\pm 0.8$	$32.5\pm 1.7$	$\mathbf{96.0\pm 0.0}$	$\mathbf{81.6\pm 0.6}$	$\mathbf{38.4\pm 0.1}$
6	6	84	$95.5\pm 0.0$	$78.3\pm 0.8$	$32.6\pm 1.7$	$\mathbf{96.0\pm 0.0}$	$\mathbf{81.6\pm 0.6}$	$\mathbf{38.5\pm 0.1}$

Table 2: Test accuracy (%) on Sudoku-Extreme under different masking ratios.

Figure 3: Sample reasoning trajectories for CE and DART on Maze and Chess. The top row shows IoU with the target solution, while the bottom row shows exact match; dotted vertical lines mark the first exact solution. In these examples, DART reaches the exact solution and remains there, whereas CE can enter the correct region transiently and then drift away.

Task	Method	Solved once (%)	Final exact (%)	First exact iter	Retention after first exact (%)	Exact-match AUC (%)
Maze $31\times 31$	CE	51.6	43.8	88.6	82.0	28.4
Maze $31\times 31$	DART	100.0	100.0	94.0	99.9	68.9
Chess	CE	84.8	0.0	28.0	11.6	4.5
Chess	DART	91.0	15.8	23.1	63.5	32.3

Table 3: Aggregate trajectory statistics over held-out samples (Maze: 128 samples; Chess: 512 samples). First exact iteration and retention are computed over samples that reach an exact solution at least once.

Figure 4: Progressive training baselines on Chess. Progressive training removes the late-iteration collapse of CE and DART, while DART alone reaches the highest peak accuracy.

Method	Peak	Peak iter	Acc. @ iter 40	Final (50)
CE	$80.81\pm 0.37$	30, 30	$0.56\pm 0.11$	$0.92\pm 0.08$
Progressive CE	$83.50\pm 0.03$	41, 45	$83.48\pm 0.02$	$83.46\pm 0.02$
DART	$\mathbf{84.83\pm 0.03}$	31, 31	$28.58\pm 10.47$	$11.13\pm 4.64$
Progressive DART	$83.59\pm 0.09$	41, 50	$\mathbf{83.58\pm 0.09}$	$\mathbf{83.52\pm 0.16}$

Table 4: Progressive training baselines on Chess (mean

\pm

std over seeds).

Panel of Figure 5 — Figure 5: Accuracy versus reasoning iterations for target-smoothing baselines. Gaussian softened targets use the same local target construction as DART but remove the adversarial critic.

Method	Peak	Acc. @ iter 250	Final (300)
CE	$73.92\pm 26.01$	$73.71\pm 26.17$	$73.88\pm 25.98$
Label smoothing ( $\epsilon=0.3$ )	$98.22\pm 0.64$	$98.06\pm 0.57$	$98.21\pm 0.65$
Gaussian softened target, no critic	$59.58\pm 38.86$	$59.14\pm 38.80$	$59.58\pm 38.86$
DART	$\mathbf{99.91\pm 0.13}$	$\mathbf{99.89\pm 0.15}$	$\mathbf{99.91\pm 0.13}$

Table 5: Target-smoothing baselines on Maze

31\times 31

(mean

\pm

std over seeds).

Method	Peak	Peak iter	Acc. @ iter 40	Final (50)
CE	$80.81\pm 0.37$	30, 30	$0.56\pm 0.11$	$0.92\pm 0.08$
Label smoothing ( $\epsilon=0.3$ )	$79.29\pm 0.04$	30, 30	$0.07\pm 0.00$	$0.14\pm 0.00$
Gaussian softened target, no critic	$82.22\pm 0.09$	33, 34	$\mathbf{80.93\pm 0.28}$	$\mathbf{79.29\pm 0.49}$
DART	$\mathbf{84.83\pm 0.03}$	31, 31	$28.58\pm 10.47$	$11.13\pm 4.64$

Table 6: Target-smoothing baselines on Chess (mean

\pm

std over seeds).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

task	optim.	lr	lr critic	decay schedule	decay factor	warm-up	epochs	clip
Mazes	adamw	1e-3	5e-5	–	–	10	50	0.01
Chess	sgd	1e-2	5e-5	[100, 110]	0.01	3	120	0.01
Sudoku	adam	1e-3	5e-5	[1000]	0.01	10	3000	0.01

Appendix

Table 7: Training hyperparameters. Dashes indicate that we did not utilize those options.

Method	Seconds / epoch	Relative to CE	GPU-hours
CE	59.0	$1.00\times$	1.97
Label smoothing	59.0	$1.00\times$	1.97
Gaussian softened target	59.0	$1.00\times$	1.97
DART	104.4	$1.77\times$	3.48
Progressive CE	83.8	$1.42\times$	2.79
Progressive DART	127.4	$2.16\times$	4.25

Appendix

Table 8: Representative Chess training cost on one RTX 5090. GPU-hours assume 120 epochs.

Panel of Figure 6 — Figure 6: Effect of the target distribution width $\sigma$ and the critic architecture on reasoning performance.

Neural Algorithmic Reasoning

Neural algorithmic reasoning studies how neural networks can learn, execute, and generalize algorithmic procedures.

See all 45 papers

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling
Sep 30, 2026Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov +4Recursive Reasoning Models Neural Algorithmic Reasoning
RND NLP, DAIMLD, Russian Federation
Form Follows Function: Recursive Stem Model
Mar 3, 2026Navid HakimiRecursive Reasoning Models Neural Algorithmic Reasoning
DART: Draft-Agreement Routing for Training-Free Adaptive Thinking Budgets in Hybrid Reasoning Models
Jun 22, 2026Jungseob Lee, Seongtae Hong, Seungjun Lee +7LLM Routing Efficient Language Model Inference
Korea University · 242dot · Konkuk University +3

Sep 30, 2026J/K move · Enter open · S save

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov, Inessa Fedorova+3

RND NLP, DAIMLD, Russian Federation

Recursive Reasoning Models Neural Algorithmic Reasoning Recurrent Transformers Algorithmic Reasoning Recursive Language Models

Open paper AskPDF arXiv

Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic tasks. Such compact solvers are natural candidates for tools that an LLM can call on narrow algorithmic subproblems. However, existing models such as HRM, TRM and URM differ in architecture, gradient propagation and training procedure simultaneously. This makes it hard to tell what drives their performance, and their optimization is still poorly understood and often unstable. In this work we address both of these gaps. First, we study these questions under a unified experimental pipeline spanning six algorithmic domains. Individual controlled ablations are performed on representative domains, while the resulting recipe is evaluated across the full suite. The study reveals a surprisingly simple recipe for stable and generalizable recursive reasoning: an intermediate gradient horizon, large physical batches and controlled updates of the recurrent state. An explicit hierarchical architecture is not needed. Second, we combine these findings into a stable 13.6M-parameter model that achieves the strongest overall performance among the evaluated recursive baselines, with particularly large gains on out-of-distribution generalization. It raises Arithmetic OOD accuracy to 71.2%, from 36.2% for the strongest baseline, while reaching 98.41% on Sudoku and 59.5% pass@2 on ARC-AGI-1. Our results show that, within the recursive architectures studied here, performance depends strongly on how recurrence is optimized and stabilized. More broadly, it shows how AI systems can be improved by optimizing their components one at a time.

Abstract

Neural Algorithmic Reasoning

Explore similar work

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

Form Follows Function: Recursive Stem Model

DART: Draft-Agreement Routing for Training-Free Adaptive Thinking Budgets in Hybrid Reasoning Models