cs.LGApr 20, 2026

One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models

Authors: Chris CameronWangzheng WangNikita IvanovAshmita BhattacharyyaDidier ChételatYingxue Zhang

Organizations: Huawei Technologies · University of Waterloo · University of Toronto

Abstract

Looped transformers scale computational depth without increasing parameter count by repeatedly applying a shared transformer block and can be used for iterative refinement, where each loop rewrites a full fixed-size prediction in parallel. On difficult problems, such as those that require search-like computation, reaching a highly structured solution starting from noise can require long refinement trajectories. Learning such trajectories is challenging when training specifies only the target solution and provides no supervision over the intermediate refinement path. Diffusion models tackle this issue by corrupting data with varying magnitudes of noise and training the model to reverse it in a \textit{single step}. However, this process misaligns training and testing behaviour. We introduce Denoising Recursion Models, a method that similarly corrupts data with noise but trains the model to reverse the corruption over \textit{multiple} recursive steps. This strategy provides a tractable curriculum of intermediate states, while better aligning training with testing and incentivizing non-greedy, forward-looking generation. Through extensive experiments, we show this approach outperforms the Tiny Recursion Model (TRM) on ARC-AGI, where it recently achieved breakthrough performance.

Explore similar work

CardsList
  1. Thinking with Looped Flows

    Sep 12, 2026Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom +3Generative Flow NetworksDeep Learning

  2. Recursive Scaling in Masked Diffusion Models

    Jun 16, 2026Alba Carballo-Castro, Julianna Piskorz, Paulius Rauba +2Masked Diffusion Language ModelsDiscrete Diffusion Model

  3. Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster

    May 18, 2026Grigory Bartosh, Teodora Pandeva, Sushrut Karmalkar +1Generative ModelsFactor