Think, But Don't Overthink: Reproducing Recursive Language Models

Daren Wang

cs.CLMar 3, 2026

Think, But Don't Overthink: Reproducing Recursive Language Models

Authors: Daren Wang

Organizations: The Chinese University of Hong Kong

Abstract

This project reproduces and extends the recently proposed Recursive Language Models'' (RLMs) framework by Zhang et al. (2026). This framework enables Large Language Models (LLMs) to process near-infinite contexts by offloading the prompt into an external REPL environment. While the original paper relies on a default recursion depth of 1 and suggests deeper recursion as a future direction, this study specifically investigates the impact of scaling the recursion depth. Using state-of-the-art open-source agentic models (DeepSeek v3.2 and Kimi K2), I evaluated pure LLM, RLM (depth=1), and RLM (depth=2) on the S-NIAH and OOLONG benchmarks. The findings reveal a compelling phenomenon: Deeper recursion causes models to overthink''. While depth-1 RLMs effectively boost accuracy on complex reasoning tasks, applying deeper recursion (depth=2) or using RLMs on simple retrieval tasks paradoxically degrades performance and exponentially inflates execution time (e.g., from 3.6s to 344.5s) and token costs. Code and data are available at: https://github.com/drbillwang/rlm-reproduction

Figures & tables

Figure 1: Performance comparison of Base LLM, RLM (Depth=1), and RLM (Depth=2) against the original paper’s benchmarks.

Figure 2: Average Execution Time (seconds) across different models and recursion depths.

Figure 3: Average Token Usage (thousands) across different models and recursion depths.

Figure 4: Average Token Cost (US$ cents) across different models and recursion depths.

Failure Mode 1: Parametric Hallucination (Context Anchoring Loss)
Model: DeepSeek v3.2 Task: S-NIAH (Depth=2) Time: 260.5s
Prompt: What are all the special magic numbers for unknown mentioned in the provided text?
Raw Output: “The special magic numbers for unknown (atomic nuclei) mentioned in the text are: 2, 8, 20, 28, 50, 82, 126. These numbers represent the nuclear magic numbers where atomic nuclei are particularly stable due to filled nuclear shells.”
Insight: The deep recursion caused the model to abandon the input context and hallucinate real-world physics constants based on its pre-trained weights.
Failure Mode 2: Formatting Collapse in REPL
Model: Kimi K2 Task: OOLONG (Depth=1) Time: 1223.6s

Figure 5: Qualitative examples of RLM trajectory failures. Deeper recursion (Depth=2) often induces parametric hallucinations, role-playing confusion within the REPL, and performative over-explanation.

Recursive Language Models

Recursive Language Models use repeated or decomposed computation to extend language-model reasoning beyond a single pass.

See all 59 papers

The $\mathbf{Y}$ -Combinator for LLMs: Solving Long-Context Rot with $λ$ -Calculus
Mar 20, 2026Amartya Roy, Rasul Tutunov, Xiaotong Ji +2Long-Context Reasoning in Language Models Long-Context Language Model Inference
The School of Interdisciplinary Research, Indian Institute of Technology (IIT) Delhi, Robert Bosch GmbH, India · Huawei Noah’s Ark Lab · Huawei Noah’s Ark +1
Chained Recursive Language Models for Multi-Iteration Reasoning
Aug 5, 2026Purbesh Mitra, Sennur UlukusLong-Context Language Model Inference Language Modeling
University of Maryland
Recursive Models for Long-Horizon Reasoning
Mar 2, 2026Chenxiao Yang, Nathan Srebro, Zhiyuan LiRecursive Language Models Language Modeling
Toyota Technological Institute at Chicago

Mar 20, 2026J/K move · Enter open · S save

The $\mathbf{Y}$ -Combinator for LLMs: Solving Long-Context Rot with $λ$ -Calculus

Amartya Roy, Rasul Tutunov, Xiaotong Ji, Matthieu Zimmer+1

The School of Interdisciplinary Research, Indian Institute of Technology (IIT) Delhi, Robert Bosch GmbH, India · Huawei Noah’s Ark Lab · Huawei Noah’s Ark+1

Long-Context Reasoning in Language Models Long-Context Language Model Inference Efficient Language Model Reasoning Recursive Language Models

Open paper AskPDF arXiv

LLMs are increasingly used as general-purpose reasoners, but long inputs remain bottlenecked by a fixed context window. Recursive Language Models (RLMs) address this by externalising the prompt and recursively solving subproblems. Yet existing RLMs depend on an open-ended read-eval-print loop (REPL) in which the model generates arbitrary control code, making execution difficult to verify, predict, and analyse. We introduce

λ

-RLM, a framework for long-context reasoning that replaces free-form recursive code generation with a typed functional runtime grounded in

λ

-calculus. It executes a compact library of pre-verified combinators and uses neural inference only on bounded leaf subproblems, turning recursive reasoning into a structured functional program with explicit control flow. We show that

λ

-RLM admits formal guarantees absent from standard RLMs, including termination, closed-form cost bounds, controlled accuracy scaling with recursion depth, and an optimal partition rule under a simple cost model. Empirically, across four long-context reasoning tasks and nine base models,

λ

-RLM outperforms standard RLM in 29 of 36 model-task comparisons, improves average accuracy by up to +21.9 points across model tiers, and reduces latency by up to 4.1x. These results show that typed symbolic control yields a more reliable and efficient foundation for long-context reasoning than open-ended recursive code generation. The complete implementation of

λ

-RLM, is open-sourced for the community at: https://github.com/lambda-calculus-LLM/lambda-RLM.

Abstract

Recursive Language Models

Explore similar work

The Y\mathbf{Y}Y-Combinator for LLMs: Solving Long-Context Rot with λλλ-Calculus

Chained Recursive Language Models for Multi-Iteration Reasoning

Recursive Models for Long-Horizon Reasoning

The $\mathbf{Y}$ -Combinator for LLMs: Solving Long-Context Rot with $λ$ -Calculus