Recursive Language Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 35

All topics

Enabling Dynamic Computation in Looped LMs
Oct 6, 2026Aayush Mishra, Arnau Padrés Masdemont, Victor Conchello Vendrell +3LLM Inference Acceleration Adaptive Computation
Qualcomm AI Research · Johns Hopkins University
Hybrid Latent Attention for Looped Language Models
Oct 6, 2026Yuhan Chen, Siyuan Zhang, Nan Wang +2Long-Context Language Model Inference LLM Inference Acceleration
Virginia Tech · Independent Researcher
SanSi: A Looped Typed Decision Model for System 1.5 Thinking
Oct 6, 2026Shuyu Gan, Young-Jun Lee, Dongyeop KangEfficient Language Model Reasoning Implicit Reasoning in Language Models
University of Minnesota
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Oct 5, 2026Benhao Huang, Chufan Shi, Junlin Chen +4Efficient Language Model Training Recurrent Transformers
Institute of Foundation Models · CMU · USC
Wikidata Search Traces: A Dataset for Training Knowledge Graph Search Agents
Oct 5, 2026Mohamed Chenene, Carlos Rosas-Hinostroza, Anastasia Stasenko +2Multi-Hop QA Knowledge Graph Question Answering
PleIAs · Lattice, ENS-PSL, CNRS, Sorbonne Nouvelle · Université Paris Dauphine-PSL +4
Loopy: Low-Bit Quantization Framework for Looped Language Models
Oct 4, 2026Zeyu LI, Yipu ZHANG, Jintao Chen +2LLM Quantization Post-Training Quantization
The Hong Kong University of Science and Technology · Duke Kunshan University
Closing the Loop: Practical Training Recipes for Looped Language Models
Sep 30, 2026Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin +5Efficient Language Model Training LLM Training
RND NLP, DAIMLD, Russian Federation
Random Recursive Models
Sep 30, 2026Jama Hussein Mohamud, Mirco RavanelliEfficient Neural Network Inference LLM Reasoning
Mila – Quebec AI Institute · Université de Montréal · Concordia University
What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling
Sep 30, 2026Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov +4Neural Algorithmic Reasoning Recurrent Transformers
RND NLP, DAIMLD, Russian Federation
Principled Thoughts for Latent Recursive LLM Systems
Sep 28, 2026Fahd Seddik, Fatemeh FardContinuous Latent Reasoning Multi-Agent LLMs
FARD Lab, University of British Columbia, Okanagan, Canada
Loop Dropout: Regularizing Shared Updates in Looped Language Models
Sep 28, 2026Zirui Zhu, Hailun Xu, Xuanlei Zhao +5Low-Rank Adaptation Parameter-Efficient Fine-Tuning
Reasoning on the Simplex: Geometric Fixed-Point Models
Sep 27, 2026Talgat Daulbaev, Ilya Glazkov, Maxim Rakhuba +1Mathematical Reasoning Recursive Language Models
Applied AI Institute · LigandPro · HSE University +1
Stream Recursion Model (SRM)
Sep 23, 2026Asael Sorensen, Charles Brock, David Chamberlain +3LLM Interpretability Mechanistic Interpretability
Sandia National Laboratories, Albuquerque, NM, USA · Institute for Complex Additive Systems Analysis, New Mexico Institute of Mining and Technology, Socorro, NM, USA · Department of Computer Science and Engineering, New Mexico Institute of Mining and Technology, Socorro, NM, USA.
Log-Depth Recurrent Language Modeling
Sep 23, 2026Yiqin Wang, Nuri Cingillioglu, Charles PertLong-Context Language Modeling Autoregressive Language Modeling
Department of Computing Imperial College London London, United Kingdom
LoopCD: Loop-wise Contrastive Decoding for Improving Reasoning in Looped Language Models
Sep 21, 2026Byeongho Yu, Junhyuk So, Eunhyeok ParkContrastive Decoding Latent Reasoning
Department of Computer Science and Engineering Pohang University of Science and Technology (POSTECH) · Graduate School of Artificial Intelligence Pohang University of Science and Technology (POSTECH)
Beyond Depth Truncation: Controlled Evaluation of Depth Utilization in Recursive Language Models
Sep 17, 2026Ha Van Dau, Thanh Tung Khuat, Nguyen Thanh DungLLM Evaluation Recurrent Transformers
NuverxAI - AI & Creative Innovation Company Limited · Ho Chi Minh City University of Technology (HCMUT)
A Dominant Supplier Slows Recursive Drift More Than It Steers It
Sep 10, 2026Yangze Liu, Zhongyi HanDrifting Models Recursive Language Models
Shandong University
Looped GPT-BERT: Trading Parameters for Computation in Small Language Modeling
Sep 9, 2026Tingshuo Fan, Hongtao Mu, Tianyu Zhou +2Language Model Pretraining Small Language Models
College of Foreign Languages and Literature, Fudan University
RLMOpt: Adaptive Prompt Optimization via Recursive Language Models
Aug 11, 2026Subhash Bangalore Satheesha, Nirvik Pande, Deepthi Duddempudi +1Large Language Model-Guided Optimization LLM Agents
Autonomize AI · Carnegie Mellon University
Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
Aug 10, 2026Kristian Schwethelm, Daniel Rueckert, Georgios KaissisLLM Inference LLM Inference Scheduling
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM), Germany · Department of Computing, Imperial College London, UK · Munich Center for Machine Learning (MCML), Germany +1
Think Deep, Speak Once: Relit, A Recursive Latent Implicit Transformer Framework
Aug 8, 2026Abhishek Panwar, Maheep Singh, Saksham BansalContinuous Latent Reasoning Efficient Language Model Inference
Indian Institute of Technology Roorkee, India
Chained Recursive Language Models for Multi-Iteration Reasoning
Aug 5, 2026Purbesh Mitra, Sennur UlukusLong-Context Language Model Inference Language Modeling
University of Maryland
TimeRLM: Recursive Language Models Enable Precise Anomaly Localization in Long-Context Time-Series
Aug 4, 2026Nicolas Zumarraga, Lorenzo Steno, Ning Wang +9Anomaly Localization Long-Context Modeling
1Agentic Systems Lab, ETH Zürich · 3Seldon Technologies · 2Stanford University +4
Energy-guided Recursive Model
Jul 11, 2026Yifei Zhao, Ying TangInference-Time Search Energy-Based Models
Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Physics, University of Electronic Science and Technology of China, Chengdu 611731, China
Entropy-Gated Latent Recursion
Jun 15, 2026Soham Bhattacharjee, Dushyant Singh Chauhan, Salem Lahlou +2LLM Inference Inference-Time Scaling
Department of Machine Learning Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, United Arab Emirates
LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling
Jun 3, 2026Wenkai Chen, Tianshu Li, Wenyong Huang +3Mixture-of-Experts Language Models Language Modeling
Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Co.,Ltd.
Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models
May 26, 2026Xiao-Wen Yang, Ziyu Han, Xi-Hua Zhang +4Latent Dynamics Modeling Test-Time Scaling
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, Nanjing, China
Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies
May 20, 2026Ming LiuSelf-Training for Language Models Model Collapse
Amazon
Consolidation-Expansion Operator Mechanics:A Unified Framework for Adaptive Learning
May 11, 2026Debashis GuhaAdaptive Learning Adaptive Computation
S P Jain School of Global Management
Sparse Layers are Critical to Scaling Looped Language Models
May 9, 2026Ryan Lee, Jacob Biloki, Edward J. Hu +1Mixture-of-Experts Language Models Recurrent Transformers
1USC Information Sciences Institute · 2Netflix · 3Independent Researcher

Oct 6, 2026, cs.AIJ/K move · Enter open · S save

Enabling Dynamic Computation in Looped LMs

Aayush Mishra, Arnau Padrés Masdemont, Victor Conchello Vendrell, Jordi Ros Giralt+2

Qualcomm AI Research · Johns Hopkins University

LLM Inference Acceleration Adaptive Computation KV-Cache Management Recursive Language Models

Open paper AskPDF arXiv

Looped LMs are parameter efficient and promise dynamic computation (saving memory and FLOPs on easy tokens). However, state-of-the-art open Looped LMs trained with this dynamic computation capability (Ouro models) do not realize it in practice as each loop iteration (depth) requires its own level of KV-cache, necessitating all loop computations. Moreover, Ouro's early-exit prior is enforced on each token equally, which results in static lower-depth like processing of all tokens regardless of difficulty. In this work, we propose a simple "best-available" KV caching strategy that works out-of-the-box, creating a new frontier in the performance vs depth space. Our approach enables up to 30% reduction in FLOPs and KV memory while retaining full-depth performance, showing the true flexibility of Looped LMs. Furthermore, training looped LMs with awareness about this KV caching strategy improves performance and efficiency. Finally, we apply a small but effective fix to the early-exit prior enforcement objective that makes tokens exit at truly heterogeneous depths based on effort. Our findings are validated on Ouro models as well as smaller looped LMs pre-trained from scratch.

Momentum

Related topics

Latest papers 35