Recurrent Model

Follow this topic Ask about this topic

Recent momentum

-33%

6 papers in the last 28 days · 0.2% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Recurrent Model.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Recurrent Model.

97 papers

Latest in Recurrent Model

Open your feed →

Persistent Recurrent Memory Between Transformer Layers - Improves Language Model Generalization
Sep 15, 2026Eduardo Novaes HeringRecurrent Model
Temporal Recurrence Favors Fewer Layers
Sep 14, 2026Ivan Anokhin, Johan Obando-Ceron, Irina Rish +1Recurrent Model Multilayer Perceptrons
Thinking with Looped Flows
Sep 12, 2026Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom +3Generative Flow Networks Recurrent Model
The information geometry of large language models is shared, learned, and controllable
Sep 11, 2026Dario PicozziRecurrent Model
Learning Length-Extrapolatable Recurrent Models
Sep 8, 2026Hanwen JiangRecurrent Model Efficient Long-Context Inference
Kalman Delta Networks: Uncertainty-aware Associative Memory
Sep 7, 2026Ngoc Bui, Tinglin Huang, Rex YingRecurrent Model Associative Memory
Pre-Whitening and BCJR Posterior Distillation for Bi-LSTM Detection in Faster-than-Nyquist Signaling
Sep 7, 2026Nurettin Safak, Osman Tokluoglu, Enver CavusBidirectional Long Short-Term Memory Recurrent Model
Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks
Sep 3, 2026Shivang Rawat, Mirko Morello, Flaviano Morone +1Recurrent Neural Networks Recurrent Model
MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
Aug 12, 2026Ming Zhang, Kaisen Yang, Shu Yu +6Recurrent Model Efficient Long-Context Inference
Phase State Space Models: Parallel, Surrogate-Free Training of Spiking Networks
Aug 7, 2026Wilkie Olin-AmmentorpSpiking Neural Networks Recurrent Neural Networks
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
Aug 4, 2026Yu Luo, Bo Dong, Wenhua Cheng +1Residual Quantization Quantization
Trajectory-Guided Forget-Recover Network for Continual LLM Unlearning
Aug 4, 2026Zezheng Wu, Xinghe Cheng, Qinggang Zhang +4Large Language Model Unlearning Unlearning Aims
Maglev: Sliding Recurrent Memory
Aug 3, 2026Bo Liu, Qiang LiuTransformer Attention Recurrent Model
Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets
Jul 31, 2026Mark ShapiroLarge Language Model Post-Training Recurrent Model
Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating
Jul 27, 2026Maruthi Vemula, Neeraj Praneeth GajulaWorking Memory Eviction
Frequency-Based Reservoir computing
Jul 27, 2026Arthur S PowanweReservoir Computing Reservoir
Strong Gravitational Lensing Posterior Sampling in Pixel-Space Using Diffusion Models and Recurrent Inference Machines
Jul 21, 2026Guillaume Payeur, Laurence Perreault-Levasseur, Gabriel Missael Barco +1Cosmology Posterior Sampling
T^2MLR: Transformer with Temporal Middle-Layer Recurrence
Jul 16, 2026Ziyang Cai, Xingyu Zhu, Yihe Dong +2Multi-Layer Transformers Latent Reasoning
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
Jul 16, 2026Changhai Zhou, Kieran Liu, Yuhua Zhou +17Efficient Long-Context Inference LLM Inference Efficiency
Per-Token Fixed-Point Convergence in Depth-Recurrent Transformers
Jul 15, 2026Joe LoganRecurrent Model Recurrent State
DeepLoop: Depth Scaling for Looped Transformers
Jul 15, 2026Shuzhen Li, Yifan Zhang, Jiacheng Guo +2Search Depth Recurrent Model
Learning-based Probabilistic Load Forecasting with Post-hoc and In-model Uncertainty
Jul 14, 2026Sarah Al-Shareeda, Gulcihan Ozdemir, Heung Seok JeonEnergy Forecasting Predictive Uncertainty
A Hybrid Mamba for Audio-Visual Navigation
Jul 14, 2026Yi Wang, Yinfeng YuObject Goal Navigation Visual Representation Learning
Extending LLM Context via Associative Recurrent Memory
Jul 13, 2026Gleb Kuzmin, Ivan Rodkin, Aydar Bulatov +8Efficient Long-Context Inference Recurrent Model
LayerNorm as Implicit Gain Control in Looped Transformers
Jul 12, 2026Matthias M. M. BuehlmaierBatch Normalization Recurrent Model
Robustly Invertible Nonlinear Dynamics and the BiLipREN: From Inversion-Based Control to Generative Trajectory Modelling
Jul 10, 2026Yurui Zhang, Ruigang Wang, Ian R. ManchesterNonlinear Dynamics Recurrent Neural Networks
Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing
Jul 8, 2026Tommaso Cerruti, Tim Rieder, George Rowlands +2Softmax Attention Model Attention
When Does Tool Use Increase the Expressive Power of Finite-Precision Recurrent Models?
Jul 7, 2026Nikola Zubić, Qian Li, Yuyi Wang +1Recurrent Model Finite
Why does Deep Learning Improve Visual SLAM?
Jul 7, 2026Giovanni Cioffi, Davide ScaramuzzaOrb-Slam3 Recurrent Model
Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training
Jul 6, 2026Jingwei Zuo, Cong Zeng, Ilyas Chahed +6Large Language Model Training Large Language Model Memory
Rethinking Neural Nonlinearity as Gating
Jul 3, 2026Muhammad Sabih, Frank Hannig, Jürgen TeichActivation Functions Discontinuous Gating
SHiPPO: Recurrent Memory with Transported Polynomial Projections
Jul 3, 2026Tomoya Mizuguchi, Bum Jun KimRecurrent Model Recurrent State
The Orthogonalized Read Is a Removable Training Scaffold for Recurrent Memory
Jul 2, 2026Keston Aquino-MichaelsRecurrent Model
G-RRM: Guiding Symbolic Solvers with Recurrent Reasoning Models
Jul 2, 2026Timo Bertram, Sidhant Bhavnani, Richard Freinschlag +3Recurrent Model Neural Guidance
Stable Self-Modulating Quantum Fast-Weight Programmers with Bounded Memory Gates
Jul 2, 2026Kuo-Chung Peng, Jiun-Cheng Jiang, Chun-Hua Lin +8Quantum Learning Modulation
Finite-Lag Operator Geometry of Recurrent Representations
Jul 2, 2026Kanishka ReddyRecurrent Model Latent Cluster Geometry
Multi-Head Recurrent Memory Agents
Jul 1, 2026Jiatong Li, Samuel Yeh, Sharon LiEfficient Long-Context Inference Recurrent Model
TiRex-2: Generalizing TiRex to Multivariate Data and Streaming
Jul 1, 2026Patrick Podest, Marco Pichler, Elias Bürger +7Time Series Foundation Models Time Series Forecasting
CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention
Jun 25, 2026Sayak DuttaRecurrent Model Causal Attention
Erase-then-Delta Attention: Decoupling Erase and Write Addresses in Delta-Rule Linear Attention
Jun 25, 2026Xiao Li, Chengruidong Zhang, Hao Luo +15Recurrent Model Concept Erasure
Internal Data Repetition Destroys Language Models
Jun 23, 2026Jessica Chudnovsky, Joshua Kazdan, Noam Levi +6Repetition Hours-Long Classifier Training Phase
Recursive QLSTM with Dynamic Variational Quantum Circuit Adaptation
Jun 22, 2026Samuel Yen-Chi Chen, Yifeng Peng, Jiun-Cheng Jiang +8Quantum Learning Variational Quantum Circuits
Compressing Observation History into Agent Memory: Distilling Transformers into Recurrent Transformers
Jun 19, 2026Philippe Weinzaepfel, Christian Wolf, Bülent Mert Sariyildiz +2Recurrent Model Diffusion Transformers
Gradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation
Jun 19, 2026Jianwei LouContinual Learning Low-Data Regimes
Comparing Transformers and Hybrid Models at the Token Level
Jun 18, 2026Yanhong Li, William MerrillToken-Level Uncertainty Protein Language Model
Linear Recurrent Unit with Semantic Modulation for Image Super-Resolution
Jun 18, 2026Mingyu Choi, Woo Kyoung Han, Sunghoon Im +1Real-World Image Super-Resolution Restoration
Why SWAVE May Not Be All You Need:A Concept-Evolution Retrospective on Complex-Valued Recurrent Language Models
Jun 16, 2026Ramprasath Ganesaraja, Swathika N, Sahil Dilip PanseRecurrent Model Protein Language Model
Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models
Jun 12, 2026Rituraj Sharma, Tu VuRecurrent Model Batch Normalization
The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning
Jun 11, 2026Ayushman Trivedi, Bhavika MelwaniCatastrophic Forgetting Continual Learning
$μ$ VLA: On Recurrent Memory for Partially Observable Manipulation in VLA ModelsμVLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models
Jun 10, 2026Egor Cherepanov, Nikita Kachaev, Daniil Zelezetsky +6Diffusion-Based Vision-Language-Actions Vision-Language-Action
Reversible Foundations: Training a 120B Sparse MoE through State-Preserving Scaling
Jun 5, 2026Rohan ShravanRecurrent Model
Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers
Jun 3, 2026Yacouba Kaloga, Shashi Kumar, Shakeel A. Sheikh +3Automatic Speech Recognition Recurrent Model
CART: Context-Anchored Recurrent Transformer -- A Parameter-Efficient Architecture with Learned Stability
May 31, 2026Chad A. CappsRecurrent Model Multi-Head Attention
Task Structure Reverses Layerwise State Encoding in Sequence Models
May 30, 2026Yuhang JiangSequence Modeling Multi-Layer Transformers
Why Linear Recurrent Memory Works in Partially Observable Reinforcement Learning
May 29, 2026Yike Zhao, Onno Eberhard, Malek Khammassi +2Recurrent Model Partially Observable Markov Decision Process
Memory by Design: Probabilistic Sequence Layers
May 29, 2026Matthew Dowling, Hyungju Jeon, Cristina Savin +1Recurrent Model Biological Memory
Chess-World-Model: A 10M-Game Benchmark for Exact State Tracking from Chess Move Sequences
May 28, 2026Benjamin Walker, Terry LyonsChess Llm-Synthesized Code World Models
Multi-Mixer Models: Flexible Sequence Modeling with Shared Representations
May 27, 2026Kevin Y. Li, Asher Trockman, Ananda Theertha Suresh +1Recurrent Model Softmax Attention
Learning Compositional Latent Structure with Vector Networks
May 27, 2026Niclas Pokel, Benjamin F. GreweModel Weights Recurrent Model
Fine-Tuning Dynamics of In-Context Factual Recall in Transformers
May 26, 2026Ruomin Huang, Eshaan Nichani, Jason D. Lee +1In-Context Learning In-Context

Sep 15, 2026 · cs.CLJ/K move · Enter open · S save

Persistent Recurrent Memory Between Transformer Layers - Improves Language Model Generalization

Eduardo Novaes Hering

We introduce a simple architectural modification to decoder-only transformers: a persistent recurrent state that observes hidden representations via cross-attention, updates itself through a GRU, and modulates subsequent processing via gated addition. Inserted between the lower and upper halves of a 6-layer transformer, this module adds only 3.7% additional parameters while reducing evaluation loss from

2.438 \pm 0.004

1.743 \pm 0.018

, corresponding to a 28.5% reduction on held-out language modeling data. The improvement is statistically significant across 5 random seeds (

p < 0.01

) and corresponds to reduced overfitting (generalization gap 0.12 vs 0.26). Through controlled ablations, we demonstrate that the improvement stems entirely from the persistent memory topology, not from auxiliary self-prediction objectives. A model with identical topology but no auxiliary loss performs equivalently, while a random auxiliary loss provides no benefit. Representation probing reveals that the persistent state encodes narrative position (52% vs 33% chance level)---information that standard attention maintains less efficiently. Our results suggest that bridging transformer layers with a lightweight recurrent memory is a simple, effective approach to improving generalization in small-scale language models.