Recurrent Neural Networks

Also known as RNN

Recurrent neural networks study how a model can process ordered inputs by repeatedly updating an internal state, which is useful for streaming data, long-term memory, and decisions under partial observability. Common approaches include gated LSTMs and GRUs, linear and state-space recurrences, continuous-time and spiking networks, and weight-shared loops for iterative reasoning; training ranges from backpropagation through time to online learning and parallel scans. Researchers test these models on state-tracking and recall tasks, language modeling, time-series forecasting, control, and physical-system prediction. A central tension is how to preserve useful information and stable learning over long sequences while keeping computation efficient and matching the accuracy and flexibility of Transformers.

Latest papers

303 papers47 papers in the last four weeks, up 15% on the four weeks before. 0.3% of all new papers.

Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity
Jul 8, 2026Loïc Cabannes, Pierre-Emmanuel Mazaré, Gergely Szilvasy +6Long-Context Retrieval Memory-Augmented Language Models
Meta FAIR · Inria Paris & ENS-PSL University · University of Tübingen
When Does Tool Use Increase the Expressive Power of Finite-Precision Recurrent Models?
Jul 7, 2026Nikola Zubić, Qian Li, Yuyi Wang +1Recurrent Neural Networks Neural Network Expressivity
Robotics and Perception Group, University of Zurich · Shenzhen International Center for Industrial and Applied Mathematics, Shenzhen Research Institute of Big Data · Tengen Intelligence Institute, CRRC Zhuzhou Institute
SHiPPO: Recurrent Memory with Transported Polynomial Projections
Jul 3, 2026Tomoya Mizuguchi, Bum Jun KimSelective SSMs Recurrent Neural Networks
Kyoto University, Japan · The University of Tokyo, Japan
Auditing Finite-Lag Statistics of Recurrent Representations
Jul 2, 2026Kanishka ReddyNeural Representation Geometry Recurrent Neural Networks
Department of Applied Mathematics, University of Washington, Seattle, WA 98105
StateFlow: Dual-State Recurrent Modeling for Long-Horizon Time Series Forecasting
Jun 30, 2026Haroon Gharwi, Yue Dai, Kai ShuMultivariate Time Series Forecasting Long-Term Time Series Forecasting
Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA · Department of Computer Science, Emory University, Atlanta, GA, USA
Anti-Collapse Dynamics and the Emergence of Multi-Time-Scale Learning in Recurrent Neural Networks
Jun 28, 2026Lorenzo LiviHeavy-Tailed Noise Recurrent Neural Networks
OPIT – Open Institute of Technology
LNN-Fly: Continuous-Time UAV Navigation for Robust Obstacle Avoidance under Timing Mismatch
Jun 27, 2026Yulin Huang, Shaojie Chen, Di Feng +3Aerial Robotics Robot Navigation
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, Guangdong 518000, China
Heterogeneous synaptic motifs bridge microscale structure and macroscale nonlinear dynamics
Jun 26, 2026Meiyi Zhang, Jinjian Yu, Louis Tao +1Dynamical Systems Recurrent Neural Networks
Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, China · Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China · School of Physics, Peking University, China +7
CANNs: A Toolkit for Research on Continuous Attractor Neural Networks
Jun 26, 2026Sichao He, Aiersi Tuerhong, Shangjun She +4Neural Representation Geometry Recurrent Neural Networks
School of Psychological and Cognitive Sciences, Peking–Tsinghua Center for Life Sciences, PKU-IDG/McGovern Institute for Brain Research, Center of Quantitative Biology, Peking University, Beijing, China · College of Mathematics and Statistics, Chongqing University, Chongqing, China
Beyond Feedforward Networks: Reentry Neural Systems as the Fundamental Basis of Subjecthood and Intrinsic Safety of Next-Generation AGI
Jun 24, 2026A. S. Ushakov, Yu. N. BerdinskRecurrent Neural Networks AI Agent Safety
Saint Petersburg State University, Department of High Energy Physics and Elementary Particles
Two-dimensional Hyperbolic RNN Neural Quantum State
Jun 24, 2026H. L. DaoNeural Quantum States Recurrent Neural Networks
Frequency Domain Reservoir Computing
Jun 23, 2026Klaus Schertler, Xiomara Runge, Andrea Ceni +2Time Series Forecasting Recurrent Neural Networks
Airbus Central Research & Technology Taufkirchen, Germany · Department of Computer Science University of Pisa, Italy · Faculty of Technology Bielefeld University, Germany
Identifying structural design principles shaping the computational abilities of recurrent neural networks
Jun 22, 2026Tom Talpir, Elad SchneidmanRecurrent Neural Networks Neural Network Expressivity
Department of Brain Sciences, Weizmann Institute of Science, Rehovot, Israel
Recursive QLSTM with Dynamic Variational Quantum Circuit Adaptation
Jun 22, 2026Samuel Yen-Chi Chen, Yifeng Peng, Jiun-Cheng Jiang +8Quantum Machine Learning Recurrent Neural Networks
Wells Fargo · Stevens Institute of Technology · National Taiwan University +3
FLYNN: Robust Neural Network for Robot Navigation using Fly Brain Topology
Jun 21, 2026Benquan Wang, Jingdao ChenRobot Navigation Neural Network Robustness
Department of Computer Science Engineering, Mississippi State University, Mississippi State, MS 39762, USA
Soliton-like Waves in a Two-Dimensional Recurrent Spiking Neural Network with Weighted Spike-Timing-Dependent Plasticity
Jun 19, 2026Ch. MeessenDynamical Systems Recurrent Neural Networks
Independent researcher, computational neuroscience, France
Recurrent neural networks approximate continuous functions
Jun 18, 2026Valentin Abadie, Clemens Hutter, Helmut BölcskeiReLU Neural Networks Recurrent Neural Networks
Linear Recurrent Unit with Semantic Modulation for Image Super-Resolution
Jun 18, 2026Mingyu Choi, Woo Kyoung Han, Sunghoon Im +1Recurrent Neural Networks Image Super-Resolution
Korea University · DGIST
Urdu Katib Handwritten Dataset: A Historical Document Dataset for Offline Urdu Handwritten Text Recognition with CRNN-Based Baseline Evaluation
Jun 17, 2026Ramza Basharat, Muhammad Usman AliConvolutional RNNs Recurrent Neural Networks
Department of Computer Science, University of Gujrat, Pakistan · Department of Computer Science, University of Gujrat, Gujrat, Punjab, Pakistan
Explicit Interaction Architectures for Dynamical Learning: A Controlled Study of Structural Inductive Bias
Jun 17, 2026Augusto SartiDynamic Neural Networks Nonlinear System Identification
Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Piazza L. Da Vinci 32, 20133 Italy.
Task-Restricted Symmetries in Recurrent Weight Space
Jun 16, 2026Simon DrägerRecurrent Neural Networks
Salk Institute for Biological Studies, La Jolla, CA, USA.
Why SWAVE May Not Be All You Need:A Concept-Evolution Retrospective on Complex-Valued Recurrent Language Models
Jun 16, 2026Ramprasath Ganesaraja, Swathika N, Sahil Dilip PanseRecurrent Neural Networks Complex-Valued Neural Networks
EdgeVerve Systems Limited
From Compression to Deployment: Real-Time and Energy-Efficient FastGRNN on Ultra-Constrained Microcontrollers
Jun 15, 2026Emre Can KizilatesGated Recurrent Unit Efficient Neural Network Inference
Independent Researcher, Izmir, Turkey
When the Past Matters: FlashBack Memory for Precipitation Nowcasting
Jun 15, 2026Yuhao Du, Boxiao Huang, Chengrong Wu +1Memory-Augmented Neural Networks Precipitation Forecasting
College of Atmospheric Sciences, Lanzhou University, Lanzhou 730000, China · Fuqua School of Business, Duke University, North Carolina 27705, USA · Department of Computer Science, University of Manchester, Manchester M139PL, UK
Harnessing cortical geometry, wiring, and function as inductive biases for recurrent neural networks
Jun 12, 2026Mo Shakiba, Rana Rokni, Mohammad Mohammadi +1Recurrent Neural Networks
Neuromatch Academy, Neuromatch, Inc., USA · McGovern Institute for Brain Research, Massachusetts Institute of Technology (MIT)
Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models
Jun 12, 2026Rituraj Sharma, Tu VuRecurrent Neural Networks Recurrent Transformers
Virginia Tech
Robustness Verification of Recurrent Neural Networks with Abstraction Refinement
Jun 10, 2026Li-Jen Lin, Chih-Duo HongNeural Network Verification Neural Network Robustness
Dept. of Management Information Systems National Chengchi University Taipei, Taiwan
Masked Neural Detection for Constrained Channel Coding in Molecular Communication
Jun 10, 2026Melih Şahin, Ozgur B. AkanNeural Decoding Recurrent Neural Networks
Centre for neXt Communications (CXC), Department of Engineering, University of Cambridge, CB3 0FA Cambridge, U.K. · Centre for neXt Communications (CXC), Department of Electrical and Electronics Engineering, Koç University, 34450 Istanbul, Türkiye
LSTM-Based Detection of Structural Breaks in Property Insurance Loss Reserving: A Climate-Informed Approach
Jun 9, 2026Thomas Mbrice, Shashwat PanigrahiRecurrent Neural Networks Change-Point Detection
Stony Brook University · Department of Computer Science
Towards Critical Branching Mechanism in Recurrent Neural Networks
Jun 9, 2026Feixiang Ren, Ling FengRecurrent Neural Networks Long Short-Term Memory Networks
Department of Physics, National University of Singapore Singapore · Institute of High Performance Computing Agency for Science, Technology and Research (A*STAR) Singapore

Jul 8, 2026J/K move · Enter open · S save

Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity

Loïc Cabannes, Pierre-Emmanuel Mazaré, Gergely Szilvasy, Matthijs Douze+5

Meta FAIR · Inria Paris & ENS-PSL University · University of Tübingen

Long-Context Retrieval Memory-Augmented Language Models Recurrent Neural Networks Delta-Rule Linear Attention In-Context Learning Gated Delta Networks+1

Open paper AskPDF arXiv

Linear attention models allow a fixed state size and a fixed amount of compute per token. However, due to their limited state size, linear attention models fall behind in long-context recall compared to softmax-attention-based transformer architectures. Increasing the state size of linear attention improves recall performance but at the cost of higher FLOPs. In this work, we introduce Sparse Delta Memory (SDM), an architecture that scales the hidden state of gated linear RNNs to orders of magnitude higher capacity using a sparse addressing scheme. SDM extends the Gated DeltaNet architecture by replacing the dense key-value outer product with sparse reads and writes to a large explicit memory. We show that, under an isoFLOP constraint and with an identical number of parameters, a higher state memory capacity significantly improves performance on in-context learning and long-context retrieval tasks. Moreover, by learning the initial state of the SDM memory and therefore using it as a parametric memory, we show that the model further improves on a wide range of common-knowledge and reasoning tasks.