Recurrent Neural Networks

Also known as RNN

Recurrent neural networks study how a model can process ordered inputs by repeatedly updating an internal state, which is useful for streaming data, long-term memory, and decisions under partial observability. Common approaches include gated LSTMs and GRUs, linear and state-space recurrences, continuous-time and spiking networks, and weight-shared loops for iterative reasoning; training ranges from backpropagation through time to online learning and parallel scans. Researchers test these models on state-tracking and recall tasks, language modeling, time-series forecasting, control, and physical-system prediction. A central tension is how to preserve useful information and stable learning over long sequences while keeping computation efficient and matching the accuracy and flexibility of Transformers.

Latest papers

302 papers47 papers in the last four weeks, up 15% on the four weeks before. 0.3% of all new papers.

C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions
Apr 15, 2026Kenji Kubo, Shunsuke Kamiya, Masanori Koyama +3Test-Time Scaling Neural Algorithmic Reasoning
Graduate School of Engineering, The University of Tokyo
Anomaly Detection in IEC-61850 GOOSE Networks: Evaluating Unsupervised and Temporal Learning for Real-Time Intrusion Detection
Apr 14, 2026Joseph MooreNetwork Intrusion Detection Unsupervised Anomaly Detection
Computer Science, Boise State University, Boise, Idaho USA
Predicting Blastocyst Formation in IVF: Integrating DINOv2 and Attention-Based LSTM on Time-Lapse Embryo Images
Apr 14, 2026Zahra Asghari Varzaneh, Niclas Wölner-Hanssen, Reza Khoshkangini +2Microscopy Image Analysis Recurrent Neural Networks
Sustainable Digitalisation Research Center, Department of Computer Science and Media Technology, Malmö University, Malmö, Sweden · School of Information Technology, Halmstad University, Halmstad, Sweden · Kepler Universitätsklinikum, Linz, Austria +1
A Mechanistic Analysis of Looped Reasoning Language Models
Apr 13, 2026Hugh Blayney, Álvaro Arroyo, Johan Obando-Ceron +4Recurrent Transformers LLM Interpretability
University of Oxford · Mila – Quebec AI Institute · Universit´e de Montréal +1
Geometry-Induced Long-Range Correlations in Recurrent Neural Network Quantum States
Apr 9, 2026Asif Bin Ayub, Amine Mohamed Aboussalah, Mohamed Hibat-AllahNeural Quantum States Recurrent Neural Networks
Department of Applied Mathematics, University of Waterloo, Waterloo, ON N2L 3G1, Canada · Perimeter Institute for Theoretical Physics, 31 Caroline St N, Waterloo, ON N2L 2Y5, Canada · NYU Tandon School of Engineering, 6 MetroTech Center, Brooklyn, New York 11201, United States +1
Introducing Echo Networks for Computational Neuroevolution
Apr 9, 2026Christian Kroos, Fabian KüchNeuroevolution Recurrent Neural Networks
Audio & Media Technologies Fraunhofer Institute for Integrated Circuits, IIS Erlangen, Germany
Contextual Control without Memory Growth in a Context-Switching Task
Apr 3, 2026Song-Ju KimRecurrent Neural Networks
SOBIN Institute LLC, Kawanishi, Hyogo, Japan
Activity-Dependent Plasticity in Morphogenetically-Grown Recurrent Networks
Apr 3, 2026Sergii Medvid, Andrii Valenia, Mykola GlybovetsRecurrent Neural Networks Neuroevolution
National University of Kyiv-Mohyla Academy, Kyiv, Ukraine
Time-Warping Recurrent Neural Networks for Transfer Learning
Apr 2, 2026Jonathon HirschiTransfer Learning Recurrent Neural Networks
University of Colorado Denver
On the Role of Depth in the Expressivity of RNNs
Apr 2, 2026Maude Lizaire, Michael Rizvi-Martel, Éric Dupuis +1Recurrent Neural Networks Neural Network Expressivity
Mila & DIRO, Université de Montréal · Independent · Mila & DIRO, CIFAR AI Chair, Université de Montréal
Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization
Apr 2, 2026Shota Takashiro, Masanori Koyama, Takeru Miyato +3OOD Generalization Recurrent Neural Networks
The University of Tokyo, Tokyo, Japan · University of Tübingen, Tübingen, Germany
LAtent Phase Inference from Short time sequences using SHallow REcurrent Decoders (LAPIS-SHRED)
Apr 1, 2026Yuxuan Bao, Xingyue Zhang, J. Nathan KutzSpatiotemporal Forecasting Latent Dynamics Modeling
Department of Applied Mathematics, University of Washington. · School of Environmental and Forest Sciences, University of Washington. · Autodesk Research, London, UK.
A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR
Apr 1, 2026Merveilles Agbeti-Messan, Pierrick Tranouez, Stéphane Nicolas +2Selective SSMs Transformer
LITIS EA4108, University of Rouen Normandy, France · LITIS EA4108, INSA of Rouen Normandy, France
Immediate Derivatives Suffice for Online Recurrent Adaptation
Mar 30, 2026Aur Shalev MerinRecurrent Neural Networks Gradient Descent Dynamics
Independent Researcher
Identifying Connectivity Distributions from Neural Dynamics Using Flows
Mar 27, 2026Timothy Doyeon Kim, Ulises Pereira-Obilinovic, Yiliu Wang +2Query Identifiability Recurrent Neural Networks
Allen Institute, Seattle, WA, USA · University of Washington, Seattle, WA, USA
Towards Safe Learning-Based Non-Linear Model Predictive Control through Recurrent Neural Network Modeling
Mar 25, 2026Mihaela-Larisa Clement, Mónika Farsang, Agnes Poks +4Nonlinear MPC Model Predictive Control
Institute of Computer Engineering, TU Wien, Vienna, Austria · AIT Austrian Institute of Technology, Vienna, Austria · Institute of Mechanics and Mechatronics, TU Wien, Vienna, Austria
Fusing Memory and Attention: A study on LSTM, Transformer and Hybrid Architectures for Symbolic Music Generation
Mar 22, 2026Soudeep Ghoshal, Sandipan Chakraborty, Pradipto Chowdhury +1Hybrid Transformer Architectures Symbolic Music Generation
Kalinga Institute of Industrial Technology (KIIT), Bhubaneswar, India · IMC University of Applied Sciences Krems, Krems, Austria
Recovering Sparse Neural Connectivity from Partial Measurements: A Covariance-Based Approach with Granger-Causality Refinement
Mar 19, 2026Quilee SimeonBrain Network Analysis Sparse Recovery
Massachusetts Institute of Technology
Massive Redundancy in Gradient Transport Enables Sparse Online Learning
Mar 16, 2026Aur Shalev MerinOnline Learning Recurrent Neural Networks
Independent Researcher
M $^2$ RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling
Mar 15, 2026Mayank Mishra, Shawn Tan, Ion Stoica +2Recurrent Neural Networks Language Modeling
UC Berkeley · Learning Machine · MIT-IBM Watson Lab +2
Distributed Acoustic Sensing for Urban Traffic Monitoring: Spatio-Temporal Attention in Recurrent Neural Networks
Mar 14, 2026Izhan Fakhruzi, Manuel Titos, Carmen Benítez +1Spatio-Temporal Attention Distributed Acoustic Sensing
Department of Signal Theory, Telematics and Communications, and the Center for Information and Communication Technologies Research (CITIC), University of Granada, 18014 Granada, Spain
Surrogate models for nuclear fusion with parametric Shallow Recurrent Decoder Networks: applications to magnetohydrodynamics
Mar 11, 2026M. Lo Verso, C. Introini, E. Cervi +3Magnetohydrodynamics Neural Surrogate Modeling
Department of Energy, Politecnico di Milano, Milano, 20133, Italy · MATHEP Group, Dept. of Energy "Galileo Ferraris", Politecnico di Torino, Torino, Italy · Autodesk Research, 6 Agar Street, London UK +1
SCORE: Replacing Layer Stacking with Contractive Recurrent Depth
Mar 11, 2026Guillaume GodinResidual Learning Recurrent Neural Networks
1Osmo Labs PBC New York, USA
JEDI: Jointly Embedded Inference of Neural Dynamics
Mar 11, 2026Anirudh Jamkhandi, Ali Korojy, Olivier Codol +2Latent Dynamics Modeling Dynamical Systems
Mila - Quebec AI Institute · Université de Montréal
The Costs of Reproducibility in Music Separation Research: a Replication of Band-Split RNN
Mar 10, 2026Paul Magron, Romain Serizel, Constance DouwesMusic Source Separation Audio Source Separation
Université de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France · Centrale Med, Aix Marseille Univ, CNRS, LIS, Marseille, France
TA-RNN-Medical-Hybrid: A Time-Aware and Interpretable Framework for Mortality Risk Prediction
Mar 9, 2026Zahra Jafari, Azadeh Zamanifar, Amirfarhad FarhadiMortality Prediction Longitudinal EHR Modeling
Department of Computer Engineering, SRC.,Islamic Azad university, Tehran, Iran · School of Computer Engineering, Iran university of Science and technology, Tehran, Iran
SeedPolicy: Horizon Scaling via Self-Evolving Diffusion Policy for Robot Manipulation
Mar 5, 2026Youqiang Gui, Yuxuan Zhou, Shen Cheng +4Robot Imitation Learning Long-Horizon Robotic Manipulation
Sichuan University · Independent Researcher · Dexmal Inc. +1
Why Are Linear RNNs More Parallelizable?
Mar 4, 2026William Merrill, Hongjian Jiang, Yanhong Li +2Circuit Complexity Recurrent Neural Networks
Allen Institute for AI · Rheinland-Pfälzische Technische Universität · Max-Planck Institute for Software Systems
Symbol-Equivariant Recurrent Reasoning Models
Mar 2, 2026Richard Freinschlag, Timo Bertram, Erich Kobler +2Permutation-Equivariant Neural Networks Sudoku Solving
ELLIS Unit Linz, LIT AI Lab & Institute for Machine Learning, Johannes Kepler University Linz, Austria · Clinical Research Institute for Medical AI, Johannes Kepler University Linz, Austria
Bi-cLSTM: Residual-Corrected Bidirectional LSTM for Aero-Engine RUL Estimation
Feb 28, 2026Rafi Hassan Chowdhury, Nabil Daiyan, Faria Ahmed +2RUL Estimation Predictive Maintenance
Department of Computer Science and Engineering, Rajshahi University of Engineering and Technology, Rajshahi, Bangladesh · Department of Aeronautical Engineering, Military Institute of Science and Technology, Dhaka 1216, Bangladesh

Apr 15, 2026J/K move · Enter open · S save

C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions

Kenji Kubo, Shunsuke Kamiya, Masanori Koyama, Kohei Hayashi+2

Graduate School of Engineering, The University of Tokyo

Test-Time Scaling Neural Algorithmic Reasoning Recurrent Neural Networks Majority Voting

Open paper AskPDF arXiv

Neural network models with latent recurrent processing, where identical layers are recursively applied to the latent state, have gained attention as promising models for performing reasoning tasks. A strength of such models is that they enable test-time scaling, where the models can enhance their performance in the test phase without additional training. Models such as the Hierarchical Reasoning Model (HRM) and Artificial Kuramoto Oscillatory Neurons (AKOrN) can facilitate deeper reasoning by increasing the number of recurrent steps, thereby enabling the completion of challenging tasks, including Sudoku, Maze solving, and AGI benchmarks. In this work, we introduce confidence-based voting (C-voting), a test-time scaling strategy designed for recurrent models with multiple latent candidate trajectories. Initializing the latent state with multiple candidates using random variables, C-voting selects the one maximizing the average of top-1 probabilities of the predictions, reflecting the model's confidence. Additionally, it yields 4.9% higher accuracy on Sudoku-hard than the energy-based voting strategy, which is specific to models with explicit energy functions. An essential advantage of C-voting is its applicability: it can be applied to recurrent models without requiring an explicit energy function. Finally, we introduce a simple attention-based recurrent model with randomized initial values named ItrSA++, and demonstrate that when combined with C-voting, it outperforms HRM on Sudoku-extreme (95.2% vs. 55.0%) and Maze (78.6% vs. 74.5%) tasks.