Linear RNNs

RNN: Recurrent Neural Network

Latest papers 14

All topics
CardsList
  1. How Temporal Correlations Shape Memory in Linear Recurrent Neural Networks

    Aug 31, 2026Arnol Manuel Fokam, Fasseu Sieyondji Akpevwoghene, Edem Fiifi DawsonRecurrent Neural NetworksLinear RNNs

  2. Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity

    Jul 8, 2026Loïc Cabannes, Pierre-Emmanuel Mazaré, Gergely Szilvasy +6Long-Context RetrievalMemory-Augmented Language Models

  3. Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks

    May 29, 2026Ezekiel Williams, Alexandre Payeur, Guillaume LajoieRecurrent Neural NetworksLinear RNNs

  4. Why Linear Recurrent Memory Works in Partially Observable Reinforcement Learning

    May 29, 2026Yike Zhao, Onno Eberhard, Malek Khammassi +2Partially Observable RLMarkov Models

  5. Multi-Mixer Models: Flexible Sequence Modeling with Shared Representations

    May 27, 2026Kevin Y. Li, Asher Trockman, Ananda Theertha Suresh +1Transformer AttentionLong-Context Modeling

  6. Kaczmarz Linear Attention

    May 9, 2026Jiaxuan Zou, Ruifeng Ren, Yong LiuLong-Context Language ModelingRecurrent Transformers

  7. MDN: Parallelizing Stepwise Momentum for Delta Linear Attention

    May 7, 2026Yulong Huang, Xiang Liu, Hongxiang Huang +5Momentum MethodsLinear Attention

  8. How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrences

    May 6, 2026Mariia SeleznovaNeural Network InitializationRecurrent Neural Networks

  9. Learning State-Tracking from Code Using Linear RNNs

    Feb 16, 2026Julien Siems, Riccardo Grazzi, Korbinian Pöppel +3Recurrent Neural NetworksState Tracking

  10. Learning in the Recurrent State: Gradient Descent with Linear Recurrent Networks

    Oct 15, 2024Yudou Tian, Neeraj Mohan Sushma, Harshvardhan Mestha +3In-Context LearningLinear RNNs

  11. RotRNN: Modelling Long Sequences with Rotations

    Jul 9, 2024Kai Biegun, Rares Dolga, Jake Cunningham +1Recurrent Neural NetworksLong-Context Modeling