RNN Layer

Recurrent neural network (RNN) layers are fundamental components in processing sequential data, but recent research explores their application beyond traditional time series analysis, including image processing and natural language processing. Current research focuses on enhancing RNN expressiveness for long-range dependencies, improving efficiency through novel architectures like test-time training layers and optimized bidirectional RNNs, and developing methods to better understand and characterize their performance across diverse datasets. These advancements aim to improve the accuracy and efficiency of RNNs in various applications, from resource-constrained embedded systems to large-scale language models.

Papers

September 10, 2024

Recurrent Neural Networks for Still Images
Dmitri (Dima)Lvov, Yair Smadar, Ran Bezen
Recurrent Neural Network Traditional RNNs Convolutional Recurrent Neural Network RNN Layer

July 5, 2024

Learning to (Learn at Test Time): RNNs with Expressive Hidden States
Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, Carlos Guestrin
LeArning Abstract Test Time Two Layer Neural Network Traditional RNNs Test Time Training Hidden State RNN Layer

November 28, 2023

On the Long Range Abilities of Transformers
Itamar Zimerman, Lior Wolf
Transformer Megatron Decepticons Transformer Architecture Long Range Global Convolution RNN Layer

August 4, 2023

Universal Approximation of Linear Time-Invariant (LTI) Systems through RNNs: Power of Randomness in Reservoir Computing
Shashank Jere, Lizhong Zheng, Karim Said, Lingjia Liu
Recurrent Neural Network Reservoir Computing Universal Approximation Inherent Randomness Untrained Neural Network RNN Layer

July 28, 2023

A Distance Correlation-Based Approach to Characterize the Effectiveness of Recurrent Neural Networks for Time Series Forecasting
Christopher Salazar, Ashis G. Banerjee
Recurrent Neural Network Time Series Forecasting RNN Model Correlation Analysis RNN Layer

October 17, 2022

Temporal-Spatial dependencies ENhanced deep learning model (TSEN) for household leverage series forecasting
Hu Yang, Yi Huang, Haijun Wang, Yu Chen
Deep Learning Model Spatio Temporal Temporal Dependency Financial Time Series Superior Forecasting RNN Layer

December 28, 2021

A Preordered RNN Layer Boosts Neural Machine Translation in Low Resource Settings
Mohaddeseh Bastan, Shahram Khadivi
Neural Machine Translation Translation Quality Syntactic Information RNN Layer

RNN Layer

Papers

Recurrent Neural Networks for Still Images

Learning to (Learn at Test Time): RNNs with Expressive Hidden States

On the Long Range Abilities of Transformers

Universal Approximation of Linear Time-Invariant (LTI) Systems through RNNs: Power of Randomness in Reservoir Computing

A Distance Correlation-Based Approach to Characterize the Effectiveness of Recurrent Neural Networks for Time Series Forecasting

Temporal-Spatial dependencies ENhanced deep learning model (TSEN) for household leverage series forecasting

A Preordered RNN Layer Boosts Neural Machine Translation in Low Resource Settings