\Sigma}{\Delta}$ Low Pas RNN

ΣΔ low-pass recurrent neural networks (RNNs) represent a specialized RNN architecture designed for efficient and accurate mapping onto neuromorphic hardware. Current research focuses on improving the precision and energy efficiency of these networks, often employing adaptive spiking neuron models and ΣΔ modulation for signal encoding. This approach is particularly relevant for resource-constrained applications requiring real-time inference, such as edge computing and embedded systems, where power efficiency is paramount. The development of robust and accurate ΣΔ lpRNNs promises significant advancements in low-power AI hardware and its deployment in various real-world scenarios.

Papers

May 27, 2024

Building a temperature forecasting model for the city with the regression neural network (RNN)
Nguyen Phuc Tran, Duy Thanh Tran, Thi Thuy Nga Duong
Neural Network Recurrent Neural Network Predictive Model Novel Regression Urban Environment \Sigma}{\Delta}$ Low Pas RNN Weather Prediction Model Accurate Predictive Model Urban Temperature

May 14, 2024

Gradient-Free Training of Recurrent Neural Networks using Random Perturbations
Jesus Garcia Fernandez, Sander Keemink, Marcel van Gerven
Recurrent Neural Network Back Propagation \Sigma}{\Delta}$ Low Pas RNN Gradient Free Random Perturbation

May 6, 2024

DeltaKWS: A 65nm 36nJ/Decision Bio-inspired Temporal-Sparsity-Aware Digital Keyword Spotting IC with 0.6V Near-Threshold SRAM
Qinyu Chen, Kwantae Kim, Chang Gao, Sheng Zhou, Taekwang Jang, Tobi Delbruck, Shih-Chii Liu
Decision Relevant Information \Sigma}{\Delta}$ Low Pas RNN Google Speech Command

May 3, 2024

Enhancing Bangla Language Next Word Prediction and Sentence Completion through Extended RNN with Bi-LSTM Model On N-gram Language
Md Robiul Islam, Al Amin, Aniqua Nusrat Zereen
Sentence Level \Sigma}{\Delta}$ Low Pas RNN N Gram Bi LSTM Bangla Natural Language

April 24, 2024

Neural Proto-Language Reconstruction
Chenxuan Cui, Ying Chen, Qinxin Wang, David R. Mortensen
Neural Machine Translation Model \Sigma}{\Delta}$ Low Pas RNN Protoform Reconstruction

March 17, 2024

Graph Expansion in Pruned Recurrent Neural Network Layers Preserve Performance
Suryam Arnav Kalra, Arindam Biswas, Pabitra Mitra, Biswajit Basu
Neural Network Graph Neural Network Recurrent Neural Network System Performance Recurrent Network Traditional RNNs \Sigma}{\Delta}$ Low Pas RNN Graph Expansion

March 4, 2024

adaptNMT: an open-source, language-agnostic development environment for Neural Machine Translation
Séamus Lankford, Haithem Afli, Andy Way
Machine Translation Neural Machine Translation Neural Machine Translation Model \Sigma}{\Delta}$ Low Pas RNN Language Agnostic

February 29, 2024

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando De Freitas, Caglar Gulcehre
Language Model Recurrent Neural Network Long Sequence \Sigma}{\Delta}$ Low Pas RNN Fast Inference Local Context Recurrence Relation

February 21, 2024

Data-Driven Dynamic Friction Models based on Recurrent Neural Networks
Joaquin Garcia-Suarez
Neural Network Recurrent Neural Network \Sigma}{\Delta}$ Low Pas RNN Friction Model Positive Friction

February 7, 2024

Opening the AI black box: program synthesis via mechanistic interpretability
Eric J. Michaud, Isaac Liao, Vedang Lad, Ziming Liu, Anish Mudide, Chloe Loughridge, Zifan Carl Guo, Tara Rezaei Kheirkhah, Mateja Vukelić, Max Tegmark
Artificial Intelligence Black Box Program Synthesis Learning Algorithm Mechanistic Interpretability \Sigma}{\Delta}$ Low Pas RNN

December 12, 2023

CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor
Shuyang Sun, Runjia Li, Philip Torr, Xiuye Gu, Siyang Li
Segmentation Task Single CLIP \Sigma}{\Delta}$ Low Pas RNN Segmentation Dataset Mask Prediction Image Text Datasets Training Objective Open Vocabulary Image Segmentation

December 6, 2023

Exploring Answer Information Methods for Question Generation with Transformers
Talha Chafekar, Aafiya Hussain, Grishma Sharma, Deepak Sharma
Transformer Megatron Decepticons Question Generation \Sigma}{\Delta}$ Low Pas RNN Multiple Target Automatic Evaluation Metric Answer Cross Embeddings

December 2, 2023

ResNLS: An Improved Model for Stock Price Forecasting
Yuanzhe Jia, Ali Anaissi, Basem Suleiman
LSTM Network \Sigma}{\Delta}$ Low Pas RNN Model Improvement Stock Price Prediction Stock Price Forecasting

November 29, 2023

DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generation
Aleksandr Timofeev, Anastasiia Fadeeva, Andrei Afonin, Claudiu Musat, Andrii Maksai
Generative Model Data Augmentation Encoder Decoder Model \Sigma}{\Delta}$ Low Pas RNN Long Form Generation Long Text Data Digital Ink

October 26, 2023

Bifurcations and loss jumps in RNN training
Lukas Eisenmann, Zahra Monfared, Niclas Alexander Göring, Daniel Durstewitz
Recurrent Neural Network \Sigma}{\Delta}$ Low Pas RNN Bifurcation Diagram RNN T Training Loss Spike

September 22, 2023

Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks
Sina Khajehabdollahi, Roxana Zeraati, Emmanouil Giannakakis, Tim Jakob Schäfer, Georg Martius, Anna Levina
Recurrent Neural Network System Performance \Sigma}{\Delta}$ Low Pas RNN Timescale Analysis Recurrent Connection Neuronal Dynamic

July 21, 2023

Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues
Antonio Orvieto, Soham De, Caglar Gulcehre, Razvan Pascanu, Samuel L. Smith
\Sigma}{\Delta}$ Low Pas RNN K$ Universality Smallest Eigenvalue Causal Order Recurrence Relation Linear RNN

June 24, 2023

May 27, 2023

A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU
Farhad Mortezapour Shiri, Thinagaran Perumal, Norwati Mustapha, Raihani Mohamed
Deep Learning Deep Learning Model Long Short Term Memory CNN Network \Sigma}{\Delta}$ Low Pas RNN Gated Recurrent Unit

\Sigma}{\Delta}$ Low Pas RNN

Papers

Building a temperature forecasting model for the city with the regression neural network (RNN)

Gradient-Free Training of Recurrent Neural Networks using Random Perturbations

DeltaKWS: A 65nm 36nJ/Decision Bio-inspired Temporal-Sparsity-Aware Digital Keyword Spotting IC with 0.6V Near-Threshold SRAM

Enhancing Bangla Language Next Word Prediction and Sentence Completion through Extended RNN with Bi-LSTM Model On N-gram Language

Neural Proto-Language Reconstruction

Graph Expansion in Pruned Recurrent Neural Network Layers Preserve Performance

adaptNMT: an open-source, language-agnostic development environment for Neural Machine Translation

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

Data-Driven Dynamic Friction Models based on Recurrent Neural Networks

Opening the AI black box: program synthesis via mechanistic interpretability

CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor

Exploring Answer Information Methods for Question Generation with Transformers

ResNLS: An Improved Model for Stock Price Forecasting

DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generation

Bifurcations and loss jumps in RNN training

Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks

Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues

Intensity-free Convolutional Temporal Point Process: Incorporating Local and Global Event Contexts

Weighted Automata Extraction and Explanation of Recurrent Neural Networks for Natural Language Tasks

A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU