Static Deep Neural Network

Static deep neural networks (DNNs), while powerful, face limitations in adapting to dynamic data or resource constraints. Current research focuses on enhancing their efficiency and robustness through techniques like compressed replay for continual learning, optimized compilation frameworks for dynamic DNN variants, and novel architectures such as hypernetworks for improved performance at lower computational costs. These advancements aim to improve the practicality and reliability of DNNs across diverse applications, particularly in resource-limited environments and scenarios requiring adaptation to evolving data streams.

Papers

July 17, 2024

FETCH: A Memory-Efficient Replay Approach for Continual Learning in Image Classification
Markus Weißflog, Peter Protzel, Peer Neubert
Continual LEArning Image Classification Replay Based Class Incremental Continual Learning FETCH Robot Static Deep Neural Network Replay Training

May 29, 2024

Valid Conformal Prediction for Dynamic GNNs
Ed Davis, Ian Gallagher, Daniel John Lawson, Patrick Rubin-Delanchy
Conformal Prediction Dynamic Graph Neural Network Unfolded Graph Neural Network Static Deep Neural Network

February 29, 2024

SoD$^2$: Statically Optimizing Dynamic Deep Neural Network
Wei Niu, Gagan Agrawal, Bin Ren
Code Generation DNN Framework Static Deep Neural Network Dynamic Deep Memory Allocation

February 23, 2024

MSPipe: Efficient Temporal GNN Training via Staleness-aware Pipeline
Guangming Sheng, Junwei Su, Chao Huang, Chuan Wu
Temporal Dependency Temporal Graph Temporal Graph Neural Network Temporal GNN Static Deep Neural Network

January 17, 2024

Fluid Dynamic DNNs for Reliable and Adaptive Distributed Inference on Edge Devices
Lei Xun, Mingyu Hu, Hengrui Zhao, Amit Kumar Singh, Jonathon Hare, Geoff V. Merrett
Edge Device DNN Model Fluid Dynamic Distributed Inference Static Deep Neural Network

August 17, 2023

Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks
Mirazul Haque, Wei Yang
Neural Network Deep Neural Network Native Robustness Dynamic Neural Network Static Deep Neural Network

May 27, 2023

Input-Aware Dynamic Timestep Spiking Neural Networks for Efficient In-Memory Computing
Yuhang Li, Abhishek Moitra, Tamar Geller, Priyadarshini Panda
Neural Network Spiking Neural Network Memory Computing Single Neural Network Binary Spike Static Deep Neural Network

October 17, 2022

A Novel Membership Inference Attack against Dynamic Neural Networks by Utilizing Policy Networks Information
Pan Li, Peizhuo Lv, Shenchen Zhu, Ruigang Liang, Kai Chen
Membership Inference Attack Dynamic Neural Network Policy Network Static Deep Neural Network

March 7, 2022

HyperMixer: An MLP-based Low Cost Alternative to Transformers
Florian Mai, Arnaud Pannatier, Fabio Fehr, Haolin Chen, Francois Marelli, Francois Fleuret, James Henderson
Transformer Megatron Decepticons Language Understanding MLP Like Model Static Deep Neural Network Simple MLP

Static Deep Neural Network

Papers

FETCH: A Memory-Efficient Replay Approach for Continual Learning in Image Classification

Valid Conformal Prediction for Dynamic GNNs

SoD$^2$: Statically Optimizing Dynamic Deep Neural Network

MSPipe: Efficient Temporal GNN Training via Staleness-aware Pipeline

Fluid Dynamic DNNs for Reliable and Adaptive Distributed Inference on Edge Devices

Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks

Input-Aware Dynamic Timestep Spiking Neural Networks for Efficient In-Memory Computing

A Novel Membership Inference Attack against Dynamic Neural Networks by Utilizing Policy Networks Information

HyperMixer: An MLP-based Low Cost Alternative to Transformers