Dynamic Neural Networks

Follow this topic Ask about this topic

Dynamic neural networks study how models can change what they compute or how they are structured in response to inputs, tasks, and available resources. Common approaches include early exits and conditional gating, elastic depth or width, expert routing, structural growth and pruning, and recurrent, spiking, or reservoir dynamics. These methods aim to reduce inference cost, handle changing environments, preserve learning across tasks, and model temporal processes, with evaluation spanning vision, forecasting, continual learning, and edge deployment. A central challenge is making adaptation reliable and efficient: routing and restructuring add overhead, and gains in accuracy, stability, and resource use depend strongly on the task and hardware.

Latest papers

85 papers13 papers in the last four weeks, up 86% on the four weeks before. 0.1% of all new papers.

Congestion-Aware Dynamic Axonal Delay for Spiking Neural Networks
May 2, 2026Dewei Bai, Hongxiang Peng, Yunyun Zeng +2Dynamic Neural Networks Spiking Neural Networks
University of Electronic Science and Technology of China
LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference
May 1, 2026Shashank Kapadia, Deep Naryan Mishra, Sujal Reddy Alugubelli +4Efficient Transformer Inference Dynamic Neural Networks
Walmart Inc. · Sam’s Club
NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning
Apr 29, 2026Karthik Charan Raghunathan, Christian Metzner, Laura Kriener +1Stability-Plasticity Dilemma Dynamic Neural Networks
Institute of Neuroinformatics University of Zurich & ETH Zurich Zurich, Switzerland
SWAN: World-Aware Adaptive Multimodal Networks for Runtime Variations
Apr 28, 2026Jason Wu, Shir-Kang Scott Jin, Yuyang Yuan +4Efficient Multimodal Inference Dynamic Neural Networks
University of California, Los Angeles · University of California, Riverside · U.S. Army DEVCOM Army Research Laboratory
CoRE: Concept-Reasoning Expansion for Continual Brain Lesion Segmentation
Apr 28, 2026Qianqian Chen, Anglin Liu, Jingyang Zhang +1Dynamic Neural Networks Medical Image Segmentation
Southeast University, Nanjing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
A Comparative Analysis on the Performance of Upper Confidence Bound Algorithms in Adaptive Deep Neural Networks
Apr 27, 2026Grigorios Papanikolaou, Ioannis Kontopoulos, Konstantinos TserpesMulti-Armed Bandits Dynamic Neural Networks
Department of Electrical and Computer Engineering National Technical University of Athens, Greece
Scalable Memristive-Friendly Reservoir Computing for Time Series Classification
Apr 21, 2026Coşku Can Horuz, Andrea Ceni, Claudio Gallicchio +1Dynamic Neural Networks Energy-Efficient ML
Adaptive AI Research Group University of Lübeck, Germany · Department of Computer Science University of Pisa, Italy
When Do Early-Exit Networks Generalize? A PAC-Bayesian Theory of Adaptive Depth
Apr 17, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuNeural Network Generalization PAC-Bayesian Generalization Bounds
The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China
SAND: Spatially Adaptive Network Depth for Fast Sampling of Neural Implicit Surfaces
Apr 15, 2026Chuanxiang Yang, Junhui Hou, Yuan Liu +5Implicit Neural Representations Efficient Neural Network Inference
Shandong University, China · City University of Hong Kong, China · Hong Kong University of Science and Technology, China +2
Learning Inference Concurrency in DynamicGate MLP Structural and Mathematical Justification
Apr 15, 2026Yongil ChoiDynamic Neural Networks Online Learning
Sorynorydotcom Co., Ltd./AI Open Research Lab
DVM: A Bytecode Virtual Machine Approach for Dynamic Tensor Computation
Mar 25, 2026Jingzhi Fang, Xiong Gao, Renwei Zhang +6Efficient Inference Dynamic Neural Networks
Huawei, China · Hunan University
Circuit-Inspired High-Order Neural Networks with Unified Neural Dynamics Modeling for PDE Solving and Visual Perception
Mar 25, 2026Tongfei Chen, Jingying Yang, Linlin Yang +8Dynamic Neural Networks PDE Operator Learning
School of Artificial Intelligence (Institute of Artificial Intelligence), Beihang University, Beijing, China. · Communication University of China, Beijing, China. · School of Automation Science and Electrical Engineering, Beihang University, Beijing, China. +3
Centrality-Based Pruning for Efficient Echo State Networks
Mar 21, 2026Sudip LaudariReservoir Computing Neural Network Pruning
Independent Researcher.
DynamicGate MLP Conditional Computation via Learned Structural Dropout and Input Dependent Gating for Functional Plasticity
Mar 17, 2026Yong Il ChoiDynamic Neural Networks Conditional Computation
Sorynorydotcom Co., Ltd./AI Open Research Lab
MorphSNN: Adaptive Graph Diffusion and Structural Plasticity for Spiking Neural Networks
Mar 15, 2026Yongsheng Huang, Peibo Duan, Yujie Wu +7Spiking Neural Networks Dynamic Neural Networks
School of Software, Northeastern University, China · Guangdong Institute of Intelligence Science and Technolog, China · The Hong Kong Polytechnic University, China +1
ReDON: Recurrent Diffractive Optical Neural Processor with Reconfigurable Self-Modulated Nonlinearity
Feb 27, 2026Ziang Yin, Qi Jing, Raktim Sarma +3Diffractive Neural Networks Photonic Computing
Arizona State University, Tempe, AZ, USA · Center for Integrated Nanotechnologies, Sandia National Laboratories, Albuquerque, NM, USA · Rensselaer Polytechnic Institute, Troy, NY, USA
Isotropic Activation Functions Enable Deindividuated Neurons and Adaptive Topologies
Feb 26, 2026George BirdDynamic Neural Networks Neural Network Activation Functions
Department of Computer Science, University of Manchester, UK
Stream Neural Networks: Epoch-Free Learning with Persistent Temporal State
Feb 25, 2026Amama PathanDynamic Neural Networks Streaming Algorithms
BLACKLOOP Research (Independent Research)
Switchable Activation Networks
Feb 17, 2026Laha Ale, Ning Zhang, Scott A. King +1Efficient Neural Network Inference Dynamic Neural Networks
Southwest Jiaotong University · University of Windsor · Texas A&M University-Corpus Christi
MIDAS: Mosaic Input-Specific Differentiable Architecture Search
Feb 6, 2026Konstanty SubbotkoNeural Architecture Search Dynamic Neural Networks
Independent Researcher
Hardware-Algorithm Co-Optimization of Early-Exit Neural Networks for Multi-Core Edge Accelerators
Dec 4, 2025Alaa Zniber, Arne Symons, Ouassim Karrakchou +2Efficient Neural Network Inference Dynamic Neural Networks
TICLab, International University of Rabat, Morocco · MICAS, KU Leuven, Belgium
Deep Neural Networks Inspired by Differential Equations
Oct 9, 2025Yongshuai Liu, Lianfang Wang, Kuilin Qin +6Neural Network Generalization Dynamic Neural Networks
School of Mathematical Sciences, Beijing Normal University, Beijing, China · Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong
PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution
Jul 2, 2025Omkar Shende, Gayathri Ananthanarayanan, Marcello TraiolaEfficient Neural Network Inference Adaptive Inference
Department of Computer Science and Engineering, Indian Institute of Technology Dharwad, Karnataka, India · University of Rennes, CNRS, Inria, IRISA, Rennes, France
SEAL: Searching Expandable Architectures for Incremental Learning
May 15, 2025Matteo Gambella, Manuel RoveriDynamic Neural Networks Continual Learning
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
PEARL: Structural Privacy-Utility Control in Human-Centric CPS via Personalized Early-Exit Deep Reinforcement Learning
Mar 9, 2024Mojtaba Taherisadr, Salma ElmalakiRL Control Cyber-Physical Systems
University of California, Irvine, USA

May 2, 2026J/K move · Enter open · S save

Congestion-Aware Dynamic Axonal Delay for Spiking Neural Networks

Dewei Bai, Hongxiang Peng, Yunyun Zeng, Ziyu Zhang+1

University of Electronic Science and Technology of China

Dynamic Neural Networks Spiking Neural Networks

Open paper AskPDF arXiv

Spiking Neural Networks (SNNs) are widely regarded as an energy-efficient paradigm for modeling and processing temporal and event-driven information. Incorporating delays in SNNs has been proven to be an effective mechanism for improving spike alignment in event-driven tasks. However, existing delay learning approaches predominantly assign static delays to individual synapses, resulting in a large number of delay parameters and limited adaptability to input-dependent activity dynamics. To this end, we propose a Congestion-Aware Dynamic Axonal Delay (CADAD) mechanism, which decomposes the delay into a channel-wise static base delay for temporal structuring and a global, activity-conditioned shift that dynamically regulates the state update rate under varying spike intensities. The delay parameters are learned using differentiable linear interpolation and discretized at inference time, preserving the benefits of dynamic delay modulation while incurring only minimal additional cost. Experiments on speech benchmarks, including the Spiking Heidelberg Dataset, Spiking Speech Commands, and Google Speech Commands, demonstrate that introducing congestion-aware delays into synaptic signal transmission effectively improves accuracy on temporal tasks, notably achieving 93.75% accuracy on SHD, 80.69% accuracy on SSC, and 95.58% on GSC-35, while reducing the parameter count by approximately 50% compared to state-of-the-art delay-based methods with the same architecture.