Modular Neural Networks

Follow this topic Ask about this topic

Modular neural networks study how to divide computation among specialized components and coordinate them to solve a task. Common approaches include gated mixtures of experts, branch-based networks, task-specific parameter masks, and compositional neural operators for physical systems. These designs aim to improve reuse, interpretability, efficiency, and continual learning by limiting interference between tasks or domains. A central tension is that specialization can help when module boundaries match the task, but effective routing and communication are difficult, and too much separation can limit useful sharing.

Latest papers

74 papers9 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

Sparsity-driven emergence of functional modularity in multitask neural networks
Oct 23, 2026Bowen Shi, Qi Ouyang, Long QianModular Neural Networks Multi-Task Learning
Institute for Advanced Study in Physics, Zhejiang University, Hangzhou 310058, China · Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China · ZJU–Hangzhou Global Scientific and Technological Innovation Center, Hangzhou 311200, China +1
Length-Independent Stability and $L_2$ -Gain Certificates for Chains of Delayed Neural Modules
Oct 18, 2026Ömer Faruk İşcanlıNeural Network Stability Modular Neural Networks
Department of Computer Engineering, Istanbul University-Cerrahpaşa, Istanbul, Türkiye
Hybrid Parallel Bessel KAN Model: a Kolmogorov Arnold Network: integrated framework for developing a novel architecture to reconstruct the fused-basis approach via a parameter free quadratic interaction
Oct 13, 2026Mintu Pathak, Meshv Patel, Mohendra Roy +1Kolmogorov-Arnold Networks Modular Neural Networks
Department of Applied Sciences, Tezpur University, Napaam, Tezpur, 784028, Assam, India. · Department of ICT, Pandit Deendayal Energy University, Raysan, Gandhinagar, 382007, Gujarat, India.
DeepThink: A Conceptual Model for Knowledge-Addressable Recurrent Language Modeling
Oct 12, 2026Weiguo ChenRecurrent Language Models Language Modeling
Analyzing the Dynamics of WSBM-based Echo State Networks using InputDSA
Oct 11, 2026Huy Mai, Robert ClarkeReservoir Computing Dynamic Neural Networks
Vietnam National University Ho Chi Minh City, University of Science · University of Bath
Neural networks are more modular than single neurons suggest
Oct 9, 2026Shreya Saha, Benjamin Bergen, Meenakshi KhoslaModular Neural Networks Neural Network Interpretability
Electrical and Computer Engineering, University of California, San Diego · Cognitive Science Department, University of California, San Diego · Computer Science and Engineering, University of California, San Diego
Scaling of Wireless Foundation Models via Representation Diversity and Multi-Branch Architectures
Oct 3, 2026Ahmed Mohamed, Ahmed Aboulfotouh, Hatem Abou-ZeidWireless Foundation Models Representation Learning
Department of Electrical and Software Engineering, University of Calgary, Canada
Dirac-Interconnected Neural Elements: Discovering Modularity in Physical Systems Without Reduction
Oct 2, 2026Reiho Li, Razmik Arman Khosrovian, Takaharu Yaguchi +2Dynamical Systems System Identification
Hokkaido University · Kyushu University · RIKEN +1
CORTEX: Learning to Share and Specialize in Dense Language Models
Sep 28, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongDomain Adaptation for LLMs Language Modeling
Simon Fraser University · Southern University of Science and Technology · The University of British Columbia
Brain-Inspired Hierarchical Modularity for General Continual Learning
Sep 21, 2026Hongwei Yan, Kanglei Zhou, Qi Cheng +7Continual Learning for LLMs Continual Learning for VLA Models
School of Life Sciences, Tsinghua University, Beijing, China. · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China. · Tsinghua-Peking Center for Life Sciences, Beijing, China. +2
The Ups and Downs of Backprop Weights
Sep 18, 2026Giuseppe Chindemi, Benjamin F. GreweModular Neural Networks Dynamic Neural Networks
Institute of Neuroinformatics, UZH / ETH Zurich ETH AI Center Zurich, Switzerland
Modular Deep Learning Mechanisms for Auditable Next-Day Wildfire Spread Prediction
Sep 15, 2026Miguel Esparza, Aydin Ayanzadeh, Ahmad Mousavi +1Wildfire Forecasting Spatiotemporal Forecasting
Urban Resilience.AI Lab, Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843, USA
What You Can't See Is Still What You Learn: A Preregistered Sixty-Society Confirmation That Evidence Masking Drives Compositional Generalization
Sep 15, 2026Narcis MarincatCompositional Generalization Modular Neural Networks
Independent researcher.
Dual-Part Multi-Lateral Branched Network for Multi-Class Segmentation in Cardiovascular Catheterization Angiograms
Sep 4, 2026Olatunji Omisore, Ahmed Elazab, Ali Shahidinejad +1Medical Image Segmentation Modular Neural Networks
Center for Machine Learning Networking and Education Technology, CQUniversity, Rockhampton, 4701, Australia. · Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
Modular Deep Recurrent Neural Network: Application to Quadrotors
Sep 3, 2026Nima Mohajerin, Steven L. WaslanderRecurrent Neural Networks Modular Neural Networks
Dept. of Mechanical and Mechatronics Engineering University of Waterloo, Waterloo, ON,Canada
Pre-carved Niches: The Formation Dynamics of Modular Task Partitions in Early LLM Training
Sep 1, 2026Guangqi Li, Yongxin LiLanguage Model Pretraining Modular Neural Networks
Zaozhuang University, Zaozhuang, Shandong, China
Sparse Competition during Training For the Emergence of Specialized Modules
Aug 31, 2026Baptiste Rossigneux, Karim HarounSparse Neural Networks Neural Network Training Dynamics
Univ. Rennes · Inria, IRISA · University of Paris 8
LaMoC: Loss-Aware Modular Compression for LLMs
Aug 31, 2026Mohanad Odema, Jacob SongModel Compression LLM Compression
LG Electronics USA
A Unified Backbone--Expert Framework with Relation-Token and Residual--Classifier Interfaces for Automatic Modulation Recognition
Aug 15, 2026Zhixiang Deng, Houbiao Li, Zongyong CuiAutomatic Modulation Classification Modular Neural Networks
School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, 611731, China · School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China
Communication in modular robotic motor control: Bilateral controllers under realistic constraints
Aug 14, 2026Jingwen Li, Levin Kuhlmann, Jason Friedman +1Robotic Control Robotics
Department of Data Science and AI, Monash University · Department of Physical Therapy & Sagol School of Neuroscience, Tel Aviv University · Cerenaut
The Neural Division of Labor: Biologically-Inspired Modular Architectures for Robust Neuromorphic Computing
Aug 8, 2026Maksim Bazhenov, Serafim Grubas, Vakhtang PutkaradzeEfficient Neural Network Inference Neural Network Interpretability
Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA · Department of Physics, University of Alberta, Edmonton, AB T6G 2G1, Canada · Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB T6G 2G1, Canada +1
SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization
Aug 4, 2026Boyao Wang, Zhihan LeiParameter-Free Mixture-of-Experts Routing Adaptive Model Routing
Machine Learning Department Carnegie Mellon University
Neural Network-Driven Volatility Drag Mitigation under Aggressive Leverage
Jul 25, 2026Christian Bongiorno, Efstratios Manolakis, Rosario Nunzio MantegnaPortfolio Optimization Neural Network Compression
Université Paris-Saclay, CentraleSupélec, Mathématiques et Informatique pour la Complexité et les Systèmes, 91190, Gif-sur-Yvette, France · Dipartimento di Fisica e Astronomia “Ettore Majorana” Catania, Italy · Dipartimento di Fisica e Chimica Palermo, Italy +1
The Giant Hippocampus: From Structural Monoculture to a System of Systems
Jul 22, 2026Jaeho SeolTransformer Inductive Bias
Independent Researcher
Scalable Keyword Spotting via Modular Network Expansion
Jul 22, 2026Viktor Khaymonenko, Dzmitry Saladukha, Aliaksei Rak +1Class-Incremental Learning Efficient Neural Network Inference
Embedded Voice Input Team, Yandex, Russia · Embedded Voice Input Team, Yandex, Belarus
MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models
Jul 14, 2026Mingzhen Xu, Haonan Guo, Di Wang +9Hyperspectral Image Classification Parameter-Efficient Fine-Tuning
School of Computer Science, Wuhan University, Wuhan, China. · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China. · China Petroleum Pipeline Engineering Corporation, Langfang, China, and also with Hebei Key Laboratory of Underground Energy Storage Technology, Langfang, China. +4
Modular Pretraining Enables Access Control
Jul 9, 2026Ethan Roland, Murat Cubuktepe, Erick Martinez +8Modular Neural Networks
1AE Studio · 2Independent · 3Anthropic
Modular Foundation Models for Time-Series Perception in Digital Twins
Jul 3, 2026Quang Hung Pham, Ryad Zemouri, Martin Gagnon +1Self-Supervised Time Series Representation Learning Time Series Forecasting
Hydro-Québec · 75, boulevard René-Lévesque Ouest · Montréal, Québec, Canada
MSNN-LINet: Cross-Modal Learning via Continuous Linear Integration
Jun 30, 2026Gabriel ClingerCross-Modal Learning Multimodal Fusion
The George Washington University
Distilling a Modular Reservoir Through a Genomic Bottleneck
Jun 20, 2026Mani Hamidi, Sina Khajehabdollahi, Charley M. Wu +1Reservoir Computing Sparse Neural Networks
Department of Computer Science, University of Tübingen, Tübingen, Germany · Flowers Team Inria, Bordeaux, France · Max Planck Institute for Biological Cybernetics, Tübingen, Germany +3

Oct 23, 2026J/K move · Enter open · S save

Sparsity-driven emergence of functional modularity in multitask neural networks

Bowen Shi, Qi Ouyang, Long Qian

Institute for Advanced Study in Physics, Zhejiang University, Hangzhou 310058, China · Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China · ZJU–Hangzhou Global Scientific and Technological Innovation Center, Hangzhou 311200, China+1

Modular Neural Networks Multi-Task Learning

Open paper AskPDF arXiv

Neural networks can learn multiple tasks, but how learning organizes shared computation into specialized and reusable components remains poorly understood. In this work, we show that sparsity inducing regularization promotes functional modularity in multitask networks without predefined module boundaries. Using functional profiling in multilayer perceptrons and a lightweight transformer, we identified compact, task-selective subnetworks, termed functional tokens. Across rule-based task families, selectively retaining these tokens preserves their associated functions while reducing performance on other tasks. Such specialization develops preferentially in deeper layers during training, while earlier layers retain shared computation. Regularization strength and form shape module compactness and specialization, while task weighting alters functional allocation. Moreover, extracted functional tokens can be recombined through alternative pathways to reconstruct a learned composite computation without retraining, preserving their connectivity and learned weights. Our results suggest that sparsity-constrained optimization shapes not only parameter use but also the organization and reuse of learned computation.