Modular Neural Networks

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 48

All topics

CORTEX: Learning to Share and Specialize in Dense Language Models
Sep 28, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongDomain Adaptation for LLMs Language Modeling
Simon Fraser University · Southern University of Science and Technology · The University of British Columbia
Brain-Inspired Hierarchical Modularity for General Continual Learning
Sep 21, 2026Hongwei Yan, Kanglei Zhou, Qi Cheng +7Continual Learning for LLMs Continual Learning for VLA Models
School of Life Sciences, Tsinghua University, Beijing, China. · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China. · Tsinghua-Peking Center for Life Sciences, Beijing, China. +2
Modular Deep Learning Mechanisms for Auditable Next-Day Wildfire Spread Prediction
Sep 15, 2026Miguel Esparza, Aydin Ayanzadeh Ahmad Mousavi, Ali MostafaviWildfire Forecasting Spatiotemporal Forecasting
Urban Resilience.AI Lab, Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843, USA · Department of Information Systems, University of Maryland, Baltimore County, Baltimore, MD 21250, USA · Department of Mathematics and Statistics, American University, Washington, D.C. 20016, USA
What You Can't See Is Still What You Learn: A Preregistered Sixty-Society Confirmation That Evidence Masking Drives Compositional Generalization
Sep 15, 2026Narcis MarincatCompositional Generalization Modular Neural Networks
Independent researcher.
Pre-carved Niches: The Formation Dynamics of Modular Task Partitions in Early LLM Training
Sep 1, 2026Guangqi Li, Yongxin LiLanguage Model Pretraining Modular Neural Networks
Zaozhuang University, Zaozhuang, Shandong, China
Sparse Competition during Training For the Emergence of Specialized Modules
Aug 31, 2026Baptiste Rossigneux, Karim HarounSparse Neural Networks Neural Network Training Dynamics
Univ. Rennes · Inria, IRISA · University of Paris 8
LaMoC: Loss-Aware Modular Compression for LLMs
Aug 31, 2026Mohanad Odema, Jacob SongModel Compression LLM Compression
LG Electronics USA
The Neural Division of Labor: Biologically-Inspired Modular Architectures for Robust Neuromorphic Computing
Aug 8, 2026Maksim Bazhenov, Serafim Grubas, Vakhtang PutkaradzeEfficient Neural Network Inference Neural Network Interpretability
Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA · Department of Physics, University of Alberta, Edmonton, AB T6G 2G1, Canada · Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB T6G 2G1, Canada +1
SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization
Aug 4, 2026Boyao Wang, Zhihan LeiParameter-Free Mixture-of-Experts Routing Adaptive Model Routing
Machine Learning Department Carnegie Mellon University
Neural Network-Driven Volatility Drag Mitigation under Aggressive Leverage
Jul 25, 2026Christian Bongiorno, Efstratios Manolakis, Rosario Nunzio MantegnaPortfolio Optimization Neural Network Compression
Université Paris-Saclay, CentraleSupélec, Mathématiques et Informatique pour la Complexité et les Systèmes, 91190, Gif-sur-Yvette, France · Dipartimento di Fisica e Astronomia “Ettore Majorana” Catania, Italy · Dipartimento di Fisica e Chimica Palermo, Italy +1
The Giant Hippocampus: From Structural Monoculture to a System of Systems
Jul 22, 2026Jaeho SeolTransformer Inductive Bias
Independent Researcher
Scalable Keyword Spotting via Modular Network Expansion
Jul 22, 2026Viktor Khaymonenko, Dzmitry Saladukha, Aliaksei Rak +1Class-Incremental Learning Efficient Neural Network Inference
Embedded Voice Input Team, Yandex, Russia · Embedded Voice Input Team, Yandex, Belarus
MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models
Jul 14, 2026Mingzhen Xu, Haonan Guo, Di Wang +9Hyperspectral Image Classification Parameter-Efficient Fine-Tuning
School of Computer Science, Wuhan University, Wuhan, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China · China Petroleum Pipeline Engineering Corporation, Langfang, China +6
Modular Pretraining Enables Access Control
Jul 9, 2026Ethan Roland, Murat Cubuktepe, Erick Martinez +8Modular Neural Networks
1AE Studio · 2Independent · 3Anthropic
Modular Foundation Models for Time-Series Perception in Digital Twins
Jul 3, 2026Quang Hung Pham, Ryad Zemouri, Martin Gagnon +1Self-Supervised Time Series Representation Learning Time Series Forecasting
Hydro-Québec · 75, boulevard René-Lévesque Ouest · Montréal, Québec, Canada
MSNN-LINet: Cross-Modal Learning via Continuous Linear Integration
Jun 30, 2026Gabriel ClingerCross-Modal Learning Multimodal Fusion
The George Washington University
Distilling a Modular Reservoir Through a Genomic Bottleneck
Jun 20, 2026Mani Hamidi, Sina Khajehabdollahi, Charley M. Wu +1Reservoir Computing Sparse Neural Networks
Department of Computer Science, University of Tübingen, Tübingen, Germany · Flowers Team Inria, Bordeaux, France · Max Planck Institute for Biological Cybernetics, Tübingen, Germany +3
SSH-Net: A Deep Neural Network for Predicting Failure Time Distribution Functions under Competing Risks with Application to GPU Data
Jun 18, 2026Jie Min, Yueyao Wang, Mengkun ChenSurvival Analysis Modular Neural Networks
Department of Mathematics & Statistics, University of South Florida, Tampa, FL · School of Statistics and Data Science, Zhejiang Gongshang University, Hangzhou, China · Department of Statistics,AbstractVirginia Tech, Blacksburg, VA
Modularity-Free Conflict-Averse Training for Generalized PINNs
Jun 18, 2026Heejo Kong, Beomchul Park, Sung-Jin Kim +1Neural PDE Solvers Gradient Interference
Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea · Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea
GrapNet: A Programmable Dynamic-Architecture Neural Graph Substrate
Jun 17, 2026Zirong LiDynamic Neural Networks Modular Neural Networks
Tiangong University
KANLib -- A Modular, Extensible and Fast Kolmogorov-Arnold Network Implementation
Jun 16, 2026Julian Hoever, Gregor SchieleKolmogorov-Arnold Networks Modular Neural Networks
Intelligent Embedded Systems University of Duisburg-Essen Duisburg, Germany
Dimensionality Controls When Modularity Helps in Continual Learning
Jun 16, 2026Kathrin Korte, Christian Medeiros Adriano, Joachim Winther Pedersen +2Continual Learning Modular Neural Networks
IT University of Copenhagen, Denmark · Hasso Plattner Institute, University of Potsdam, Germany
Energy-Conserved Neural Pipelines: Attenuating Error Propagation in Modular Neural Networks via Physical Conservation Constraints
Jun 9, 2026David Young, Swan Yi HtetNeural Network Robustness Noise Robustness
ORION Robotics
BMCR: Adaptive Backbone Module Composition via Reinforcement Learning for Remote Sensing Object Detection
Jun 4, 2026Wenlin Liu, Xikun Hu, Ping ZhongReinforcement Learning Dynamic Neural Networks
College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China
Multi-Column RBF Neural Network Using Adaptive and Non-Adaptive Particle Swarm Optimization
Jun 3, 2026Ammar Hoori, Yuichi MotaiMetaheuristic Optimization Neural Network Optimization
Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA · Department of Electrical and Computer Engineering, Virginia Commonwealth University, Richmond, VA 23284, USA
PE-MHL: Physics-Encoded Modular Hybrid Layers for Scalable Learning of Complex Systems
Jun 2, 2026Ismail Hassaballa, Mircea LazarNonlinear System Identification Physics-Informed ML
Eindhoven University of Technology Eindhoven, Netherlands
TimeBlocks: Foundational and Continual Time-Series Blockbase -- Extended Version
Jun 1, 2026David Campos, Bin Yang, Tung Kieu +3Continual Test-Time Adaptation Multi-Task Learning
Aalborg University, Denmark · University of Amsterdam Netherlands · East China Normal University China +1
Learning Compositional Latent Structure with Vector Networks
May 27, 2026Niclas Pokel, Benjamin F. GreweRepresentation Learning Recurrent Neural Networks
Institute of Neuroinformatics, UZH / ETH Zurich ETH AI Center Zurich, Switzerland
Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models
May 21, 2026Yuting He, Chenyu You, Shuo LiMedical Imaging Foundation Models Multimodal Pretraining
Case Western Reserve University · Stony Brook University
Evolutionary Algorithm for Reservoir Learning and Yielding
May 19, 2026Julien Testu, Pierrick Legrand, Xavier HinautReservoir Computing Neural Architecture Search
UB, Mnemosyne · ENSC, Bordeaux INP · Mnemosyne

Sep 28, 2026, cs.AIJ/K move · Enter open · S save

CORTEX: Learning to Share and Specialize in Dense Language Models

Chuiyang Meng, Ming Tang, Vincent W. S. Wong

Simon Fraser University · Southern University of Science and Technology · The University of British Columbia

Domain Adaptation for LLMs Language Modeling Modular Neural Networks

Open paper AskPDF arXiv

Large language models are trained on heterogeneous data mixtures, where different knowledge domains require both shared knowledge and specialization. Existing modular approaches typically impose explicit components or discover modules through interpretability analysis after training. In this work, we propose CORTEX, a learning dynamics-inspired framework that learns internal modularization within dense language models. CORTEX partitions trainable matrices into parameter groups and learns module assignments from domain-conditioned gradient and cross-domain gradient similarity. We introduce the selective lesion score and module-domain mutual information to characterize the target-domain lesion effects and alignment, and analyze how module assignment affects the trade-off between assignment bias and update magnitude. Experiments with 160M, Qwen3-8B, and Qwen3-32B backbone models show that CORTEX achieves the highest synthetic-domain exact match and largest average perplexity reduction, while remaining competitive on real-domain evaluations and forming identifiable modules.

Momentum

Related topics

Latest papers 48