Neural Network Growth

Follow this topic Ask about this topic

Latest papers

10 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

CAGE-NAS: Certified Functional Descent for Efficient Model Growth
Oct 1, 2026Santiago Florido Gomez, Stéphane RivaudGradient Descent Neural Network Optimization
Institut Polytechnique de Paris Paris, France · Inria Saclay, TAU Team Paris, France
Neuron-Level Architecture Growth: A Controlled Evaluation for EEG Time-Series Decoding
Sep 27, 2026Adam Mounir, Stella Douka, Arnault H. Caillet +2Neural Decoding EEG Decoding
Inria TAU – LISN, Université Paris-Saclay, France · Yneuro, Paris, France · Imperial College London, UK +1
Across the Loss Landscape with Progressive Growth
Aug 25, 2026Paul Caillon, Christophe Cerisara, Alexandre AllauzenDeep Learning Optimization Loss Landscape
MILES Team, LAMSADE, Université Paris Dauphine-PSL · LORIA CNRS, Campus Scientifique, rue du Jardin Botanique, 54500 Vandoeuvre-les-Nancy · ESPCI-PSL
Plasticity of Growing and Elastic Neural Networks in Online Continual Learning
Aug 2, 2026Jeong Min Kong, Richard S. SuttonLoss of Plasticity Dynamic Neural Networks
Department of Electrical and Computer Engineering University of California, Los Angeles (UCLA) · Department of Computing Science University of Alberta Alberta Machine Intelligence Institute (Amii)
Exact Network Surgery: Functional Invariance and Gradient Plasticity in Reactive Computational Graphs
Jul 18, 2026Abdallah KhemaisNeural Network Growth Neural Network Training Dynamics
ISITCOM, University of Sousse
Growing a Neural Network in Breadth, Depth, and Time
May 24, 2026Eivinas Butkus, Kedar Garzón Gupta, Nikolaus KriegeskorteRecurrent Neural Networks Dynamic Neural Networks
1Columbia University · 2NSF AI Institute for Artificial and Natural Intelligence
Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models
May 22, 2026Maxim Henry, Adrien Deliège, Sébastien Piérard +1Deep Learning Optimization Efficient ViTs
Montefiore Institute, University of Liège, Liège, Belgium
On the Stability of Growth in Structural Plasticity
May 14, 2026Lute Lillo, Nick CheneyDynamic Neural Networks Neural Network Growth
University of Vermont
Enhancing Efficiency and Performance in Deepfake Audio Detection through Neuron-level Dropin & Neuroplasticity Mechanisms
Mar 25, 2026Yupei Li, Shuaijie Shao, Manuel Milling +1Neural Network Growth Audio Deepfake Detection
Department of Computer Science, University College London, London, United Kingdom · Chair of Health Informatics, Technical University of Munich, Munich, Germany · Chair of Health Informatics and Department of Computing, Technical University of Munich and Imperial College London, Munich and London, Germany and United Kingdom
Growing Networks with Autonomous Pruning
Mar 20, 2026Charles De Lambilly, Stefan DuffnerDynamic Sparse Training Neural Network Growth
INSA Lyon, CNRS, Ecole Centrale de Lyon, Universite Claude Bernard Lyon 1, Université Lumière Lyon 2, LIRIS, UMR5205, , Villeurbanne, 69621, France

Oct 1, 2026J/K move · Enter open · S save

CAGE-NAS: Certified Functional Descent for Efficient Model Growth

Santiago Florido Gomez, Stéphane Rivaud

Institut Polytechnique de Paris Paris, France · Inria Saclay, TAU Team Paris, France

Gradient Descent Neural Network Optimization Neural Network Growth Neural Architecture Search

Open paper AskPDF arXiv

The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be expanded. CAGE-NAS formulates this decision in function space through an admissibility criterion on approximations of the functional gradient. As long as a representation enables a certified Functional Gradient Descent step, the architecture remains fixed; when the criterion fails, a function-preserving expansion is applied and the resulting representation is evaluated again. As the main instance, we study the family induced by the tangent space, using a regularized projection of the functional gradient. In a controlled setting with exact certification, CAGE-NAS produces architectures positioned above the 99.8th performance percentile by held-out RMSE among all admissible alternatives within the same parameter budget, without enumerating them during the growth trajectory.