Shallow Neural Networks

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 27

All topics

Fluctuations of Nonlinear Observables in Mean Field Neural Network Training
Oct 7, 2026Arnaud Descours, Geoffrey LacourNeural Network Training Dynamics Uncertainty Quantification
UCBL · MaIAGE
Removing spurious minima for planar features by skip connections
Oct 1, 2026Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin +1Shallow Neural Networks ReLU Neural Networks
Freie Universität Berlin · Technische Universität Berlin
The Conflict Between Logic and Memory: Learning Higher-Order Interactions in Shallow MLPs
Sep 30, 2026Gongyue Zhang, Honghai LiuShallow Neural Networks Multilayer Perceptrons
Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks
Sep 17, 2026Xianjun Li, Yunfei YangShallow Neural Networks ReLU Neural Networks
School of Statistics and Data Science, Capital University of Economics and Business, Beijing, China. · School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China.
Learning Orthogonal Multi-Index Models Beyond Small Initialization: Incremental Learning, Competitive Dynamics and Symmetry
Sep 11, 2026Mo Zhou, Weihang Xu, Simon S. Du +1Shallow Neural Networks Orthogonal Representation Learning
University of Washington · Amazon, Inc.
Shallow neural network approximation in mixed Sobolev spaces
Sep 4, 2026Yuwen Li, Guozhi ZhangNeural Network Approximation Theory Shallow Neural Networks
School of Mathematical Sciences, Zhejiang University, 866 Yuhangtang Road, Hangzhou 310058, Zhejiang, China
Every Layer Counts: An Exponential $L_2$ Depth Hierarchy for ReLU Networks
Aug 24, 2026Itay SafranShallow Neural Networks ReLU Neural Networks
Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev
Approximation Rates for Metaplectic Neural Networks
Aug 9, 2026Ahmed Abdeljawad, Marcello Carioni, Elena CorderoShallow Neural Networks Neural Network Approximation Theory
Johann Radon Institute of Computational and Applied Mathematics (RICAM) Austrian Academy of Sciences Altenberger Straße 69, A-4040 Linz, Austria · Department of Applied Mathematics University of Twente 7500AE Enschede, The Netherlands · Dipartimento di Matematica Università degli Studi di Torino via Carlo Alberto 10, 10123 Torino, Italy
A Counterexample to Fourier Alignment in Single-Neuron Modular Addition
Aug 5, 2026Gautam Neelakantan MemanaShallow Neural Networks ReLU Neural Networks
Shallower ReLU Network Representations via Exact Linear Algebra
Jul 22, 2026Kilian Rueß, Gennadiy Averkov, Florestan Brunck +7Shallow Neural Networks ReLU Neural Networks
University of Technology Nuremberg · Brandenburg University of Technology Cottbus–Senftenberg · University of Copenhagen +3
Self-organizing Architecture of Receptron Units: a Hardware-Aware Framework for Edge Intelligence
Jul 22, 2026Stefano Radice, Ludovico Casaccia, Riccaro Emanuele Beccalli +2Shallow Neural Networks Neuromorphic Computing
Department of Physics “Aldo Pontremoli” and CIMaINa University of Milano Milano, Italy
Expressivity of Shallow Neural Networks Over Finite Fields
Jul 19, 2026Maksym Zubkov, Carol Wu, Shiwei Yang +2Shallow Neural Networks Polynomial Neural Networks
Department of Mathematics, University of British Columbia, 1984 Mathematics Rd, Vancouver, V6T 1Z2, BC, Canada
All you need is SAMPAT
Jul 10, 2026Jayadeva, Madhur AswaniShallow Neural Networks Interpretable ML
Department of Electrical Engineering, Indian Institute of Technology, Delhi
Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse
Jul 3, 2026Shuang Liang, Tom Jacobs, Guido MontúfarShallow Neural Networks ReLU Neural Networks
Department of Statistics & Data Science University of California, Los Angeles Los Angeles, CA 90095, USA · CISPA Helmholtz Center for Information Security · Departments of Mathematics and Statistics & Data Science University of California, Los Angeles; and Max Planck Institute for Mathematics in the Sciences
Born Discrete, Made Smooth: Variational Formulation of Shallow Neural Networks
Jul 2, 2026Matej Benko, Pierre Bousquet, Iwona Chlebicka +1Shallow Neural Networks Neural Network Generalization
Institute of Mathematics, Faculty of Mechanical Engineering, Brno University of Technology · Technická 2896/2, 616 69, Brno, Czech Republic · Université de Toulouse, INSA Toulouse, CNRS, IMT, F-31062 Toulouse Cedex 9, France +2
Neuro-Bayesian-Symbolic Residual Attention Shallow Network: Explainable Deep Learning for Cybersecurity Risk Assessment
Jun 29, 2026Nicolaie Popescu-Bodorin, Madeleine TogherShallow Neural Networks Explainable Artificial Intelligence
Engineering & Computer Science Department Spiru Haret University, Bucharest, Romania · Computer Information Science Department Higher Colleges Technologies, Sharjah, U.A.E.
Convex training of Lipschitz-regularized shallow neural networks
Jun 17, 2026Chao Yin, Antoine Lesage-LandryAdversarial Training Shallow Neural Networks
Polytechnique Montr´eal, GERAD & Mila, Montr´eal, QC, Canada, H3T 1J4.
Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
May 21, 2026Margalit Glasgow, Joan BrunaShallow Neural Networks Mean-Field Theory
Massachusetts Institute of Technology · Courant Institute School of Mathematics, Computing and Data Science, New York University
Machine-Learning-Enhanced Non-Invasive Testing for MASLD Fibrosis: Shallow-Deep Neural Networks Versus FIB-4, Tabular Foundation Models, and Large Language Models
May 19, 2026Athanasios Angelakis, Gabriele De Vito, Eleni-Myrto Trifylli +1Shallow Neural Networks Tabular ML
BioML Lab, RI CODE, UniBw, Munich, Germany · Department of Epidemiology and Data Science, Amsterdam UMC, Amsterdam, Netherlands · Alpha Indicium, Rijswijk, Netherlands +2
Shallow ReLU $^s$ Networks in $L^p$ -Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization
May 18, 2026Weizhao Li, Fanghui Liu, Lei ShiShallow Neural Networks Nonparametric Regression
School of Mathematical Sciences Fudan University Shanghai, China · School of Mathematical Sciences Institute of Natural Sciences and MOE-LSC2026 Shanghai Jiao Tong University Shanghai, China · School of Mathematical Sciences Shanghai Key Laboratory for Contemporary Applied Mathematics25 Fudan University Shanghai, China
Learning with Shallow Neural Networks on Cluster-Structured Features
May 14, 2026Elisabetta Cornacchia, Laurent MassouliéShallow Neural Networks Neural Network Generalization
INRIA, DI/ENS, PSL.
On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
May 11, 2026Romain Petit, Clarice Poon, Gabriel PeyréShallow Neural Networks Gradient Descent
CNRS and ENS, PSL Universit´e · Mathematics Institute, University of Warwick
Parameterized Complexity of Stationarity Testing for Piecewise-Affine Functions and Shallow CNN Losses
May 11, 2026Yuhan YeShallow Neural Networks Nonconvex Optimization
MIT
A Complete Symmetry Classification of Shallow ReLU Networks
Apr 15, 2026Pranavkrishnan RamakrishnanShallow Neural Networks Neural Representation Geometry
Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU
Oct 29, 2025Jingzhou LiuShallow Neural Networks Dynamical Systems
Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU $^k$ Networks on the Sphere
Oct 5, 2025Tong Mao, Jinchao XuShallow Neural Networks Neural Network Approximation Theory
Shenzhen Loop Area Institute, Shenzhen 518000, P. R. China · King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia.
AYLA: Architecting a loss landscape in shallow neural networks to accelerate feature recovery
Apr 2, 2025Behnam Gheshlaghi, Shahin AtakishiyevShallow Neural Networks Representation Learning

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Fluctuations of Nonlinear Observables in Mean Field Neural Network Training

Arnaud Descours, Geoffrey Lacour

UCBL · MaIAGE

Neural Network Training Dynamics Uncertainty Quantification Mean-Field Theory Shallow Neural Networks

Open paper AskPDF arXiv

Mean field limits describe the training dynamics of wide neural networks through the evolution of the empirical distribution of their parameters. Although functional central limit theorems characterize the asymptotic fluctuations of this distribution, quantities of practical interest are typically nonlinear observables of the parameter distribution rather than the distribution itself. In this work, we show how these mean field fluctuations propagate to finite dimensional nonlinear observables for shallow neural networks trained by stochastic gradient descent. Working in the weighted Sobolev space in which the limiting fluctuation process is constructed, we apply a functional Delta method under ordinary Fr{é}chet differentiability, without requiring Lions derivatives with respect to the measure variable. We obtain a central limit theorem for the observables and, under a suitable representation of their differentials, an explicit covariance formula inherited from the underlying mean field fluctuation theory. We also study whether prescribed quantities of interest can be recovered from the selected observations. Under a constant rank assumption, we prove that a quantity of interest factors locally through the observation functional if and only if, throughout a neighborhood, the kernel of the differential of the observation is contained in that of the quantity of interest. Thus, a differential condition expressed directly in the ambient Sobolev space yields an exact nonlinear local factorization. These results provide a framework both for quantifying finite-width uncertainty on observable, statistically or physically meaningful quantities and for assessing whether the chosen observations contain the information required to identify them.

Momentum

Related topics

Latest papers 27