Shallow Neural Networks

Ask about this topic

Overview

Shallow neural networks—typically models with one hidden layer—are studied for what functions they can represent, how they learn, and when their compact structure is useful. Approximation theory uses tools such as Fourier and spherical-harmonic analysis to give error and generalization guarantees, while optimization research examines gradient descent, loss landscapes, symmetry, and mean-field dynamics. Practical approaches include changing activation functions, regularizing parameter norms, and making individual units more expressive, with applications ranging from differential equations to compact classifiers. A central tension is that width and richer units can improve approximation, but depth may be essential for some tasks, and representational capacity alone does not ensure that training finds a good solution.

Latest papers

CardsList

57 papers11 papers in the last four weeks, up 120% on the four weeks before. 0.1% of all new papers.

  1. Convex training of Lipschitz-regularized shallow neural networks

    Jun 17, 2026Chao Yin, Antoine Lesage-LandryAdversarial TrainingShallow Neural Networks

    Polytechnique Montr´eal, GERAD & Mila, Montr´eal, QC, Canada, H3T 1J4.

  2. Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks

    May 21, 2026Margalit Glasgow, Joan BrunaShallow Neural NetworksMean-Field Theory

    Massachusetts Institute of Technology · Courant Institute School of Mathematics, Computing and Data Science, New York University

  3. Machine-Learning-Enhanced Non-Invasive Testing for MASLD Fibrosis: Shallow-Deep Neural Networks Versus FIB-4, Tabular Foundation Models, and Large Language Models

    May 19, 2026Athanasios Angelakis, Gabriele De Vito, Eleni-Myrto Trifylli +1Shallow Neural NetworksTabular ML

    BioML Lab, RI CODE, UniBw, Munich, Germany · Department of Epidemiology and Data Science, Amsterdam UMC, Amsterdam, Netherlands · Alpha Indicium, Rijswijk, Netherlands +2

  4. Shallow ReLUs^s Networks in LpL^p-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization

    May 18, 2026Weizhao Li, Fanghui Liu, Lei ShiShallow Neural NetworksNonparametric Regression

    School of Mathematical Sciences Fudan University Shanghai, China · School of Mathematical Sciences Institute of Natural Sciences and MOE-LSC2026 Shanghai Jiao Tong University Shanghai, China · School of Mathematical Sciences Shanghai Key Laboratory for Contemporary Applied Mathematics25 Fudan University Shanghai, China

  5. Learning with Shallow Neural Networks on Cluster-Structured Features

    May 14, 2026Elisabetta Cornacchia, Laurent MassouliéShallow Neural NetworksNeural Network Generalization

    INRIA, DI/ENS, PSL.

  6. On the global convergence of gradient descent for wide shallow models with bounded nonlinearities

    May 11, 2026Romain Petit, Clarice Poon, Gabriel PeyréShallow Neural NetworksGradient Descent

    CNRS and ENS, PSL Universit´e · Mathematics Institute, University of Warwick

  7. Parameterized Complexity of Stationarity Testing for Piecewise-Affine Functions and Shallow CNN Losses

    May 11, 2026Yuhan YeShallow Neural NetworksNonconvex Optimization

    MIT

  8. A Complete Symmetry Classification of Shallow ReLU Networks

    Apr 15, 2026Pranavkrishnan RamakrishnanShallow Neural NetworksNeural Representation Geometry

  9. Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks

    Apr 10, 2026Jie Huang, Bruno Loureiro, Stefano Sarao MannelliOverparameterizationLoss Landscape

    Physics, Chalmers University of Technology and University of Gothenburg · Ecole Normale Superieure, PSL & CNRS · Data Science and AI, Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg +1

  10. Tensor-Augmented Convolutional Neural Networks: Enhancing Expressivity with Generic Tensor Kernels

    Apr 9, 2026Chia-Wei Hsing, Wei-Lin TuConvolutional Neural NetworksNeural Network Expressivity

    blueqat Inc., 2-24-12-39F, Shibuya, Shibuya-ku, Tokyo 150-6139, Japan · Center for Quantum Science and Engineering, National Taiwan University, Taipei 10617, Taiwan · Graduate School of Science and Technology, Keio University, Yokohama, Kanagawa 223-8522, Japan +1

  11. Mathematical analysis of one-layer neural network with fixed biases, a new activation function and other observations

    Apr 9, 2026Fabricio Macià, Shu NakamuraShallow Neural NetworksNeural Network Training Dynamics

    M2ASAI, Universidad Politécnica de Madrid. ETSI Navales. Avda. de la Memoria, 4. 28040 Madrid, Spain. · Department of Mathematics, Faculty of Sciences, Gakushuin University, 1-5-1, Mejiro, Toshima, Tokyo, 171-8588 Japan

  12. Towards Initialization-dependent and Non-vacuous Generalization Bounds for Overparameterized Shallow Neural Networks

    Apr 1, 2026Yunwen Lei, Yufeng XieGeneralization BoundsOverparameterized Neural Network

    Department of Mathematics, The University of Hong Kong

  13. Expressibility of neural quantum states: a Walsh-complexity perspective

    Mar 27, 2026Taige WangNeural Quantum StatesNeural Network Expressivity

    Department of Physics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Physics, Harvard University, Cambridge, MA 02138, USA

  14. Quotient Geometry, Effective Curvature, and Implicit Bias in Simple Shallow Neural Networks

    Mar 23, 2026Hang-Cheng Dong, Pengcheng ChengShallow Neural NetworksNeural Network Training Dynamics

    School of Instrumentation Science and Engineering, Harbin Institute of Technology Harbin, 150001, China · Harbin Institute of Technology Suzhou Research Institute Suzhou, 215000, China · School of Mathematics, Jilin University Changchun, 130012, China

  15. Gradient Descent with Projection Finds Over-Parameterized Neural Networks for Learning Low-Degree Polynomials with Nearly Minimax Optimal Rate

    Mar 22, 2026Yingzhen Yang, Ping LiLow-Degree Polynomial MethodNeural Network Generalization

    School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ 85281, USA · VecML Inc., Bellevue, WA 98004, USA

  16. A Dual Certificate Approach to Sparsity in Infinite-Width Shallow Neural Networks

    Mar 18, 2026Leonardo Del Grande, Christoph Brune, Marcello CarioniTV RegularizationShallow Neural Networks

    Department of Applied Mathematics, University of Twente, 7500AE Enschede, The Netherlands

  17. Scaling Laws and Pathologies of Single-Layer PINNs: Network Width and PDE Nonlinearity

    Mar 13, 2026Faris ChaudhryPhysics-Informed MLNeural Scaling Laws

    Department of Computer Science, Imperial College London

  18. How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?

    Mar 5, 2026Kuo-Wei Lai, Guanghui Wang, Molei Tao +1High-Dimensional Linear RegressionRandom Feature Methods

    School of Electrical and Computer Engineering, Georgia Institute of Technology · College of Computing, Georgia Institute of Technology · School of Mathematics, Georgia Institute of Technology +1

  19. From Shallow Bayesian Neural Networks to Gaussian Processes: General Convergence, Identifiability and Scalable Inference

    Feb 26, 2026Gracielle Antunes de Araújo, Flávio B. GonçalvesNyström MethodBayesian Neural Networks

    Universidade do Estado de Minas Gerais (UEMG), FaEnge – Faculdade de Engenharia. · Universidade Federal de Minas Gerais (UFMG), ICEx – Instituto de Ciências Exatas.

  20. Smoothness Adaptivity in Constant-Depth Neural Networks: Optimal Rates via Smooth Activations

    Feb 23, 2026Yuhao Liu, Zilin Wang, Lei Wu +1Neural Network Approximation TheoryNeural Network Activation Functions

    Department of Mathematical Sciences, Tsinghua University · School of Mathematical Sciences, Peking University · Center for Machine Learning Research, Peking University +1

  21. From Approximation Rates to Loss-Landscape Barrier Decay in Shallow ReLU Networks

    Feb 19, 2026Saveliy BaturinShallow Neural NetworksReLU Neural Networks

    Independent Researcher, Russia.

  22. SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximation

    Nov 26, 2025Zachary Morrow, Michael Penwarden, Brian Chen +3Polynomial Neural NetworksNeural Network Approximation Theory

  23. Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLU

    Oct 29, 2025Jingzhou LiuShallow Neural NetworksDynamical Systems

  24. Configuration-Dependent Lower Bounds for Approximation by Shallow ReLUk^k Networks on the Sphere

    Oct 5, 2025Tong Mao, Jinchao XuShallow Neural NetworksNeural Network Approximation Theory

    Shenzhen Loop Area Institute, Shenzhen 518000, P. R. China · King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia.

  25. On shallow feedforward neural networks with inputs from a topological space

    Apr 3, 2025Vugar IsmailovFeedforward Neural NetworksNeural Network Approximation Theory

    Institute of Mathematics and Mechanics, Baku, Azerbaijan · Center for Mathematics and its Applications, Khazar University, Baku, Azerbaijan

  26. AYLA: Architecting a loss landscape in shallow neural networks to accelerate feature recovery

    Apr 2, 2025Behnam Gheshlaghi, Shahin AtakishiyevShallow Neural NetworksRepresentation Learning

  27. Overcoming a Learning Bottleneck in Impartial Games: A Circuit Complexity Approach

    Nov 10, 2024Søren RiisCircuit ComplexityNeural Network Expressivity

    Centre for Fundamentals of AI and Computational Theory, Queen Mary University of London, London, United Kingdom.