Overview

Shallow neural networks—typically models with one hidden layer—are studied for what functions they can represent, how they learn, and when their compact structure is useful. Approximation theory uses tools such as Fourier and spherical-harmonic analysis to give error and generalization guarantees, while optimization research examines gradient descent, loss landscapes, symmetry, and mean-field dynamics. Practical approaches include changing activation functions, regularizing parameter norms, and making individual units more expressive, with applications ranging from differential equations to compact classifiers. A central tension is that width and richer units can improve approximation, but depth may be essential for some tasks, and representational capacity alone does not ensure that training finds a good solution.

Latest papers

CardsList

64 papers11 papers in the last four weeks, up 83% on the four weeks before. 0.1% of all new papers.

  1. Barron Approximation and locally optimal weight densities for shallow neural networks

    Nov 15, 2026G. WelperShallow Neural NetworksNeural Network Optimization

  2. Geometry-Dependent Approximation and Stability Properties of Shallow ReLUk^k Networks on Bounded Euclidean Domains

    Nov 10, 2026Yuqing Liu, Tong Mao, Jinchao XuShallow Neural NetworksReLU Neural Networks

  3. Sharp Generalization for Nonparametric Regression in Interpolation Space by Shallow Over-Parameterized Neural Networks with Channel Attention

    Nov 1, 2026Yingzhen Yang, Ping LiNeural Network GeneralizationNonparametric Regression

  4. Boundary Geometry Determines Optimal Shallow ReLU Approximation under Weighted Coefficient Constraints

    Oct 27, 2026Jia-Qi Yang, Lei ShiNeural Network Approximation TheoryShallow Neural Networks

  5. Silent Alignment in Two-Layer Linear Networks: A High-Dimensional Perspective

    Oct 21, 2026Simon Martin, Francis Bach, Giulio BiroliTwo-Layer Neural NetworksNeural Network Training Dynamics

  6. Two ReLU⁡\operatorname{ReLU} Layers is All You Need

    Oct 20, 2026Florestan Brunck, Christoph Hertrich, Kilian Ruess +2ReLU Neural NetworksShallow Neural Networks

  7. Diffusion Score Estimation with Shallow Neural Networks: Regularity, Approximation, and Finite-Sample Rates

    Oct 19, 2026Fan Liu, Yahong Yang, Wei ZhuScore-Based Generative ModelingDiffusion Models

  8. A "feature ODE" describing the learning behavior of shallow MLPs on simple functions

    Oct 19, 2026Joseph Turnbull, Berkan Ottlik, Jamie SimonNeural Network Training DynamicsShallow Neural Networks

  9. Activation-Adjacent Layer Normalization: Effects on Approximation Power

    Oct 15, 2026Yunhao Ni, Yuxin Guo, Jie Luo +2Layer NormalizationNeural Network Approximation Theory

  10. Removing spurious minima for planar features by skip connections

    Oct 1, 2026Jakob Paul Zimmermann, Andrei Balakin, Moritz Grillo +1Shallow Neural NetworksReLU Neural Networks

  11. Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks

    Sep 17, 2026Xianjun Li, Yunfei YangShallow Neural NetworksReLU Neural Networks

  12. Algebraic Expressivity Certificates for Shallow Polynomial Neural Networks

    Sep 11, 2026Sepehr Akbari, Shahrzad JamshidiNeural Network ExpressivityShallow Neural Networks

  13. Shallow neural network approximation in mixed Sobolev spaces

    Sep 4, 2026Yuwen Li, Guozhi ZhangNeural Network Approximation TheoryFunction Approximation

  14. Every Layer Counts: An Exponential L2L_2 Depth Hierarchy for ReLU Networks

    Aug 24, 2026Itay SafranShallow Neural NetworksReLU Neural Networks

  15. Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces

    Aug 21, 2026Pablo M. Berná, Antonio Falcó, Diego MondéjarNeural Network Approximation TheoryFunction Approximation

  16. Convex losses and their applications to SVM, SVR, and Shallow Neural Networks

    Aug 14, 2026Filippo PorteraSupport Vector MachinesNeural Network Generalization

  17. A Novel Fourier Feature Network for Solving Partial Differential Equations

    Aug 13, 2026Qihong Yang, Zhijie Su, Yangtao Deng +1Shallow Neural NetworksFourier Feature Embeddings

  18. Approximation Rates for Metaplectic Neural Networks

    Aug 9, 2026Ahmed Abdeljawad, Marcello Carioni, Elena CorderoShallow Neural NetworksNeural Network Approximation Theory

  19. Shallower ReLU Network Representations via Exact Linear Algebra

    Jul 22, 2026Kilian Rueß, Gennadiy Averkov, Florestan Brunck +7Shallow Neural NetworksReLU Neural Networks

  20. Self-organizing Architecture of Receptron Units: a Hardware-Aware Framework for Edge Intelligence

    Jul 22, 2026Stefano Radice, Ludovico Casaccia, Riccaro Emanuele Beccalli +2Shallow Neural NetworksNeuromorphic Computing