Shallow Neural Network

Shallow neural networks, characterized by a single hidden layer, are a focus of research aiming to understand fundamental aspects of neural network learning and approximation capabilities. Current research investigates their optimization dynamics, exploring various training algorithms like gradient flow and its variants, and analyzing the impact of network architecture (e.g., activation functions, width) and data properties on performance and generalization. This research is significant because it provides crucial insights into the theoretical underpinnings of deep learning, informing the design of more efficient and robust models for applications ranging from function approximation to signal processing and time series analysis.

Papers

November 30, 2022

Limitations on approximation by deep and shallow neural networks
Guergana Petrova, Przemysław Wojtaszczyk
Average Approximation Fundamental Limitation Lower Bound Shallow Neural Network

November 8, 2022

Finite Sample Identification of Wide Shallow Neural Networks with Biases
Massimo Fornasier, Timo Klock, Marco Mondelli, Michael Rauchensteiner
Neural Network Gradient Descent Topic Bias Shallow Neural Network Finite Sample Input Output Pair

October 27, 2022

Learning Single-Index Models with Shallow Neural Networks
Alberto Bietti, Joan Bruna, Clayton Sanford, Min Jae Song
Gradient Flow Shallow Neural Network Shallow Network Low Dimensional Structure Single Index Model

October 25, 2022

Proximal Mean Field Learning in Shallow Neural Networks
Alexis Teter, Iman Nodozi, Abhishek Halder
Mean Field Learning Dynamic Shallow Neural Network Wasserstein Gradient Flow Proximal Gradient Descent

October 6, 2022

Matching Text and Audio Embeddings: Exploring Transfer-learning Strategies for Language-based Audio Retrieval
Benno Weck, Miguel Pérez Fernández, Holger Kirchhoff, Xavier Serra
Deep Learning Model Jina Embeddings General Strategy Shallow Neural Network Audio Embeddings Text Matching Language Based Audio Retrieval

September 26, 2022

Mental arithmetic task classification with convolutional neural network based on spectral-temporal features from EEG
Zaineb Ajra, Binbin Xu, Gérard Dray, Jacky Montmain, Stephane Perrey
Convolutional Neural Network Deep Neural Network Mathematical Reasoning Shallow Neural Network Spectro Temporal Mental Health Assessment

September 24, 2022

From Local to Global: Spectral-Inspired Graph Neural Networks
Ningyuan Huang, Soledad Villar, Carey E. Priebe, Da Zheng, Chengyue Huang, Lin Yang, Vladimir Braverman
Graph Neural Network World Event Shallow Neural Network Spectral Graph

September 19, 2022

Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks
Yunwen Lei, Rong Jin, Yiming Ying
Strong Generalization Gradient Descent Core Stability Gradient Method Shallow Neural Network

September 17, 2022

Approximation results for Gradient Descent trained Shallow Neural Networks in $1d$
R. Gentile, G. Welper
Gradient Descent Non Convex Optimization Shallow Neural Network Shallow Network Approximation Property Width Network

August 17, 2022

Shallow neural network representation of polynomials
Aleksandr Beknazaryan
Shallow Neural Network Shallow Network Polynomial System Smooth Function Regression Function Polynomial Expansion

July 18, 2022

Explainable Deep Belief Network based Auto encoder using novel Extended Garson Algorithm
Satyam Kumar, Vadlamani Ravi
Deep Neural Network Auto Encoder Shallow Neural Network Explainable Deep

July 15, 2022

Local Approximations, Real Interpolation and Machine Learning
Eric Setterqvist, Natan Kruglyak, Robert Forchheimer
Neural Network Machine Learning Shallow Neural Network Classification Error Local Approximation Linear Interpolation

May 18, 2022

A Classification of $G$-invariant Shallow Neural Networks
Devanshu Agrawal, James Ostrowski
Classification Code Gene Level GNN Shallow Neural Network Invariant Layer

February 22, 2022

Distilled Neural Networks for Efficient Learning to Rank
F. M. Nardini, C. Rulli, S. Trani, R. Venturini
Knowledge Distillation Efficient Learning Stable Rank Shallow Neural Network Tree Based Ensemble

February 3, 2022

Non-Vacuous Generalisation Bounds for Shallow Neural Networks
Felix Biggs, Benjamin Guedj
PAC Bayesian Shallow Neural Network Sigmoid Function Vanilla Gradient Descent Non Vacuous Generalization Bound

January 28, 2022

Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks
Bartłomiej Polaczyk, Jacek Cyranka
Stochastic Gradient Descent Gradient Flow Global Convergence Shallow Neural Network Overparametrization Bound

January 26, 2022

Physics-informed ConvNet: Learning Physical Field from a Shallow Neural Network
Pengpeng Shi, Zhi Zeng, Tianshou Liang
Physic Informed Machine Learning Physic Informed Shallow Neural Network Fully Connected Neural Network Shallow Layer Physical Law Learning Nonlinear Operator Mapping Wave Speed

December 20, 2021

Integral representations of shallow neural network with Rectified Power Unit activation function
Ahmed Abdeljawad, Philipp Grohs
Shallow Neural Network Shallow Network Integral Regression

December 9, 2021

Millimeter Wave Localization with Imperfect Training Data using Shallow Neural Networks
Anish Shastri, Joan Palacios, Paolo Casari
Shallow Neural Network mmWave Signal Localization Algorithm

December 5, 2021

On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons
Fangshuo Liao, Anastasios Kyrillidis
Neural Network Early Stage Convergence Shallow Neural Network Random Masking