Gradient Descent AI Research Papers - Page 36

February 27, 2022

Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond
Matan Schliserman, Tomer Koren
Stochastic Convex Optimization Gradient Method Gradient Free Separable Data Core Stability Generalization Property Generalization Bound Implicit Bias Gradient Descent
Benign Underfitting of Stochastic Gradient Descent
Tomer Koren, Roi Livni, Yishay Mansour, Uri Sherman
Stochastic Gradient Descent Stochastic Convex Optimization Gradient Descent
Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization
Idan Amir, Roi Livni, Nathan Srebro
Stochastic Convex Optimization Lipschitz Loss Gradient Descent Ball Rolling Optimal Learning

February 25, 2022

An initial alignment between neural network and target is needed for gradient descent to learn
Emmanuel Abbe, Elisabetta Cornacchia, Jan Hązła, Christopher Marquis
Initial Alignment Neural Network Noisy Gradient Descent Symmetric Neural Network Target Amazon Ally Gradient Descent

February 24, 2022

On the influence of stochastic roundoff errors and their bias on the convergence of the gradient descent method with low-precision floating-point computation
Lu Xia, Stefano Massei, Michiel E. Hochstenbach, Barry Koren
Stochastic Rounding Absolute Stance Bias Gradient Descent Early Stage Convergence

February 20, 2022

A History of Meta-gradient: Gradient Methods for Meta-learning
Richard S. Sutton
Gradient Method Step Size Historical Text Gradient Descent Meta Gradient Meta Model NCD Method

February 18, 2022

Tackling benign nonconvexity with smoothing and stochastic gradients
Harsh Vardhan, Sebastian U. Stich
Nonconvexity Setting Smoothing Factor Stochastic Gradient Nonconvex Function Gradient Descent Global Convergence Non Convex Optimization Problem

February 17, 2022

Gradients without Backpropagation
Atılım Güneş Baydin, Barak A. Pearlmutter, Don Syme, Frank Wood, Philip Torr
Back Propagation Mode Automatic Differentiation Forward Gradient Gradient Descent Natural Gradient

February 16, 2022

Learning a Single Neuron for Non-monotonic Activation Functions
Lei Wu
Two Layer Neural Network Non Monotonic Activation Function Gradient Descent Single Neuron

February 15, 2022

Random Feature Amplification: Feature Learning and Generalization in Neural Networks
Spencer Frei, Niladri S. Chatterji, Peter L. Bartlett
Feature Learning Two Layer ReLU Gradient Descent Dynamic Random Feature Neural Network Gradient Descent Strong Generalization

February 14, 2022

Splitting numerical integration for matrix completion
Qianqian Song
Gradient Descent Matrix Completion Numerical Integration Split and Fit Low Dimensional Manifold

February 12, 2022

On Federated Learning with Energy Harvesting Clients
Cong Shen, Jing Yang, Jie Xu
Client Availability Non Convex Loss Function Convergence Bound Gradient Descent Energy Harvesting

February 11, 2022

February 10, 2022

Domain Adversarial Training: A Game Perspective
David Acuna, Marc T Law, Guojun Zhang, Sanja Fidler
Domain Adaptation Domain Adversarial Multiplayer Game Gradient Descent

February 9, 2022

February 8, 2022

Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning
Yang Zhao, Hao Zhang, Xiuyuan Hu
Loss Function Deep Learning Generalization Performance Strong Generalization Gradient Norm Model Generalization Gradient Descent

February 7, 2022

Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably
Tianyi Liu, Yan Li, Enlu Zhou, Tuo Zhao
Parameterized Model Stable Rank Low Rank Matrix Recovery Gradient Descent Neural Network

Gradient Descent

Papers - Page 36

Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond

Benign Underfitting of Stochastic Gradient Descent

Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization

An initial alignment between neural network and target is needed for gradient descent to learn

On the influence of stochastic roundoff errors and their bias on the convergence of the gradient descent method with low-precision floating-point computation

A History of Meta-gradient: Gradient Methods for Meta-learning

Tackling benign nonconvexity with smoothing and stochastic gradients

Gradients without Backpropagation

Learning a Single Neuron for Non-monotonic Activation Functions

Random Feature Amplification: Feature Learning and Generalization in Neural Networks

Splitting numerical integration for matrix completion

On Federated Learning with Energy Harvesting Clients

Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data

The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention

A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Domain Adversarial Training: A Game Perspective

On the Implicit Bias of Gradient Descent for Temporal Extrapolation

Improving Computational Complexity in Statistical Models with Second-Order Information

Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning

Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably