Uniform Stability

Uniform stability, a measure of an algorithm's sensitivity to changes in the training data, is a crucial concept in machine learning, aiming to guarantee generalization performance and robustness. Current research focuses on establishing uniform stability bounds for various algorithms, including stochastic gradient descent and its compositional variants, and exploring its application in challenging scenarios like adversarial training and federated learning, often within the context of specific model architectures such as neural networks and physics-informed neural networks. These investigations are vital for improving the reliability and trustworthiness of machine learning models by providing theoretical guarantees on their generalization ability and resistance to noise and adversarial attacks.

Papers

September 8, 2024

Some Results on Neural Network Stability, Consistency, and Convergence: Insights into Non-IID Data, High-Dimensional Settings, and Physics-Informed Neural Networks
Ronald Katende, Henry Kasumba, Godwin Kakuba, John M. Mango
Federated Learning Physic Informed Neural Network High Dimensional Strong Consistency Learning Rate Convergence Guarantee Stable Neural Network Non IID Data Uniform Stability

May 3, 2024

Uniformly Stable Algorithms for Adversarial Training and Beyond
Jiancong Xiao, Jiawei Zhang, Zhi-Quan Luo, Asuman Ozdaglar
Adversarial Training Adversarial Loss Moreau Envelope Uniform Stability Stable Algorithm

July 7, 2023

Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms
Ming Yang, Xiyuan Wei, Tianbao Yang, Yiming Ying
Strong Generalization Core Stability Compositional Optimization Uniform Stability

July 17, 2022

Uniform Stability for First-Order Empirical Risk Minimization
Amit Attia, Tomer Koren
Empirical Risk Minimization Optimization Algorithm Smooth Optimization Uniform Stability

April 4, 2022

Differentially Private Sampling from Rashomon Sets, and the Universality of Langevin Diffusion for Convex Optimization
Arun Ganesh, Abhradeep Thakurta, Jalaj Upadhyay
Langevin Dynamic Convex Optimization K$ Universality Privacy Analysis Exponential Mechanism Private Data Analysis Approximate Differential Privacy Rashomon Set Uniform Stability

Uniform Stability

Papers

Some Results on Neural Network Stability, Consistency, and Convergence: Insights into Non-IID Data, High-Dimensional Settings, and Physics-Informed Neural Networks

Uniformly Stable Algorithms for Adversarial Training and Beyond

Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms

Uniform Stability for First-Order Empirical Risk Minimization

Differentially Private Sampling from Rashomon Sets, and the Universality of Langevin Diffusion for Convex Optimization