Convergence Theory

Convergence theory in machine learning investigates the conditions under which algorithms reliably approach optimal solutions, focusing on quantifying the rate and quality of this convergence. Current research emphasizes analyzing the convergence of generative models like diffusion probabilistic models and score-based samplers, often focusing on minimizing assumptions about the data distribution and improving convergence rates with respect to data dimensionality. This work is crucial for establishing the reliability and efficiency of these algorithms, impacting the development of robust and scalable machine learning systems across various applications, including federated learning and quantum computing. Furthermore, understanding convergence in diverse settings, such as those with heterogeneous data or Byzantine failures, is driving the development of more resilient and efficient algorithms.

Papers

September 27, 2024

$O(d/T)$ Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions
Gen Li, Yuling Yan
Convergence Rate Diffusion Probabilistic Model Score Based Diffusion Model Modeling Assumption Optimal Convergence Rate Convergence Theory ODE Sampler

August 5, 2024

A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models
Gen Li, Yuting Wei, Yuejie Chi, Yuxin Chen
Diffusion Model Probability Flow ODE Diffusion Sampler Non Asymptotic Analysis Non Asymptotic Convergence Convergence Theory

May 24, 2024

CAFe: Cost and Age aware Federated Learning
Sahan Liyanaarachchi, Kanchana Thilakarathna, Sennur Ulukus
Convergence Rate Hidden CoST Global Model Local Gradient Self Service CAFE Convergence Theory Horizontal Federated Learning Noisy Gradient

February 17, 2024

Random Projection Neural Networks of Best Approximation: Convergence theory and practical applications
Gianluca Fabiani
Neural Network Average Approximation Function Approximation Practical Application Random Projection Differentiable Activation Function Convergence Theory

November 6, 2023

Convergence Analysis of Sequential Federated Learning on Heterogeneous Data
Yipeng Li, Xinchen Lyu
Federated Learning Convergence Analysis Heterogeneous Data Convergence Guarantee Non Convex Objective Convergence Theory

February 13, 2023

Do PAC-Learners Learn the Marginal Distribution?
Max Hopkins, Daniel M. Kane, Shachar Lovett, Gaurav Mahajan
Mutual Information Marginal Distribution Distribution Free Probably Approximately Correct Convergence Theory

November 3, 2022

A Convergence Theory for Federated Average: Beyond Smoothness
Xiaoxiao Li, Zhao Song, Runzhou Tao, Guangyi Zhang
Stochastic Gradient Descent Convergence Analysis Gradient Norm FedAvg Algorithm Convergence Theory

September 21, 2022

On the Convergence Theory of Meta Reinforcement Learning with Personalized Policies
Haozhi Wang, Qing Wang, Yunfeng Shao, Dong Li, Jianye Hao, Yinchuan Li
Policy Gradient Meta Reinforcement Learning Policy Alignment Meta Rl Algorithm Convergence Theory Online Policy Customization

August 25, 2022

A simplified convergence theory for Byzantine resilient stochastic gradient descent
Lindon Roberts, Edward Smyth
Gradient Descent Stochastic Gradient Byzantine Robust Convergence Theory

May 25, 2022

A Convergence Theory for Over-parameterized Variational Quantum Eigensolvers
Xuchen You, Shouvanik Chakrabarti, Xiaodi Wu
Intermediate Scale Quantum Variational Quantum Parameterized Regime Convergence Theory Quantum Application