Non Convex Loss Function

Non-convex loss functions pose significant challenges in machine learning due to the absence of a globally optimal solution, hindering the development of efficient and robust algorithms. Current research focuses on developing novel algorithms for federated learning, Byzantine-resilient training, and differentially private optimization, often employing techniques like gradient normalization, adaptive client selection, and model sparsification to mitigate the difficulties associated with non-convexity. These advancements are crucial for improving the efficiency, privacy, and robustness of machine learning models in distributed and sensitive data settings, impacting various applications from healthcare to large-scale data analysis.

Papers

October 24, 2024

Faster Algorithms for User-Level Private Stochastic Convex Optimization
Andrew Lowy, Daogao Liu, Hilal Asi
Differential Privacy Private Stochastic Non Convex Loss Function Fast Algorithm Private Gradient User Level Differential Privacy Smooth Loss Function

October 10, 2024

A Generalization Result for Convergence in Learning-to-Optimize
Michael Sucker, Peter Ochs
Optimization Purpose Early Stage Convergence Optimization Algorithm Generalization Property Non Convex Loss Function Learned Optimizers Deterministic Optimization

October 3, 2024

FedScalar: A Communication efficient Federated Learning
M. Rostami, S. S. Kia
Machine Learning Federated Learning Timely Communication Communication Efficiency Non Convex Loss Function

September 15, 2024

Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions
Siqiao Mu, Diego Klabjan
Machine Unlearning Harmful Unlearning Non Convex Loss Function Non Convex Objective Convex Loss Function Nonconvex Function

August 19, 2024

Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasets
Xingrun Yan, Shiyuan Zuo, Rongfei Fan, Han Hu, Li Shen, Puning Zhao, Yong Luo
Data Set Federated Learning Convergence Analysis Non Convex Loss Function Edge Server Stable Convergence

August 18, 2024

Byzantine-resilient Federated Learning Employing Normalized Gradients on Non-IID Datasets
Shiyuan Zuo, Xingrun Yan, Rongfei Fan, Li Shen, Puning Zhao, Jie Xu, Han Hu
Data Set Non Convex Loss Function Byzantine Robust Federated Learning Federated Learning Approach Optimality Gap Federated Stochastic Byzantine Robust Method

August 16, 2024

The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacy
Jiating Ma, Yipeng Zhou, Qi Li, Quan Z. Sheng, Laizhong Cui, Jiangchuan Liu
Absolute Stance Bias Real Power Client Selection Convex Loss Non Convex Loss Function Heterogeneous Privacy

August 10, 2024

fastkqr: A Fast Algorithm for Kernel Quantile Regression
Qian Tang, Yuwen Gu, Boxiang Wang
Quantile Regression Non Convex Loss Function Fast Algorithm

July 26, 2024

FedAR: Addressing Client Unavailability in Federated Learning with Local Update Approximation and Rectification
Chutian Jiang, Hansong Zhou, Xiaonan Zhang, Shayok Chakraborty
Global Model Non Convex Loss Function Optimal Convergence Rate Client Availability Noise Rectification Local Client Model

April 1, 2024

Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization
Qi Zhang, Yi Zhou, Ashley Prater-Bennette, Lixin Shen, Shaofeng Zou
Robust Optimization Distributionally Robust Optimization Non Convex Loss Function Convex Loss Function

March 20, 2024

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity
Shiyuan Zuo, Xingrun Yan, Rongfei Fan, Han Hu, Hangguan Shan, Tony Q. S. Quek
Adaptive Importance Data Heterogeneity Non Convex Loss Function Byzantine Attack Byzantine Robust Federated Learning Convex Loss Function

February 17, 2024

How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization
Andrew Lowy, Jonathan Ullman, Stephen J. Wright
Natural Gradient Convex Function Non Convex Loss Function Fast Rate Private Convex Optimization Private Empirical Risk Minimization

February 10, 2024

Understanding the Training Speedup from Sampling with Approximate Losses
Rudrajit Das, Xi Chen, Bertram Ieong, Parikshit Bansal, Sujay Sanghavi
Faster Training Non Convex Loss Function Planning Loss Greedy Approach Loss Estimation

February 8, 2024

Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss
Zhenlong Liu, Lei Feng, Huiping Zhuang, Xiaofeng Cao, Hongxin Wei
Loss Function Gradient Descent Membership Inference Attack Privacy Risk Membership Inference Convex Loss Non Convex Loss Function

January 24, 2024

How to Collaborate: Towards Maximizing the Generalization Performance in Cross-Silo Federated Learning
Yuchang Sun, Marios Kountouris, Jun Zhang
Strong Generalization Generalization Performance Non Convex Loss Function Cross Silo Federated Learning Collaborative Training

November 29, 2023

Adam-like Algorithm with Smooth Clipping Attains Global Minima: Analysis Based on Ergodicity of Functional SDEs
Keisuke Suzuki
General Analysis Adam Optimizer Non Convex Loss Function Global Minimum Markov Process Adaptive Clipping Geometric Ergodicity

November 9, 2023

Accelerated Shapley Value Approximation for Data Evaluation
Lauren Watson, Zeno Kujawa, Rayna Andreeva, Hao-Tsung Yang, Tariq Elahi, Rik Sarkar
Shapley Value Efficient Learning Non Convex Loss Function

October 30, 2023

Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification
Yiwei Li, Chien-Wei Huang, Shuai Wang, Chong-Yung Chi, Tony Q. S. Quek
Differential Privacy Primal Dual Federated Prompt Cooperation Non Convex Non Convex Loss Function Primal Dual Algorithm Non Smooth Non Convexity Model Sparsification

October 27, 2023

Submodel Partitioning in Hierarchical Federated Learning: Algorithm Design and Convergence Analysis
Wenzhi Fang, Dong-Jun Han, Christopher G. Brinton
Convergence Analysis Non Convex Loss Function Hierarchical Federated Learning Algorithm Design Federated Learning Architecture Submodel Extraction Model Partition

October 23, 2023

Efficient Active Learning Halfspaces with Tsybakov Noise: A Non-convex Optimization Approach
Yinan Li, Chicheng Zhang
Non Convex Optimization General Nonconvex Non Convex Loss Function Margin Halfspaces Learning Halfspaces Label Complexity Pi Noise

Non Convex Loss Function

Papers

Faster Algorithms for User-Level Private Stochastic Convex Optimization

A Generalization Result for Convergence in Learning-to-Optimize

FedScalar: A Communication efficient Federated Learning

Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions

Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasets

Byzantine-resilient Federated Learning Employing Normalized Gradients on Non-IID Datasets

The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacy

fastkqr: A Fast Algorithm for Kernel Quantile Regression

FedAR: Addressing Client Unavailability in Federated Learning with Local Update Approximation and Rectification

Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity

How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization

Understanding the Training Speedup from Sampling with Approximate Losses

Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss

How to Collaborate: Towards Maximizing the Generalization Performance in Cross-Silo Federated Learning

Adam-like Algorithm with Smooth Clipping Attains Global Minima: Analysis Based on Ergodicity of Functional SDEs

Accelerated Shapley Value Approximation for Data Evaluation

Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification

Submodel Partitioning in Hierarchical Federated Learning: Algorithm Design and Convergence Analysis

Efficient Active Learning Halfspaces with Tsybakov Noise: A Non-convex Optimization Approach