Non Convex Objective

Non-convex objective functions pose significant challenges in optimization problems across various machine learning applications, hindering the development of efficient and provably convergent algorithms. Current research focuses on developing and analyzing novel first-order and second-order methods, including variants of gradient descent, Adam, AdaGrad, and proximal gradient methods, to address the complexities of non-convex landscapes in settings such as federated learning and deep neural networks. These advancements are crucial for improving the performance and theoretical understanding of many machine learning models, impacting areas like data privacy (through certified unlearning), distributed optimization, and robust decision-making.

Papers

October 29, 2024

Vertical Federated Learning with Missing Features During Training and Inference
Pedro Valdeira, Shiqiang Wang, Yuejie Chi
Neural Network Training Data Scientific Inference Efficient Training Vertical Federated Learning Non Convex Objective

October 22, 2024

Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality
Sebastian Kassing, Simon Weissmann
Early Stage Convergence Lyapunov Function Non Convex Objective Universal Rate Convex Objective Heavy Ball

October 14, 2024

Learning to Optimize for Mixed-Integer Non-linear Programming
Bo Tang, Elias B. Khalil, Ján Drgoňa
Non Convex Objective Learned Optimizers Nonlinear Programming Integer Arithmetic Nonlinear Constraint

September 15, 2024

Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions
Siqiao Mu, Diego Klabjan
Machine Unlearning Harmful Unlearning Non Convex Loss Function Non Convex Objective Convex Loss Function Nonconvex Function

August 1, 2024

Towards Certified Unlearning for Deep Neural Networks
Binchi Zhang, Yushun Dong, Tianhao Wang, Jundong Li
Deep Neural Network Machine Unlearning Unlearning Method Non Convex Objective Convex Machine Learning

May 2, 2024

Sharp Bounds for Sequential Federated Learning on Heterogeneous Data
Yipeng Li, Xinchen Lyu
Federated Learning Heterogeneous Data Convergence Guarantee Convex Function Non Convex Objective Convex Objective Sharp Bound

March 18, 2024

Decentralized Stochastic Subgradient Methods for Nonsmooth Nonconvex Optimization
Siyuan Zhang, Nachuan Xiao, Xin Liu
Non Convex Objective Nonsmooth Nonconvex Optimization Decentralized Riemannian

February 23, 2024

Convergence Analysis of Split Federated Learning on Heterogeneous Data
Pengchao Han, Chao Huang, Geng Tian, Ming Tang, Xin Liu
Model Training Convergence Analysis Heterogeneous Data Non Convex Objective Split Federated Learning Convex Objective

January 20, 2024

Decentralized Optimization in Networks with Arbitrary Delays
Tomas Ortega, Hamid Jafarkhani
Network Programming Decentralized Optimization Gossip Algorithm Non Convex Objective Centralized Stochastic Gradient Descent Arbitrary Delay

November 26, 2023

A convergence result of a continuous model of deep learning via \L{}ojasiewicz--Simon inequality
Noboru Isobe
Deep Learning Loss Function Convergence Analysis Wasserstein Gradient Flow Non Convex Objective Continuous Model

November 6, 2023

Convergence Analysis of Sequential Federated Learning on Heterogeneous Data
Yipeng Li, Xinchen Lyu
Federated Learning Convergence Analysis Heterogeneous Data Convergence Guarantee Non Convex Objective Convergence Theory

September 18, 2023

A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learning
Pedro Valdeira, Yuejie Chi, Cláudia Soares, João Xavier
Communication Efficient Non Convex Objective Coordinate Descent

August 11, 2023

DF2: Distribution-Free Decision-Focused Learning
Lingkai Kong, Wenhao Mu, Jiaming Cui, Yuchen Zhuang, B. Aditya Prakash, Bo Dai, Chao Zhang
LeArning Abstract Gradient Approximation Non Convex Objective Forecasting Task Predict Then Optimize Downstream Decision Making

July 25, 2023

Federated Distributionally Robust Optimization with Non-Convex Objectives: Algorithm and Analysis
Yang Jiao, Kai Yang, Dongjin Song
Native Robustness General Analysis Practical Algorithm Robust Optimization Distributionally Robust Optimization Non Convex Objective Robustness Trade

July 20, 2023

Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case
Meixuan He, Yuqing Liang, Jinlan Liu, Dongpo Xu
Early Stage Convergence Non Convex Objective Non Convex Stochastic Optimization Non Ergodic

June 26, 2023

Nonconvex Stochastic Bregman Proximal Gradient Method for Nonconvex Composite Problems
Kuangyu Ding, Jingyang Li, Kim-Chuan Toh
Deep Learning Application Proficiency Gradient Method Proximal Gradient Non Convex Objective Bregman Information Lipschitz Gradient Bregman Proximal

June 6, 2023

Understanding Progressive Training Through the Framework of Randomized Coordinate Descent
Rafał Szlendak, Elnur Gasanov, Peter Richtárik
New Framework Objective Function Non Convex Objective Coordinate Descent Progressive Training

June 4, 2023

Complexity of Block Coordinate Descent with Proximal Regularization and Applications to Wasserstein CP-dictionary Learning
Dohyun Kwon, Hanbaek Lyu
Financial Application Complexity Matter Dictionary Learning Non Convex Objective Proximal Gradient Descent Block Coordinate Descent

May 31, 2023

Federated Learning in the Presence of Adversarial Client Unavailability
Lili Su, Ming Xiang, Jiaming Xu, Pengkun Yang
Speech Presence Non Convex Objective Convex Objective

May 29, 2023

Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions
Bohan Wang, Huishuai Zhang, Zhi-Ming Ma, Wei Chen
Early Stage Convergence Gradient Norm Non Convex Objective Smoothness Assumption Simple Proof

Non Convex Objective

Papers

Vertical Federated Learning with Missing Features During Training and Inference

Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality

Learning to Optimize for Mixed-Integer Non-linear Programming

Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions

Towards Certified Unlearning for Deep Neural Networks

Sharp Bounds for Sequential Federated Learning on Heterogeneous Data

Decentralized Stochastic Subgradient Methods for Nonsmooth Nonconvex Optimization

Convergence Analysis of Split Federated Learning on Heterogeneous Data

Decentralized Optimization in Networks with Arbitrary Delays

A convergence result of a continuous model of deep learning via \L{}ojasiewicz--Simon inequality

Convergence Analysis of Sequential Federated Learning on Heterogeneous Data

A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learning

DF2: Distribution-Free Decision-Focused Learning

Federated Distributionally Robust Optimization with Non-Convex Objectives: Algorithm and Analysis

Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case

Nonconvex Stochastic Bregman Proximal Gradient Method for Nonconvex Composite Problems

Understanding Progressive Training Through the Framework of Randomized Coordinate Descent

Complexity of Block Coordinate Descent with Proximal Regularization and Applications to Wasserstein CP-dictionary Learning

Federated Learning in the Presence of Adversarial Client Unavailability

Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions