General Convex Loss

General convex loss functions are a cornerstone of many machine learning problems, aiming to optimize models by minimizing a convex objective function. Current research focuses on developing efficient algorithms for solving these optimization problems, particularly in high-dimensional settings and under conditions like noisy data or concept drift, with a focus on methods like stochastic gradient descent and second-order methods. These advancements are crucial for improving the performance and scalability of various machine learning applications, including online learning, federated learning, and distance metric learning, by providing theoretical guarantees and practical improvements in convergence rates and computational efficiency. The development of robust and efficient algorithms for general convex loss functions remains a significant area of active research with broad implications across the field.

Papers

February 14, 2024

Second Order Methods for Bandit Optimization and Control
Arun Suggala, Y. Jennifer Sun, Praneeth Netrapalli, Elad Hazan
External Control Convex Optimization Second Order Optimal Regret Online Convex Optimization General Convex Loss

June 5, 2023

Linear Distance Metric Learning with Noisy Labels
Meysam Alishahi, Anna Little, Jeff M. Phillips
Loss Function Ground Truth Noisy Label Metric Learning Metric Space General Convex Loss

April 6, 2023

Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach
Urte Adomaityte, Gabriele Sicuro, Pierpaolo Vivo
Classification Code Regularization Model High Dimension Heavy Tailed Gaussian Distribution Convex Regularization General Convex Loss Gaussian Universality Self Fitting Method

March 27, 2023

Learning Rate Schedules in the Presence of Distribution Shift
Matthew Fahrbach, Adel Javanmard, Vahab Mirrokni, Pratik Worah
Distribution Shift Speech Presence Learning Rate Schedule Online Linear Regression General Convex Loss

March 13, 2023

General Loss Functions Lead to (Approximate) Interpolation in High Dimensions
Kuo-Wei Lai, Vidya Muthukumar
Loss Function Gradient Descent Average Approximation High Dimension Interpolation Regime Convex Loss General Convex Loss Tailed Loss

November 22, 2022

May 13, 2022

Optimal Comparator Adaptive Online Learning with Switching Cost
Zhiyu Zhang, Ashok Cutkosky, Ioannis Ch. Paschalidis
Hidden CoST Adaptive Learning Optimal Algorithm Adaptive Algorithm Adaptive Regret General Convex Loss

February 12, 2022

Adaptive Bandit Convex Optimization with Heterogeneous Curvature
Haipeng Luo, Mengxiao Zhang, Peng Zhao
Convex Optimization Bandit Feedback Adversarial Bandit Bandit Convex General Convex Loss Gradient Feedback Heterogeneous Curvature

January 30, 2022

Faster Convergence of Local SGD for Over-Parameterized Models
Tiancheng Qin, S. Rasoul Etesami, César A. Uribe
Neural Network Faster Convergence Non Convex Loss Function Parameterized Model Local SGD General Convex Loss

November 4, 2021

Recurrent Neural Network Training with Convex Loss and Regularization Functions by Extended Kalman Filtering
Alberto Bemporad
Recurrent Neural Network Stochastic Gradient Descent Extended Kalman Filter Convex Loss Regularization Function General Convex Loss