Smooth Loss Function

Smooth loss functions are crucial in machine learning for enabling efficient optimization algorithms and improving model robustness. Current research focuses on developing novel smooth loss functions tailored for specific challenges, such as handling outliers, high-dimensional data, and adversarial examples, often within the context of gradient descent methods and their variants like Frank-Wolfe. These advancements are impacting various fields, including brain image registration, natural language processing, and causal inference, by enhancing model accuracy, stability, and generalization performance. The development of theoretically sound and computationally efficient algorithms for optimizing these functions remains a key area of investigation.

Papers

October 24, 2024

Faster Algorithms for User-Level Private Stochastic Convex Optimization
Andrew Lowy, Daogao Liu, Hilal Asi
Differential Privacy Private Stochastic Non Convex Loss Function Fast Algorithm Private Gradient User Level Differential Privacy Smooth Loss Function

October 1, 2024

Tight Rates for Bandit Control Beyond Quadratics
Y. Jennifer Sun, Zhou Lu
Convex Optimization Optimal Regret Optimal Rate Smooth Loss Function Bandit Convex Quadratic Number

September 2, 2024

Large Scale Unsupervised Brain MRI Image
Yuxi Zhang, Xiang Chen, Jiazheng Wang, Min Liu, Yaonan Wang, Dongdong Liu, Renjiu Hu, Hang Zhang
Loss Function Segmentation Label Brain MR Image Unsupervised Version Smooth Loss Function

June 8, 2024

Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss
Bowen Zhang, Chunping Li
Contrastive Learning Task Specific Semantic Relation Semantic Textual Similarity Smooth Loss Function Text Pair Regression Framework

May 29, 2024

MGDA Converges under Generalized Smoothness, Provably
Qi Zhang, Peiyao Xiao, Shaofeng Zou, Kaiyi Ji
Early Stage Convergence Multi Objective Optimization Multi Objective Gradient Norm Smooth Loss Function Generalized Smoothness

May 12, 2024

PHUDGE: Phi-3 as Scalable Judge
Mahesh Deshwal, Apoorva Chawla
Training Data Fine Tuned Judge Model Phi 3 Smooth Loss Function

March 11, 2024

Gaussian Loss Smoothing Enables Certified Training with Tight Convex Relaxations
Stefan Balauca, Mark Niklas Müller, Yuhao Mao, Maximilian Baader, Marc Fischer, Martin Vechev
Training Data Convex Relaxation Fermi Paradox Robust Neural Network Smooth Loss Function

February 8, 2024

Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients
Yuri Fonseca, Caio Peixoto, Yuri Saporito
Stochastic Gradient Instrumental Variable Smooth Loss Function Nonparametric Instrumental Variable

January 30, 2024

HawkEye: Advancing Robust Regression with Bounded, Smooth, and Insensitive Loss Function
Mushir Akhtar, M. Tanveer, Mohd. Arshad
Loss Function Robust Regression Support Vector Regression Norm Bounded Smooth Loss Function

December 21, 2023

On the Convergence of Loss and Uncertainty-based Active Learning Algorithms
Daniel Haimovich, Dima Karamshuk, Fridolin Linder, Niek Tax, Milan Vojnovic
Loss Function Stochastic Gradient Descent Early Stage Convergence Interest Loss Uncertainty Based Active Learning Smooth Loss Function

November 19, 2023

TimeSQL: Improving Multivariate Time Series Forecasting with Multi-Scale Patching and Smooth Quadratic Loss
Site Mo, Haoxin Wang, Bixiong Li, Songhai Fan, Yuankai Wu, Xianggen Liu
Multivariate Time Series Forecasting Time Series Prediction Smooth Loss Function World Dynamic Long Range Correlation Temporal Query Multi Scale Patch

September 5, 2023

RoBoSS: A Robust, Bounded, Sparse, and Smooth Loss Function for Supervised Learning
Mushir Akhtar, M. Tanveer, Mohd. Arshad
Loss Function Supervised Learning Many Sparse Support Vector Machine Sparse Optimization Norm Bounded Smooth Loss Function

March 17, 2023

Alternate Loss Functions for Classification and Robust Regression Can Improve the Accuracy of Artificial Neural Networks
Mathew Mithra Noel, Arindam Banerjee, Yug Oswal, Geraldine Bessie Amali D, Venkataraman Muthiah-Nakarajan
Neural Network Classification Code Loss Function Robust Regression Smooth Loss Function

March 2, 2023

Tight Risk Bounds for Gradient Descent on Separable Data
Matan Schliserman, Tomer Koren
Gradient Descent Stochastic Gradient Descent Gradient Method Gradient Free Separable Data Risk Bound Smooth Loss Function

August 30, 2022

Using Taylor-Approximated Gradients to Improve the Frank-Wolfe Method for Empirical Risk Minimization
Zikai Xiong, Robert M. Freund
Empirical Risk Minimization Gradient Approximation Smooth Loss Function Frank Wolfe

July 17, 2022

Fast Composite Optimization and Statistical Recovery in Federated Learning
Yajie Bao, Michael Crawshaw, Shan Luo, Mingrui Liu
Robust Recovery Smooth Loss Function Composite Optimization

February 10, 2022

Generalization Bounds via Convex Analysis
Gábor Lugosi, Gergely Neu
Generalization Bound P$ Norm Smooth Loss Function Convex Analysis Subgaussian Assumption

January 6, 2022

Federated Optimization of Smooth Loss Functions
Ali Jadbabaie, Anuran Makur, Devavrat Shah
Empirical Risk Minimization Federated Averaging Federated Optimization Federated Learning Framework Federated Stochastic Smooth Loss Function