Huber Loss

The Huber loss function is a robust loss function used in machine learning that combines the strengths of mean squared error (MSE) and mean absolute error (MAE), mitigating the impact of outliers while maintaining sensitivity to smaller errors. Current research focuses on adapting the Huber loss, often through dynamic parameter adjustments or variations like the pseudo-Huber loss, to improve performance in diverse applications such as image generation (diffusion models), time series analysis (nonlinear system parameter estimation), and federated learning. This adaptability makes the Huber loss a valuable tool across various fields, enhancing the robustness and accuracy of models in the presence of noisy or corrupted data. Its impact is seen in improved model performance and more reliable estimations in numerous applications, from 5G network optimization to reinforcement learning.

Papers

August 27, 2024

Residual-based Adaptive Huber Loss (RAHL) -- Design of an improved Huber loss for CQI prediction in 5G networks
Mina Kaviani, Jurandy Almeida, Fabio L. Verdi
Loss Function G Network Huber Loss

May 31, 2024

Unified Directly Denoising for Both Variance Preserving and Variance Exploding Diffusion Models
Jingjing Wang, Dan Zhang, Feng Luo
Denoising Diffusion Model Huber Loss

May 22, 2024

A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacy
Puning Zhao, Lifeng Lai, Li Shen, Qingming Li, Jiafei Wu, Zhe Liu
Estimation Task User Level Differential Privacy Huber Loss

March 25, 2024

Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Loss
Artem Khrapov, Vadim Popov, Tasnima Sadekova, Assel Yermekova, Mikhail Kudinov
Diffusion Model Loss Function Reverse Diffusion Huber Loss Diffusion Loss

January 4, 2024

A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning
Parvin Malekzadeh, Konstantinos N. Plataniotis, Zissis Poulos, Zeyu Wang
Distributional Reinforcement Learning Parameter Update Huber Loss Huber Distribution

December 3, 2023

Robust Non-parametric Knowledge-based Diffusion Least Mean Squares over Adaptive Networks
Soheil Ashkezari-Toussi, Hadi sadoghi-Yazdi
Robust Estimation Kernel Density Adaptive Network Non Gaussian Huber Loss Diffusion Learning Collective Estimation

August 24, 2023

A Huber Loss Minimization Approach to Byzantine Robust Federated Learning
Puning Zhao, Fei Yu, Zhiguo Wan
Adversarial Attack Federated Learning Byzantine Robust Federated Learning Huber Loss Optimal Dependence Modern News Aggregator

August 23, 2023

Machine learning in parameter estimation of nonlinear systems
Kaushal Kumar
Machine Learning Nonlinear Dynamic Parameter Estimation Complex Dynamical System Huber Loss

May 26, 2023

Robust Nonparametric Regression under Poisoning Attack
Puning Zhao, Zhiguo Wan
Poisoning Attack Robust Regression Huber Loss Nadaraya Watson

March 9, 2023

Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards
Xiang Li, Qiang Sun
Linear Bandit Linear Function Approximation Robust Reinforcement Learning Huber Loss Heavy Tailed Reward Variance Aware Variance Dependent Regret Online Sequential Decision Making

January 18, 2023

An Analysis of Loss Functions for Binary Classification and Regression
Jeffrey Buzas
General Analysis Loss Function Novel Regression Binary Classification Logistic Regression Margin Loss Huber Loss

September 15, 2022

DEQGAN: Learning the Loss Function for PINNs with Generative Adversarial Networks
Blake Bullwinkel, Dylan Randle, Pavlos Protopapas, David Sondak
Generative Adversarial Network Loss Function Physic Informed Neural Network Partial Differential Equation Differential Equation Huber Loss

August 9, 2022

Statistical Properties of the log-cosh Loss Function Used in Machine Learning
Resve A. Saleh, A. K. Md. Ehsanes Saleh
Machine Learning Loss Function Statistical Property Huber Loss

June 9, 2022

Robust Matrix Completion with Heavy-tailed Noise
Bingyan Wang, Jianqing Fan
Matrix Factorization Matrix Completion Low Rank Matrix Completion Heavy Tailed Noise Huber Loss

April 4, 2022

Re-examining Distillation For Continual Object Detection
Eli Verwimp, Kuo Yang, Sarah Parisot, Hong Lanqing, Steven McDonagh, Eduardo Pérez-Pellitero, Matthias De Lange, Tinne Tuytelaars
Object Detection Model Training Model Object Classification Continual Object Detection Huber Loss One Stage Self Distillation

February 22, 2022

Nonconvex Extension of Generalized Huber Loss for Robust Learning and Pseudo-Mode Statistics
Kaan Gokcesu, Hakan Gokcesu
Robust Learning Convex Loss General Nonconvex Robust Loss Function Huber Loss Exponential Convergence Pseudo Stochastic

Huber Loss

Papers

Residual-based Adaptive Huber Loss (RAHL) -- Design of an improved Huber loss for CQI prediction in 5G networks

Unified Directly Denoising for Both Variance Preserving and Variance Exploding Diffusion Models

A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacy

Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Loss

A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning

Robust Non-parametric Knowledge-based Diffusion Least Mean Squares over Adaptive Networks

A Huber Loss Minimization Approach to Byzantine Robust Federated Learning

Machine learning in parameter estimation of nonlinear systems

Robust Nonparametric Regression under Poisoning Attack

Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards

An Analysis of Loss Functions for Binary Classification and Regression

DEQGAN: Learning the Loss Function for PINNs with Generative Adversarial Networks

Statistical Properties of the log-cosh Loss Function Used in Machine Learning

Robust Matrix Completion with Heavy-tailed Noise

Re-examining Distillation For Continual Object Detection

Nonconvex Extension of Generalized Huber Loss for Robust Learning and Pseudo-Mode Statistics