Robust Loss Functions

Follow this topic Ask about this topic

Robust loss functions study how to train models when data contain outliers, incorrect labels, heavy-tailed noise, or other unreliable observations. Common approaches include bounded or truncated residual losses, symmetric and generalized cross-entropy losses, divergence-based objectives, and sample reweighting; they are used in settings from regression and image classification to scientific estimation. These methods can improve performance under contamination, but their results depend on the type and level of noise and on choices such as loss shape and scale. A central challenge is gaining robustness without sacrificing clean-data accuracy, calibration, or efficiency, while establishing guarantees that match real-world benchmarks.

Latest papers

50 papers6 papers in the last four weeks, down 14% on the four weeks before. 0.0% of all new papers.

Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels
May 19, 2026Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe GiguèreNeural Network Robustness Noisy-Label Learning
Department of Computer Science and Software Engineering Université Laval, Québec, Canada
Risk-Calibrated Learning: Minimizing Fatal Errors in Medical AI
Apr 14, 2026Abolfazl Mohammadi-Seif, Ricardo Baeza-YatesMedical Image Classification AI Safety
Department of Engineering, Universitat Pompeu Fabra, Barcelona, Spain
Socrates Loss: Unifying Confidence Calibration and Classification by Leveraging the Unknown
Apr 14, 2026Sandra Gómez-Gálvez, Tobias Olenyi, Gillian Dobbie +1Uncertainty Calibration Classification
University of Auckland · Technical University of Munich
Component-Adaptive and Lesion-Level Supervision for Improved Small Structure Segmentation in Brain MRI
Apr 9, 2026Minh Sao Khue Luu, Evgeniy N. Pavlovskiy, Bair N. TuchinovMedical Image Segmentation Robust Loss Functions
The Artificial Intelligence Research Center of Novosibirsk State University, Novosibirsk 630090, Russia
Sparse $ε$ insensitive zone bounded asymmetric elastic net support vector machines for pattern classification
Apr 9, 2026Haiyan Du, Hu YangSupport Vector Machines Robust Loss Functions
College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, China
Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise
Apr 7, 2026Yuanjie Shi, Peihong Li, Zijian Zhang +2Margin-Based Learning Learning with Noisy Labels
School of EECS, Washington State University, Pullman, WA, USA
Outlier-Robust Nonlinear Moving Horizon Estimation using Adaptive Loss Functions
Apr 6, 2026Nestor Deniz, Guido Sanchez, Fernando Auat Cheein +1Moving Horizon Estimation Robust Estimation
Instituto de Investigación en Senales, Sistemas e Inteligencia Computacional, sınc(i), UNL-CONICET, Santa Fe, Argentina · Harper Adams University, Engineer Department, Edgmond, Newport TF10 8NB
Beyond the Mean: Distribution-Aware Loss Functions for Bimodal Regression
Mar 20, 2026Abolfazl Mohammadi-Seif, Carlos Soares, Rita P. Ribeiro +1Aleatoric Uncertainty Uncertainty Quantification
Department of Engineering, Universitat Pompeu Fabra, Roc Boronat St., Barcelona, 08018, Barcelona, Spain. · Department of Engineering, University of Porto, Praça de Gomes Teixeira, Porto, 4099-002, Portugal. · Faculty of Sciences, University of Porto, 4169-007 Porto, Portugal. +1
Minimax Generalized Cross-Entropy
Mar 20, 2026Kartheek Bondugula, Santiago Mazuelas, Aritz Pérez +1Robust Loss Functions Min-Max Optimization
Basque Center for Applied Mathematics, Bilbao, Spain · Ikerbasque, Basque Foundation for Science · Johns Hopkins University, Baltimore, USA
rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks
Mar 18, 2026Suryasis Jana, Abhik GhoshNeural Network Robustness Robust Classification
Indian Statistical Institute, Kolkata, India
Global Truncated Loss Minimization for Robust and Threshold-Resilient Geometric Estimation
Mar 16, 2026Tianyu Huang, Liangzu Peng, Xinyue Zhang +5Robust Loss Functions Branch and Bound
Hong Kong Centre For Logistics Robotics, The Chinese University of Hong Kong. · Center for Innovation in Data Engineering and Science (IDEAS), University of Pennsylvania. · Mobile Perception Lab of the School of Information and Technology, ShanghaiTech University. +1
Robust support vector model based on bounded asymmetric elastic net loss for binary classification
Mar 6, 2026Haiyan Du, Hu YangRobust Loss Functions Support Vector Machines
College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, China
Margin and Consistency Supervision for Calibrated and Robust Vision Models
Mar 6, 2026Salim KhazemDistribution Shift Robustness Model Calibration
Talan Research Center, Paris, France.
MCEL: Margin-Based Cross-Entropy Loss for Error-Tolerant Quantized Neural Networks
Mar 5, 2026Mikail Yayla, Akash KumarNeural Network Robustness Robust Loss Functions
Ruhr-Universität Bochum, Germany
Functional Properties of the Focal-Entropy
Mar 3, 2026Jaimin Shah, Martina Cardone, Alex DytsoFocal Loss Class-Imbalanced Learning
University of Minnesota, USA · Qualcomm Flarion Technology, Inc., USA
Heaviside Low-Rank Support Matrix Machine
Feb 28, 2026Xianchao Xiu, Shenghao Sun, Xinrong Li +1Classification Robust Loss Functions
School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200400, China. · National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University, Shenyang, 110819, China. · Department of Mathematics and Statistics, Loyola University Maryland, Baltimore, MD 21210, USA.
Enabling clinical use of foundation models for computational pathology
Feb 25, 2026Audun L Henriksen, Ole-Johan Skrede, Lisa van der Schee +31Computational Pathology Pathology Foundation Models
Institute for Cancer Genetics and Informatics, Oslo University Hospital, Oslo, Norway · Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands · Department of Oncology, University of Oxford, Oxford, UK +20
Wasserstein Robust Classification Error: Surrogate Loss Misalignment and Distributional Attacks
Oct 11, 2025Bach C. Le, Tung V. Dao, Binh T. Nguyen +1Robust Classification Adversarial Robustness
College of Engineering & Computer Science, VinUniversity, Hanoi, Vietnam
DPG loss functions for learning parameter-to-solution maps by neural networks
Jun 23, 2025Pablo Cortés Castillo, Wolfgang Dahmen, Jay GopalakrishnanPDE Solving PDE Surrogate Modeling
Fragility-aware Classification for Risk Characterization and Robust Learning
Feb 18, 2025Zheng Cui, Daniel Zhuoyu Long, Jin Qi +2Robust Classification Robust Loss Functions
School of Management, Zhejiang University · Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong · Department of Industrial Engineering and Decision Analytics, Hong Kong University of Science and Technology +1

May 19, 2026J/K move · Enter open · S save

Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels

Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe Giguère

Department of Computer Science and Software Engineering Université Laval, Québec, Canada

Neural Network Robustness Noisy-Label Learning Robust Loss Functions Multiclass Classification Robust Classification

Open paper AskPDF arXiv

Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem. The symmetry condition provides theoretical guarantees for robustness to such noise. In this work, we study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component and a class-insensitive term. In particular, symmetrizing the cross-entropy loss leads to a linear multi-class extension of the unhinged loss. Unlike in the binary case, the multi-class version must have specific coefficients in order to satisfy the symmetry condition. Under suitable assumptions, we show that this multi-class unhinged loss is the unique convex multi-class symmetric loss. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi-class unhinged loss. We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error while allowing control of the beta-smoothness of the loss. Experiments on standard noisy-label benchmarks show competitive performance compared with existing robust loss functions.