Loss Function

Loss functions are crucial components of machine learning models, guiding the learning process by quantifying the difference between predicted and actual values. Current research emphasizes developing loss functions tailored to specific challenges, such as class imbalance in classification (addressed through asymmetric losses and hyperparameter distributions) and robustness to noise and outliers (using bounded and smooth alternatives to standard functions like mean squared error). These advancements improve model accuracy, efficiency, and generalizability across diverse applications, including medical image analysis, time series prediction, and physics-informed neural networks. The ongoing exploration of loss function design directly impacts the performance and reliability of machine learning models in various scientific and engineering domains.

Papers

December 8, 2021

The perils of being unhinged: On the accuracy of classifiers minimizing a noise-robust convex loss
Philip M. Long, Rocco A. Servedio
Loss Function Label Noise Simple Classifier Binary Classifier Convex Loss Function Noise Robust Loss

December 7, 2021

Understanding Square Loss in Training Overparametrized Neural Network Classifiers
Tianyang Hu, Jun Wang, Wenjia Wang, Zhenguo Li
Deep Learning Training Data Loss Function Cross Entropy Loss Neural Tangent Kernel Neural Network Classifier Square Loss

December 2, 2021

December 1, 2021

November 30, 2021

November 25, 2021

A Softmax-free Loss Function Based on Predefined Optimal-distribution of Latent Features for Deep Learning Classifier
Qiuyu Zhu, Xuewen Zu
Loss Function Latent Representation Latent Feature Deep Learning Classifier Softmax Loss

November 24, 2021

MIO : Mutual Information Optimization using Self-Supervised Binary Contrastive Learning
Siladittya Manna, Umapada Pal, Saumik Bhattacharya
Contrastive Learning Loss Function Mutual Information Self Supervised Contrastive Learning Contrastive Self Supervision

November 21, 2021

A Data-Driven Line Search Rule for Support Recovery in High-dimensional Data Analysis
Peili Li, Yuling Jiao, Xiliang Lu, Lican Kang
Loss Function High Dimensional Logistic Regression Regression Problem Line Search

November 18, 2021

Loss Functions for Discrete Contextual Pricing with Observational Data
Max Biggs, Ruijiang Gao, Wei Sun
Loss Function Observational Data Discrete Environment Demand Prediction Debiased Contrastive Contextual Dynamic Pricing Pricing Decision

November 13, 2021

Physics-guided Loss Functions Improve Deep Learning Performance in Inverse Scattering
Zicheng Liu, Mayank Roy, Dilip K. Prasad, Krishna Agarwal
Deep Learning Loss Function Near Field Inverse Scattering

November 12, 2021

November 9, 2021

Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search
Pengfei Xia, Ziqiang Li, Bin Li
Native Robustness Adversarial Example Loss Function Adversarial Robustness Adversarial Risk Approximation Error

November 5, 2021

Increasing Fairness in Predictions Using Bias Parity Score Based Loss Function Regularization
Bhanu Jain, Manfred Huber, Ramez Elmasri
Loss Function Human Prediction Procedural Fairness Bias Score Fairness Loss Recidivism Model

November 3, 2021

A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization
Renzhe Xu, Xingxuan Zhang, Zheyan Shen, Tong Zhang, Peng Cui
Loss Function Theoretical Analysis Covariate Shift Stable Learning Importance Weighting Covariate Shift Generalization

September 14, 2021

A Note on Knowledge Distillation Loss Function for Object Classification
Defang Chen
Knowledge Distillation Loss Function Short Note Object Classification Entropy Regularized Knowledge Distillation Loss

Loss Function

Papers

The perils of being unhinged: On the accuracy of classifiers minimizing a noise-robust convex loss

Understanding Square Loss in Training Overparametrized Neural Network Classifiers

Mixing Deep Learning and Multiple Criteria Optimization: An Application to Distributed Learning with Multiple Datasets

A higher order Minkowski loss for improved prediction ability of acoustic model in ASR

CLAWS: Contrastive Learning with hard Attention and Weak Supervision

Output-weighted and relative entropy loss functions for deep learning precursors of extreme events

The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration

LossPlot: A Better Way to Visualize Loss Landscapes

A Softmax-free Loss Function Based on Predefined Optimal-distribution of Latent Features for Deep Learning Classifier

MIO : Mutual Information Optimization using Self-Supervised Binary Contrastive Learning

A Data-Driven Line Search Rule for Support Recovery in High-dimensional Data Analysis

Loss Functions for Discrete Contextual Pricing with Observational Data

Physics-guided Loss Functions Improve Deep Learning Performance in Inverse Scattering

Impact of loss functions on the performance of a deep neural network designed to restore low-dose digital mammography

Distributed Sparse Regression via Penalization

Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search

Increasing Fairness in Predictions Using Bias Parity Score Based Loss Function Regularization

A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization

A Note on Knowledge Distillation Loss Function for Object Classification