Loss Function
Loss functions are crucial components of machine learning models, guiding the learning process by quantifying the difference between predicted and actual values. Current research emphasizes developing loss functions tailored to specific challenges, such as class imbalance in classification (addressed through asymmetric losses and hyperparameter distributions) and robustness to noise and outliers (using bounded and smooth alternatives to standard functions like mean squared error). These advancements improve model accuracy, efficiency, and generalizability across diverse applications, including medical image analysis, time series prediction, and physics-informed neural networks. The ongoing exploration of loss function design directly impacts the performance and reliability of machine learning models in various scientific and engineering domains.
Papers
Incorporating Expert Opinion on Observable Quantities into Statistical Models -- A General Framework
Philip Cooney, Arthur White
Evaluation of Data Augmentation and Loss Functions in Semantic Image Segmentation for Drilling Tool Wear Detection
Elke Schlager, Andreas Windisch, Lukas Hanna, Thomas Klünsner, Elias Jan Hagendorfer, Tamara Teppernegg
DOMINO: Domain-aware Loss for Deep Learning Calibration
Skylar E. Stolte, Kyle Volle, Aprinda Indahlastari, Alejandro Albizu, Adam J. Woods, Kevin Brink, Matthew Hale, Ruogu Fang
Understanding Self-Distillation in the Presence of Label Noise
Rudrajit Das, Sujay Sanghavi
Online Loss Function Learning
Christian Raymond, Qi Chen, Bing Xue, Mengjie Zhang
Refined Regret for Adversarial MDPs with Linear Function Approximation
Yan Dai, Haipeng Luo, Chen-Yu Wei, Julian Zimmert
Complex Critical Points of Deep Linear Neural Networks
Ayush Bharadwaj, Serkan Hoşten