Planning Loss

Planning loss, a crucial aspect of training machine learning models, focuses on minimizing the discrepancy between predicted outcomes and actual results during the learning process. Current research investigates efficient loss functions, particularly for complex tasks like object detection and generative modeling, exploring techniques like ranking-based losses and their bucketed approximations to improve computational efficiency, as well as alternative loss functions that directly optimize for performance metrics. These advancements aim to enhance model training speed, accuracy, and robustness, impacting various applications from computer vision and natural language processing to physics-informed neural networks and reinforcement learning.

Papers

February 12, 2023

Flag Aggregator: Scalable Distributed Training under Failures and Augmented Losses using Convex Optimization
Hamidreza Almasi, Harsh Mishra, Balajee Vamanan, Sathya N. Ravi
Training Data Potential Scalability Convex Optimization Convergence Guarantee Subspace Learning Country Recognition Planning Loss Byzantine Robust Aggregation Byzantine Failure One Shot Aggregation

February 8, 2023

Cut your Losses with Squentropy
Like Hui, Mikhail Belkin, Stephen Wright
Cross Entropy Loss Planning Loss

February 7, 2023

Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy
Blake Woodworth, Konstantin Mishchenko, Francis Bach
Gradient Descent Stochastic Gradient Objective Function Gradient Computation Planning Loss Proximal Point Zero Cost Proxy Fast Optimization

January 27, 2023

Robust variance-regularized risk minimization with concomitant scaling
Matthew J. Holland
Multiplicative Size Scaling Interest Loss Planning Loss Variance Regularization Robust Mean Estimation Gradient Based Solver

January 25, 2023

On the inconsistency of separable losses for structured prediction
Caio Corro
Structured Prediction Structural Information Hard to Easy Inconsistency Planning Loss Bayes Rule Likelihood Loss

December 30, 2022

POMRL: No-Regret Learning-to-Plan with Increasing Horizons
Khimya Khetarpal, Claire Vernade, Brendan O'Donoghue, Satinder Singh, Tom Zahavy
Model Uncertainty Meta Reinforcement Learning Single Task Planning Loss New Horizon Planning Horizon

December 15, 2022

Using Two Losses and Two Datasets Simultaneously to Improve TempoWiC Accuracy
Mohammad Javad Pirhadi, Motahhare Mirzaei, Sauleh Eetemadi
Language Model Data Set Multiple Sens Planning Loss Sense Disambiguation

December 13, 2022

Losses over Labels: Weakly Supervised Learning via Direct Loss Construction
Dylan Sam, J. Zico Kolter
Pseudo Label Weak Supervision Weakly Supervised Learning Label Information Weak Label Planning Loss

November 7, 2022

Proper losses for discrete generative models
Rafael Frongillo, Dhamma Kimpara, Bo Waggoner
Generative Model Planning Loss Data Sampling Discrete Generative Model

November 2, 2022

Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing
Yonggan Fu, Yang Zhang, Kaizhi Qian, Zhifan Ye, Zhongzhi Yu, Cheng-I Lai, Yingyan Lin
Speech Representation Self Supervised Speech Representation Planning Loss Speech Supervised Learning Model Blessing Generation Device Automatic Speech Recognition Multilingual Application

October 4, 2022

Are All Losses Created Equal: A Neural Collapse Perspective
Jinxin Zhou, Chong You, Xiao Li, Kangning Liu, Sheng Liu, Qing Qu, Zhihui Zhu
Loss Function Neural Collapse Cross Entropy Planning Loss

September 1, 2022

The Geometry and Calculus of Losses
Robert C. Williamson, Zac Cranko
Loss Function Geometric Analysis Convex Set Statistical Learning First Integral Planning Loss Bayesian Decision

June 6, 2022

On the Convergence of Optimizing Persistent-Homology-Based Losses
Yikai Zhang, Jiachen Yao, Yusu Wang, Chao Chen
Loss Function Early Stage Convergence Persistent Homology Efficient Optimization Planning Loss Topological Loss Topology Aware

January 30, 2022

Do We Need to Penalize Variance of Losses for Learning with Label Noise?
Yexiong Lin, Yu Yao, Yuxuan Du, Jun Yu, Bo Han, Mingming Gong, Tongliang Liu
Strong Generalization Noisy Label Label Noise Variance Information Planning Loss Noise Transition Matrix

January 10, 2022

Wind Park Power Prediction: Attention-Based Graph Networks and Deep Learning to Capture Wake Losses
Lars Ødegaard Bentsen, Narada Dilp Warakagoda, Roy Stenbro, Paal Engelstad
Deep Learning Graph Attention Attention Network Graph Network Planning Loss Attention Graph Capturing Maneuver Bidirectional LSTM

December 6, 2021

Nonstochastic Bandits with Composite Anonymous Feedback
Nicolò Cesa-Bianchi, Tommaso Cesari, Roberto Colomboni, Claudio Gentile, Yishay Mansour
Planning Loss Reward Feedback

November 8, 2021

Losses, Dissonances, and Distortions
Pablo Samuel Castro
Natural Gradient Piano Performance Planning Loss Jazz Improvisation Distortion Realism Music Performance Cognitive Dissonance