Adaptive Loss Weighting

Follow this topic Ask about this topic

Adaptive loss weighting studies how to allocate training emphasis when examples, tasks, or objectives differ in difficulty, importance, noise, or learning progress. Common approaches use gradient norms or uncertainty, track loss and prediction dynamics, align gradients, or learn weights through meta-learning and bilevel optimization. These methods appear in settings such as physics-informed neural networks, multi-task learning, long-tailed classification, diffusion, language-model fine-tuning, forecasting, and federated learning. A central tension is that weights that improve one metric or objective can harm another, so stability, task-specific performance, and evaluation beyond a single weighted loss remain important.

Latest papers

92 papers10 papers in the last four weeks, down 23% on the four weeks before. 0.1% of all new papers.

Quadratic Direct Forecast for Training Multi-Step Time-Series Forecast Models
Oct 28, 2025Hao Wang, Licheng Pan, Yuan Lu +7Time Series Forecasting Adaptive Loss Weighting
Xiaohongshu Inc. · State Key Lab of General AI, School of Intelligence Science and Technology, Peking University · College of Engineering, Purdue University +6
iCost: A Novel Instance-Complexity-Based Cost-Sensitive Learning Framework
Sep 19, 2024Asif Newaz, Asif Ur Rahman Adib, Taskeed JabidClass-Imbalanced Learning Cost-Sensitive Learning
Department of Electrical and Electronic Engineering, Islamic University of Technology (IUT), Gazipur, Bangladesh · Department of Electrical and Electronic Engineering, Ahsanullah University of Science and Technology (AUST), Dhaka, Bangladesh · Department of Computer Science and Engineering, East West University, Dhaka, Bangladesh

Oct 28, 2025J/K move · Enter open · S save

Quadratic Direct Forecast for Training Multi-Step Time-Series Forecast Models

Hao Wang, Licheng Pan, Yuan Lu, Zhichao Chen+6

Xiaohongshu Inc. · State Key Lab of General AI, School of Intelligence Science and Technology, Peking University · College of Engineering, Purdue University+6

Time Series Forecasting Adaptive Loss Weighting

Open paper AskPDF arXiv

The design of learning objectives is central to training time-series forecasting models. Existing learning objectives such as mean squared error mostly treat each future step as an independent, equally weighted task, which leads to the following two challenges: (1) they overlook the label autocorrelation effect among future steps, leading to biased learning objectives; (2) they fail to set heterogeneous task weights for different forecasting tasks corresponding to varying future steps, limiting the forecasting performance. To fill this gap, we propose a novel quadratic-form weighted learning objective, addressing both issues simultaneously. Specifically, the off-diagonal elements of the weighting matrix account for the label autocorrelation effect, whereas the non-uniform diagonals are expected to match the preferred weights of the forecasting tasks with varying future steps. On this basis, we propose a Quadratic Direct Forecast (QDF) learning algorithm, which trains the forecast model using the adaptively updated quadratic-form weighting matrix. Experiments show that our QDF effectively improves the performance of various forecast models, achieving state-of-the-art results. Code is available at https://github.com/Master-PLC/QDF.

Latest papers

Quadratic Direct Forecast for Training Multi-Step Time-Series Forecast Models

iCost: A Novel Instance-Complexity-Based Cost-Sensitive Learning Framework