Overview

Adaptive loss weighting studies how to allocate training emphasis when examples, tasks, or objectives differ in difficulty, importance, noise, or learning progress. Common approaches use gradient norms or uncertainty, track loss and prediction dynamics, align gradients, or learn weights through meta-learning and bilevel optimization. These methods appear in settings such as physics-informed neural networks, multi-task learning, long-tailed classification, diffusion, language-model fine-tuning, forecasting, and federated learning. A central tension is that weights that improve one metric or objective can harm another, so stability, task-specific performance, and evaluation beyond a single weighted loss remain important.

Latest papers

CardsList

99 papers7 papers in the last four weeks, down 46% on the four weeks before. 0.1% of all new papers.

  1. Physics-informed neural networks in the small-data regime: A controlled comparison with classical PDE surrogates

    Nov 9, 2026Kirill Katsuba, Oleg Rudenko, Egor Bugaev +1PDE Surrogate ModelingPhysics-Informed ML

  2. Task-Contribution Dynamics of Adam in Physics-Informed Neural Networks: Slow-Manifold Analysis and Gradient Modulation

    Nov 6, 2026Enzhan Zheng, Xiaoliang SongNeural Network Training DynamicsAdaptive Gradient Methods

  3. When Scores Conflict with Preferences: Pre-Alignment Diagnosis and Adaptive Correction of Constraint Drift in DPO

    Nov 5, 2026Ping Liu, Yan YanDirect Preference OptimizationPreference Optimization

  4. Self-Privileged Fine-Tuning

    Nov 3, 2026Yangrong Hu, Xueyu Zhou, Yue Wu +1Token-Level Loss WeightingSupervised Fine-Tuning

  5. MIRA: Mutual Information guided calibration for Reliable Test-Time Adaptation

    Nov 1, 2026Hyeongyu Kim, Mingyeong Jang, Youngjun Song +2Test-Time AdaptationDistribution Shift Robustness

  6. Beyond Data Schedule: Fast LLM Pretraining on Data Mixtures by Loss-Reweighting

    Oct 23, 2026Jiazheng Li, Huaqing Zhang, Jing Xu +2Language Model PretrainingAdaptive Loss Weighting

  7. Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation

    Oct 20, 2026Guangyu Liu, Siyuan Gan, Shilong Zhou +3Token-Level Loss WeightingLanguage Model Distillation

  8. On the Stability and Convergence of Flow Map Training

    Oct 14, 2026Ishan Nath, Tom Stanic, Avishek Joey Bose +2Flow Map LearningNeural Network Training Dynamics

  9. MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation

    Oct 8, 2026Zipeng Wang, Xinpeng Dong, Yuefan Wang +6On-Policy DistillationAdaptive Loss Weighting

  10. DLC: A Metric-Guided Dynamic Loss Controller for Multi-Objective Training

    Oct 8, 2026Jaewan Ko, Janghoon ChoiAdaptive Loss WeightingImage Restoration

  11. Gen-PINNs: Generative Adversarial Physics Informed Neural Networks for solving partial differential equations

    Oct 7, 2026Muhammad M. Akmal, Kamy Sepehrnoori, Michael J. PyrczPDE SolvingPhysics-Informed ML

  12. How Much Evidence Should a Coding Agent's Self-Correction Carry? Adaptive Dirichlet Evidence for Self-Distillation

    Oct 6, 2026Yunbo Long, Guangya Hao, Yuhan Liu +4LLM Self-CorrectionCoding Agents

  13. Component-Level Evaluation of Adaptive PINN Training for CFD-Oriented Crystal Growth Simulation

    Oct 4, 2026Niruta Chapagain, Rohit Raj, Bertwin Kurisinkal Shine +1Neural PDE SolversPDE Surrogate Modeling

  14. Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning

    Oct 3, 2026Ilya Kuruzov, Dmitrii Vishovan, Kirill Novoselov +3Federated LearningFederated Learning Aggregation

  15. GAW-PO: Preference Optimization with Gradient-Aligned Token Weights

    Oct 1, 2026Andreea Dutulescu, Stefan Ruseti, Mihai Masala +2LLM AlignmentDirect Preference Optimization

  16. Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Features

    Sep 27, 2026Cunchun Li, Haonan He, Yifan Gao +4Supervised Fine-TuningAdaptive Loss Weighting

  17. Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL

    Sep 26, 2026Qinwei Ma, Jingzhe Shi, Simin Fan +2RL for Image GenerationAdaptive Loss Weighting

  18. Age of Learning: Temporal Persistence of Prediction Errors as a Learning Signal

    Sep 26, 2026Chenyang Wang, Stefan Forsström, Roger Olsson +2Adaptive Loss Weighting

  19. Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting

    Sep 25, 2026Paweł Mąka, Piotr Andruszkiewicz, Yusuf Can Semerci +2Multimodal MTMachine Translation

  20. FairMean: Promoting Fairness in Distributed Learning under Label Poisoning Attacks

    Sep 22, 2026Huigan Zheng, Jiaojiao Zhang, Yongxiang LiuAlgorithmic FairnessAdaptive Loss Weighting

  21. Adaptive Margin Ordinal Loss: Penalizing Center-Class Hedging in Ordinal Classification

    Sep 9, 2026Manisha KandelOrdinal RegressionAdaptive Loss Weighting

  22. TRaIL-Odom: Tightly Coupled Continuous Time Radar-IMU-LiDAR Odometry with Adaptive Doppler Weighting

    Sep 3, 2026Chiyun Noh, Turcan Tuna, William Talbot +3Adaptive Loss WeightingRadar-Inertial Odometry

  23. Information Density Imbalance in Visual Object Detection

    Sep 2, 2026Ziwei Zhao, Yanxi Lu, Yuwei Hu +8Object DetectionAdaptive Loss Weighting

  24. Online Self-Weighted Fine-Tuning

    Sep 1, 2026Haiquan Wen, Yiwei He, Bei Peng +1Supervised Fine-TuningLLM Fine-Tuning

  25. Error-Type-Aware Loss Reweighting for Robust Named Entity Recognition with Noisy LLM Labels

    Aug 31, 2026Elena Merdjanovska, Jonas Golde, Alan AkbikNamed Entity RecognitionLLM-Assisted Annotation

  26. Adaptive Physics-Informed Neural Networks for the Blasius Boundary-Layer Problem

    Aug 28, 2026Mehari Fentahun Endalew, Xiaoming John ZhangPhysics-Informed MLComputational Fluid Dynamics

  27. Tissue-Mixture Entropy-Weighted Reconstruction for Partial-Volume-Aware Brain MRI Super-Resolution

    Aug 27, 2026Xiao Tong, Wenyun Yang, Ziheng Zhang +3Medical Image Super-ResolutionBrain MRI