Adaptive Loss Weighting

We propose a self-supervised physics-informed neural network (PINN) framework that adaptively balances physics-based and data-driven supervision for scientific machine learning under data scarcity. Unlike prior PINNs that rely on fixed or heuristic weighting of physics residuals and data loss, our approach introduces a learnable blending neuron that dynamically adjusts the relative contribution of each term based on their uncertainties. This mechanism enables stable training and improved generalization without manual tuning. To further enhance efficiency, we integrate a transfer learning strategy that reuses representations from related domains and adapts them to new physical systems with limited data. We validate the framework for the prediction of heat transfer in liquid-metal miniature heat sinks using only 87 CFD datapoints, where the adaptive PINN achieves an error <8%, outperforming shallow neural networks, kernel methods, and physics-only baselines. Our framework provides a general recipe for embedding physics adaptively into neural networks, offering a robust and reproducible approach for data-scarce problems across various scientific domains, including fluid dynamics and material modeling.

We study the training objectives of denoising-based generative models, with a particular focus on loss weighting and output parameterization, including noise-, clean image-, and velocity-based formulations. Through a systematic numerical study, we analyze how these training choices interact with the intrinsic dimensionality of the data manifold, model architecture, and dataset size. Our experiments span synthetic datasets with controlled geometry as well as image data, and compare training objectives using quantitative metrics for denoising accuracy (PSNR across noise levels) and generative quality (FID). Rather than proposing a new method, our goal is to disentangle the various factors that matter when training a flow matching model, in order to provide practical insights on design choices.

Medical image analysis depends on accurate segmentation and controllable synthesis, but both tasks face severe spatial imbalance: lesions occupy small regions against large backgrounds. We study adaptive spatial weighting as a task-level design principle and instantiate it in two adapters. LAW learns per-pixel loss weights for mask-conditioned diffusion by modulating a ratio prior with a feature-dependent delta map, with normalization, clamping, and Dice regularization for stability. ORDER improves lightweight segmentation by adding selective bidirectional skip attention with stage-wise confidence gating. On held-out diffusion test sets, LAW lowers FID from 158.13$\pm$0.15 to 108.43$\pm$0.71 on Polyps, from 144.13$\pm$0.31 to 89.51$\pm$0.96 on KiTS19, and from 139.22$\pm$0.38 to 112.58$\pm$0.68 on BRISC, while improving held-out mask-recovery Dice from 0.681$\pm$0.013 to 0.825$\pm$0.003 on Polyps. When the resulting images are added to nnUNet training, downstream Polyps mDice rises from 71.7$\pm$0.4 to 74.1$\pm$0.8. On the cleaned Polyps segmentation protocol, the reported ORDER configuration reaches 76.3$\pm$1.9 mDice and 67.2$\pm$2.0 mIoU at 42K parameters and 0.11 GFLOPs, versus 70.3$\pm$1.5 mDice and 59.9$\pm$1.7 mIoU for matched MK-UNet. On BRISC under the same training recipe, ORDER reaches 77.4$\pm$0.8 mDice and 68.1$\pm$0.7 mIoU. These results position adaptive spatial weighting as a practical design idea for both medical diffusion and efficient segmentation.

We study a persistent failure mode in multi-objective alignment for large language models (LLMs), in which scalarized training improves only some objectives while the others degrade. We formalize this phenomenon as cross-objective interference and, to our knowledge, conduct the first systematic study of scalarization algorithms for multi-objective LLM alignment. The study shows that interference is pervasive across algorithms yet strongly model-dependent. To understand how interference arises, we derive a local covariance law stating that an objective improves or degrades at first order according to the sign of the covariance between its reward and the scalarized score. We extend this law to the clipped surrogate objectives of modern reinforcement fine-tuning and show that it still holds under mild conditions. Building on this law, we propose COVariance-floor Enforced Reweighting (COVER), a one-sided controller that raises an objective's weight only when the covariance between its reward and the clipped advantage weight falls below a target. Through extensive experiments, we find that COVER can mitigate cross-objective interference while matching linear scalarization when objectives already co-improve. Finally, to explain why interference is model-dependent, we complement the local covariance law with a global convergence analysis. This analysis gives sufficient conditions for the non-convex scalarized objective to satisfy the Polyak--Łojasiewicz condition and relates interference to model geometry.

The design of learning objectives is central to training time-series forecasting models. Existing learning objectives such as mean squared error mostly treat each future step as an independent, equally weighted task, which leads to the following two challenges: (1) they overlook the label autocorrelation effect among future steps, leading to biased learning objectives; (2) they fail to set heterogeneous task weights for different forecasting tasks corresponding to varying future steps, limiting the forecasting performance. To fill this gap, we propose a novel quadratic-form weighted learning objective, addressing both issues simultaneously. Specifically, the off-diagonal elements of the weighting matrix account for the label autocorrelation effect, whereas the non-uniform diagonals are expected to match the preferred weights of the forecasting tasks with varying future steps. On this basis, we propose a Quadratic Direct Forecast (QDF) learning algorithm, which trains the forecast model using the adaptively updated quadratic-form weighting matrix. Experiments show that our QDF effectively improves the performance of various forecast models, achieving state-of-the-art results. Code is available at https://github.com/Master-PLC/QDF.

Class imbalance poses a significant challenge in classification tasks, often causing standard learning algorithms to become biased toward the majority class. Cost-sensitive learning (CSL) addresses this issue by assigning higher penalties to minority-class misclassifications. However, conventional CSL typically applies a uniform penalty to all minority-class instances, ignoring the fact that minority samples may differ substantially in terms of local safety, overlap, boundary ambiguity, and outlier-like behavior. Uniform penalization can therefore introduce undue bias, increasing the number of misclassifications. In this study, we propose iCost, an instance-complexity-aware CSL framework that assigns adaptive penalties to minority-class samples according to their estimated learning difficulty. This fine-grained penalization strategy ensures fairer weighting, reduces unwarranted bias, and improves overall classification performance. Two complementary complexity estimation strategies are introduced: Neighbor-iCost, based on local neighborhood composition, and Gini-iCost, based on Gini-impurity-based feature-space partitioning. Extensive experiments on 65 binary and 10 multiclass imbalanced datasets show that iCost outperforms conventional CSL by a clear margin and remains highly competitive with widely used resampling methods. To support reproducibility and practical adoption, the proposed algorithm has been released as a scikit-learn-compatible Python package through PyPI. This work offers a fresh perspective on imbalanced learning by integrating instance-level data complexity into the learning process, opening new avenues for developing adaptive, complexity-aware strategies for imbalanced classification.