DLC: A Metric-Guided Dynamic Loss Controller for Multi-Objective Training

Jaewan Ko; Janghoon Choi

cs.CVOct 8, 2026

DLC: A Metric-Guided Dynamic Loss Controller for Multi-Objective Training

Authors: Jaewan Ko, Janghoon Choi

Organizations: Major in Data Science Convergence, Graduate School of Data Science Kyungpook National University, Daegu, Republic of Korea

Abstract

In this paper, we introduce a metric-guided dynamic loss controller (DLC) for multi-objective image restoration. Conventional image restoration pipelines usually train with a fixed weighted combination of multiple losses, without changing the relative importance of fidelity, perceptual similarity, and no-reference quality during optimization. DLC is an architecture- and loss-term-agnostic training-time controller: it does not modify the restoration architecture or introduce new differentiable loss terms, but dynamically reweights the existing training losses. During training, DLC periodically evaluates the current model on a small fixed feedback subset and uses the resulting quality metrics to update the loss-weight vector through an LLM-based controller. Because DLC operates on existing loss terms rather than task-specific architectures, the same controller formulation can be instantiated across diverse image restoration training pipelines. We evaluate DLC on three restoration domains: low-light image enhancement, deraining, and real-world super-resolution, using both reference-based and no-reference quality metrics. Across these settings, DLC considers metric-dependent trade-offs during optimization and guides training toward balanced operating points across fidelity and perceptual quality. The results show that DLC can move models toward more favorable operating points across different restoration domains, supporting its role as a practical plug-in controller for multi-objective image restoration.

Figures & tables

Figure 1 : Conceptual illustration of the fidelity–perceptual trade-off in image restoration. Fidelity-oriented optimization tends to preserve reference consistency, while perceptual-oriented optimization emphasizes visual sharpness and detail. DLC aims to guide training toward a balanced operating point through metric-guided loss reweighting.

Figure 2 : Overview of DLC. At intervention step $k$ , the current restored output is evaluated by task-relevant metrics to construct a feedback state. The loss controller then produces the next loss-weight vector $\boldsymbol{\lambda}_{k+1}$ for the following training interval.

Domain	Controlled losses
Low-light Enhancement	pixel, perceptual, SSIM
Deraining	reconstruction, consistency, perceptual, adversarial
Real-world SR	L2, LPIPS, CSD

Table 1 : Domain-specific DLC instantiations.

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	NIQE $\downarrow$	CLIP-IQA $\uparrow$	BRISQUE $\downarrow$
LOL-v1	Base	23.60	0.8482	0.1231	4.5780	0.3920	23.0670
LOL-v1	DLC	23.50	0.8519	0.0974	4.4457	0.4630	19.9560
LOL-v2-Syn	Base	25.50	0.9362	0.1195	4.2735	0.5261	13.3023
LOL-v2-Syn	DLC	26.07	0.9450	0.0395	4.1369	0.5025	13.3490
LOL-v2-Real	Base	27.39	0.9007	0.0891	4.5415	0.4128	24.7530
LOL-v2-Real	DLC	30.74	0.9105	0.0622	4.5441	0.4711	24.2293

Table 2 : Low-light image enhancement results. DLC values are bolded when they improve over the baseline.

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	NIQE $\downarrow$	CLIP-IQA $\uparrow$	BRISQUE $\downarrow$
Rain100L	Base	33.2795	0.9537	0.0373	3.3761	0.7588	8.1748
Rain100L	DLC	33.5887	0.9526	0.0345	3.4173	0.7396	6.6552
RealRain1K-H	Base	29.6702	0.9280	0.1720	10.4063	0.2349	51.2430
RealRain1K-H	DLC	29.1054	0.9318	0.1513	10.6044	0.2746	51.6570

Table 3 : Deraining results. DLC values are bolded when they improve over the baseline.

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	CLIP-IQA $\uparrow$	NIQE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	FID $\downarrow$
DRealSR	Base	28.3197	0.7804	0.2960	0.2169	0.6970	6.1792	66.1088	0.6161	130.4879
DRealSR	DLC	29.1599	0.7974	0.2806	0.2124	0.6847	6.3306	64.7712	0.6166	130.1198
RealSR	Base	25.5031	0.7418	0.2672	0.2044	0.6699	5.5009	70.1478	0.6551	124.1799
RealSR	DLC	26.1454	0.7490	0.2582	0.2003	0.6652	5.8489	68.4462	0.6627	124.5460

Table 4 : Real-world super-resolution results. DLC values are bolded when they improve over the baseline.

Figure 3 : Qualitative comparison between the fixed-loss baseline and DLC across restoration domains. Zoomed crops highlight texture, structure, and perceptual differences.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	NIQE $\downarrow$	CLIP-IQA $\uparrow$	BRISQUE $\downarrow$
Fixed weight	23.11	0.8548	0.0926	4.4830	0.4580	21.0060
Rule-based	23.28	0.8485	0.0996	4.4940	0.4540	20.8500
GradNorm	22.81	0.8473	0.0934	4.3496	0.4720	19.6089
Uncertainty Weighting	23.02	0.8480	0.0946	4.3853	0.4548	19.9532
CoV Weighting	22.77	0.8482	0.0938	4.3791	0.4845	19.2890
DLC (Ours)	23.50	0.8519	0.0974	4.4457	0.4625	19.9560

Table 5 : Comparison of loss-weighting policies on LOL-v1. All methods use the same backbone, dataset, loss components, optimizer, and training budget, and differ only in the loss-weight decision policy. The best result for each metric is bolded.

Figure 4 : Qualitative comparison of loss-weighting policies on LOL-v1. The results illustrate the different visual characteristics induced by fixed, rule-based, and adaptive weighting strategies.

Controller	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	NIQE $\downarrow$	CLIP-IQA $\uparrow$	BRISQUE $\downarrow$
Qwen2.5-7B	23.2304	0.8505	0.0978	4.3938	0.4600	19.4955
Mistral-7B	23.2334	0.8505	0.0979	4.4092	0.4599	19.5009
LLaMA-3 8B	23.5045	0.8519	0.0974	4.4457	0.4625	19.9560

Table 6 : Controller LLM sensitivity on LOL-v1. Best values are bolded.

Figure 5 : Loss-weight trajectory and controller decision example on LOL-v1. Left: representative trajectories of the pixel, SSIM, and perceptual loss weights over intervention steps. Right: an example controller decision at a single intervention step, where fidelity-oriented metrics improve while perceptual or no-reference feedback degrades. DLC adjusts the next loss weights according to the observed metric trade-off. † The loss-weight vector is ordered as

(\lambda_{\mathrm{pix}},\lambda_{\mathrm{ssim}},\lambda_{\mathrm{perc}})

Domain	Per-call overhead	Total overhead	Relative overhead
Low-light enhancement	7.49–7.71 s	2.12–6.36 min	$<0.2\%$
Deraining	6.94–9.29 s	1.49–2.17 min	0.16–0.18%
Real-world SR	7.07 s	0.68 min	1.69%

Table 7 : Training-time overhead of DLC measured from representative runs. The reported overhead is incurred only during training, and the final model has no additional test-time cost.

Explore similar work

Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restoration
Aug 11, 2026Sangwoo Jo, Donggeun Ko, Jayeon Kang +3Flow Matching Perceptual Loss
Korea University, Seoul, South Korea · Aim Future, Seoul, South Korea
DRNet: All-in-One Image Restoration via Prior-Guided Dynamic Reparameterization
May 9, 2026Ao Li, Xiaoning Liu, Sheng Li +5All-in-One Image Restoration Image Restoration
School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, Sichuan, China · School of Communications and Information Engineering, Chongqing University of Posts and Telecommunication, Chongqing 400065, China
Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
Jul 28, 2026Jaeha Kim, Kyoung Mu LeeDiffusion Models for Image Restoration Low-Rank Adaptation
Dept. of ECE & ASRI, Seoul National University, Seoul, Korea · IPAI, Seoul National University, Seoul, Korea

Aug 11, 2026, cs.CVJ/K move · Enter open · S save

Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restoration

Sangwoo Jo, Donggeun Ko, Jayeon Kang, Youngsang Kwak+2

Korea University, Seoul, South Korea · Aim Future, Seoul, South Korea

Flow Matching Perceptual Loss Rate-Distortion Optimization Image Restoration Distortion-Perception Tradeoff

Open paper AskPDF arXiv

Image restoration is fundamentally constrained by the tradeoff between distortion and perception: minimizing pixel-wise error yields over-smoothed results, whereas optimizing for perceptual realism often introduces structural deviations. Recent approaches attempt to balance this tradeoff via posterior sampling or multi-stage generative pipelines, yet remain computationally expensive and architecturally complex. To overcome these limitations, we propose PCFlow (Perceptually Consistent Flow Matching), a unified framework that directly parameterizes a continuous transport from degraded observations to clean targets, jointly optimizing distortion and perceptual quality. While its latent consistency flow objective drives stable and efficient few-step inference, a Latent Consistency Perceptual Loss (LCPL) imposes semantic constraints directly on the guiding velocity field, steering the dynamics toward visually sharp data manifolds. Furthermore, recognizing the inherent conflict between structural and perceptual consistencies, we integrate a conflict-free gradient projection strategy to stabilize the multi-objective optimization landscape. Combined with lightweight, convolution-only backbone, PCFlow achieves competitive performance across diverse restoration tasks at a fraction of traditional computational costs.

Abstract

Explore similar work

Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restoration

DRNet: All-in-One Image Restoration via Prior-Guided Dynamic Reparameterization

Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors