Images under non-uniform coupled degradations require diverse restoration capabilities for different degraded regions. Existing methods employ globally uniform operations, potentially overlooking regional differences and causing incomplete restoration or even damage to image content. Moreover, a degraded region may require successive restoration actions, and earlier actions may alter residual degradation characteristics or introduce artifacts, potentially hindering subsequent restoration. To address these challenges, we propose RegionAgent, an adaptive restoration framework that trains a Vision-Language Model (VLM) through two-stage preference learning to generate a region-specific restoration action at each iteration. Specifically, we first train the VLM to select an action suited to the current region through step-level Direct Preference Optimization (DPO), using preferences derived from immediate restoration feedback to supervise only the differing component in each action pair. To account for the effects of current actions on subsequent restoration, we then apply trajectory-level DPO, using preferences derived from complete trajectory scores to supervise the sole differing action in each trajectory pair. Furthermore, as conventional image-pair datasets do not directly provide preferences over regional restoration decisions, we construct a regional restoration preference dataset comprising 10K action pairs and 20K trajectory pairs for two-stage VLM training. Experiments across multiple datasets demonstrate strong restoration performance under both mixed and single degradations. Ablation studies further validate the effectiveness of two-stage preference learning in improving restoration quality.
Figures & tables
Figure 1: Conceptual comparison of restoration paradigms. (a) Task-specific and all-in-one models restore the whole image. (b) Image-level agents select tools using iterative feedback. (c) RegionAgent selects regions and expert configurations for adaptive local refinement.
Figure 2: Overview of RegionAgent. (a) The Localizer proposes candidate regions, and the VLM selects restoration actions from the current image state. (b) The composed restorer processes the selected crop, and DSRF blends its residual correction into the image for subsequent refinement.
Figure 3: Supervision construction and policy optimization. (a) Action sampling, scoring, trajectory expansion, and filtering produce demonstrations and preference pairs. (b) Supervised fine-tuning and two-stage DPO train regional decisions.
Degradations
Method
Venue
PSNR ↑
SSIM ↑
LPIPS ↓
MANIQA ↑
CLIP-IQA ↑
MUSIQ ↑
Group A
AirNet ( 2022 )
CVPR
19.13
0.6019
0.4283
0.2581
0.3930
42.46
PromptIR ( 2023 )
NeurIPS
20.06
0.6088
0.4127
0.2633
0.4013
42.62
MiOIR ( 2024 )
arXiv
20.84
0.6558
0.3715
0.2451
0.3933
47.82
DA-CLIP ( 2024 )
ICLR
19.58
0.6032
0.4266
0.2418
0.4139
42.51
InstructIR ( 2024 )
ECCV
18.03
0.5751
0.4429
0.2660
0.3528
45.77
AdaIR ( 2025 )
ICLR
19.06
0.6061
0.4170
0.2718
0.4117
44.66
Table 1: Quantitative comparison on MiO100. Base is the shared restorer’s output before regional refinement. Best and second-best results per group are shown in bold and underlined .
Method
PSNR ↑
SSIM ↑
LPIPS ↓
CLIP-IQA ↑
MUSIQ ↑
MANIQA ↑
AirNet
25.65
0.8230
0.1820
0.4516
48.69
0.3163
PromptIR
27.04
0.8445
0.1515
0.4196
48.77
0.3169
AdaIR
26.48
0.7825
0.2604
0.4368
53.51
0.3456
DA-RCOT
26.08
0.7772
0.2579
0.4409
54.34
0.3461
LD
22.17
0.6922
0.1935
0.4253
54.94
0.3438
AutoDIR
27.81
0.8703
0.1283
0.4119
54.37
0.3053
Table 2: Average performance on single-degradation benchmarks. Best and second-best results are shown in bold and underlined , respectively. LD denotes latent diffusion.
Figure 4: Qualitative comparison on representative degraded images. Red boxes on the LQ inputs identify the enlarged regions shown for each method.
Variant
Restoration quality
Nref
PSNR ↑
SSIM ↑
LPIPS ↓
MUSIQ ↑
A
20.01
0.6264
0.3341
59.32
7.79
B
20.25
0.6389
0.3260
60.66
7.63
C
20.47
0.6469
0.3138
62.81
7.61
D
20.37
0.6437
0.3149
62.47
6.48
E
20.69
0.6544
0.3078
63.24
7.34
Table 3: Ablation studies on MiO100. We compare policy training strategies on Group C and expert composition, spatial masks, and Laplacian fusion on Group A.
Real-world image restoration is challenging due to complex and interacting mixed degradations. Recent agent-based approaches address this problem by composing multiple task-specific restoration tools. However, empirical analysis reveals that their performance is fundamentally limited by implicitly constrained planning spaces and the lack of coordination among independently pretrained tools. To address these issues, we propose OPERA (Optimized Planning-Execution Restoration Agent), a framework that jointly optimizes restoration planning and tool execution in an end-to-end manner. On the planning side, OPERA uses reinforcement learning to directly optimize tool composition over a combinatorial plan space, with the final restoration quality as the reward. On the execution side, OPERA introduces agent-guided co-training of restoration tools, enabling them to learn cooperative behaviors under sequential composition. Extensive experiments on multi-degradation benchmarks and real-world datasets demonstrate that OPERA consistently outperforms both all-in-one restoration models and existing agent-based methods across diverse and complex degradation scenarios.
Feng Zhu, Shuyang Xie, Yihan Zeng +2
Harbin Institute of Technology · Huawei Noah’s Ark Lab
Vision-language agents that orchestrate specialized tools for image restoration (IR) have emerged as a promising method, yet most existing frameworks operate in a training-free manner. They rely on heuristic task scheduling and exhaustive tool traversal, resulting in sub-optimal restoration paths and prohibitive computational cost. We argue that the core bottleneck lies in the absence of a learned policy to make decision, as a vision-language model cannot efficiently handle degradation-aware task ordering and tool composition. To this end, we propose TIR-Agent, a trainable image restoration agent that performs a direct tool-calling policy through a two-stage training pipeline of supervised fine-tuning (SFT) followed by reinforcement learning (RL). Two key designs underpin effective RL training: (i) a random perturbation strategy applied to the SFT data, which broadens the policy's exploration over task schedules and tool compositions, and (ii) a multi-dimensional adaptive reward mechanism that dynamically re-weights heterogeneous image quality metrics to mitigate reward hacking. To support high-throughput, asynchronous GPU-based tool invocation during training, we further develop a globally shared model-call pool. Experiments on both in-domain and out-of-domain degradations show that TIR-Agent outperforms 12 baselines, including 6 all-in-one models, 3 training-free agents, and 3 proprietary models, and achieves over 2.5× inference speedup by eliminating redundant tool executions.
Guoli Jia, Yisheng Zhang, Haote Hu +11
Tsinghua University, Beijing, China · Data Science & Artificial Intelligence Research Institute, China Unicom, Beijing, China · Hunan University, Changsha, China +2
Task-driven image restoration aims to improve both image quality and downstream task performance. However, existing methods predominantly focus on single degradation type and struggle to handle the diverse degradations encountered in real-world scenarios. Different degradations impose distinct restoration demands, and insufficient restoration may leave residual degradations and artifacts that impair object boundaries and semantic cues, thereby compromising downstream task performance. To address these challenges, we propose TaskIR, a two-stage task-driven unified image restoration framework that integrates degradation-adaptive restoration with task feedback refinement. In Stage I, a Degradation Representation Module (DRM) extracts degradation representations, enabling a Degradation-Guided Transformer Block (DGTB) to dynamically modulate feature transformations for adaptive restoration. In Stage II, a Task-to-Restoration Feedback Generation module (TRFG) transforms heterogeneous task features into restoration feedback by modeling task-representation discrepancies associated with the current restoration. Subsequently, a Selective Task Feedback Refinement module (STFR) assesses feedback relevance and selectively refines intermediate restoration features to mitigate interference with well-restored content. Extensive experiments demonstrate that TaskIR achieves competitive restoration quality and downstream task performance across diverse degradations and tasks.
Yanjie Tu, Qingsen Yan, Axi Niu +4
Northwestern Polytechnical University · Shenzhen Research Institute of Northwestern Polytechnical University · Xi’an University of Architecture and Technology