cs.CVSep 18, 2026

SkillIR: Evolving Scene-Aware Skills for Agentic Image Restoration

Authors: Jie Shao, Shengkai Hu, Xu Zhang, Beihang Song, Yongcheng Jing, Xu Wu, Jun Wan

Organizations: Zhongnan University of Economics and Law, Wuhan, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper studies agentic image restoration, in which multimodal agents coordinate specialized restoration tools to recover images affected by complex degradations. Existing restoration agents often derive complete tool-use plans from the original degraded image or retrieve previously successful trajectories, providing limited support for adapting individual actions to evolving intermediate restoration states. We find that accepted tool executions can change the residual degradation state and, consequently, the applicability of subsequent tools. To address this issue, we propose SkillIR, a skill-guided framework that represents restoration experience as degradation-centered action evidence rather than complete tool-use trajectories. SkillIR consolidates context-dependent action outcomes into scene-aware restoration skills that characterize applicable conditions, expected effects, and attributable failure cases. Instead of prescribing a complete restoration plan, the retrieved skills guide one bounded action at a time within a verified residual-state loop: each tool output is treated as a candidate, committed only after transition verification, and followed by reassessment of the active residual degradations. After each rollout, the resulting evidence is used to create, refine, or patch dynamic skills, enabling accumulated restoration experience to improve decision-making for subsequent inputs. Experiments on synthetic and real-world multi-degradation datasets demonstrate that SkillIR improves restoration quality and enables more reliable and effective tool use.

Figures & tables

Figure 1: Motivation and conceptual comparison of SkillIR. (a) Cross-degradation side effects: executing a restoration action (e.g., brightening) may inadvertently introduce or exacerbate other degradations. (b) Action-level progress vs. degradation-level resolution: an accepted step often only mitigates degradation severity rather than eliminating it, leaving residual artifacts. (c) Conventional agent architectures perform task-level tool orchestration without fine-grained state verification, frequently leading to premature termination or sub-optimal plans. (d) SkillIR addresses these challenges by incorporating scene-aware skill retrieval and verifying residual states after each action, enabling flexible rollback and multi-step iterative restoration until full resolution.
Figure 2: Architectural overview of SkillIR. (a) Scene-Aware Skill Construction and Evolution: SkillIR decomposes verified restoration trajectories into degradation-centered episodes and consolidates them into scene-aware skills, while retaining rejected transitions as failure lessons. Dynamic skills are updated using verified evidence from completed rollouts. (b) Skill-Guided Verified Restoration: SkillIR retrieves skills matched to the current scene and residual state to guide one bounded action at a time, verifies each candidate before commitment, and re-perceives residual degradations for subsequent decisions.
DatasetTypeMethodVenuePSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowMANIQA ↑\uparrowCLIP-IQA ↑\uparrowMUSIQ ↑\uparrow
Group AAll-in-OneAirNet ( Li et al. 2022 )CVPR’2219.130.60190.42830.25810.3930042.46
PromptIR ( Potlapalli et al. 2023 )NeurIPS’2320.060.60880.41270.26330.4013042.62
MiOIR ( Kong et al. 2024 )arXiv’2420.840.65580.37150.24510.3933047.82
DA-CLIP ( Luo et al. 2024 )ICLR’2419.580.60320.42660.24180.4139042.51
InstructIR ( Conde et al. 2024 )ECCV’2418.030.57510.44290.26600.3528045.77
AutoDIR ( Jiang et al. 2024 )ECCV’2419.640.62860.39670.25000.3767047.01
Table 1: Quantitative comparison with previous state-of-the-art methods on the three multiple-degradation subsets of the MiO100 dataset. The best and second-best results for each metric are marked in bold and underline, respectively.
Figure 3: Qualitative comparison on representative samples from MiO100 Groups A–C. Each row shows the degraded input and an enlarged view of the marked region. PSNR values are reported in dB.
Figure 4: Qualitative comparison on the paired real-world benchmark.
MethodPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowMANIQA ↑\uparrowCLIP-IQA ↑\uparrowMUSIQ ↑\uparrow
AirNet23.30670.74710.44840.23560.327235.2690
PromptIR23.86470.75420.43410.24050.327035.4811
MiOIR24.00430.74720.42390.23940.350736.8458
DA-CLIP23.73130.74370.42730.25140.333036.4113
InstructIR26.14770.76390.42370.25560.341237.8236
AutoDIR20.74700.66400.47920.24660.314842.6789
Table 2: Quantitative comparison on the paired real-world dataset.
MethodPSNR ↑\uparrowLPIPS ↓\downarrowTime (min) ↓\downarrowTool Calls ↓\downarrow
AgenticIR20.550.30721.096.11
4KAgent20.950.30172.558.26
SEAR22.130.28901.988.15
SkillIR21.730.26551.635.84
Table 3: Efficiency comparison on Group B. Time denotes the average end-to-end inference time per image. Tool Calls counts all restoration-tool invocations, including candidates subsequently rejected by transition verification, while excluding MLLM and perception-model calls.
SettingPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowMANIQA ↑\uparrowCLIP-IQA ↑\uparrowMUSIQ ↑\uparrowCalls ↓\downarrow
Full SkillIR21.730.72280.26550.36750.550562.605.84
w/o Dynamic Skills21.080.70380.28970.35260.526760.026.34
w/o Static Skills20.980.70080.29480.35030.520559.536.15
Full-Trajectory Retrieval21.340.71180.27980.35940.535361.127.88
w/o Scene Context21.460.71470.27650.36090.539561.447.61
w/o Failure Lessons21.500.71630.27390.36280.541561.737.84
Table 4: Component ablation on Group B. Full-Trajectory Retrieval replaces degradation-centered episode representation with trajectory-level experience retrieval while keeping the remaining components unchanged.
Figure 5: Cross-sample skill evolution during deployment. Moving-average and block-wise trends of restoration quality and tool usage under different skill settings.
Figure 6: A representative trajectory with candidate rejection, degradation switching, and residual re-perception.
More in this topic

Agent-Based Image Restoration

See all 12 papers

Explore similar work

CardsList
Oct 14, 2026

Look Before You Restore: Learning to Predict Before Acting for Agentic Image Restoration

Agents for mixed-degradation image restoration typically learn what a tool does to an image only after running it: failed calls waste computation and harmful outputs go unnoticed until too late. We propose PreActIR, an agentic restoration framework that predicts tool consequences before execution. For each candidate tool, a tool-conditioned world model forecasts changes in residual degradation, image quality, and fidelity, together with the risk of harm and the uncertainty of the prediction. The agent executes the top-ranked candidate, and an independent verifier commits or rolls back its output before the agent replans from the observed state. PreActIR also audits whether performance is limited by the selector or by the candidate space, and studies targeted tool adaptation as a controlled way to expand that space. In a controlled evaluation that fixes the candidate pool, verifier, and budget and employs reference-annotated degradation states, PreActIR uses 42.5% fewer real tool calls and 63.4% fewer rejected calls than reactive, execute-then-evaluate trial and error, while improving Y-PSNR by 0.92 dB.

Xu Zhang, Jun Wan, Lefei Zhang +1

Wuhan University · Zhongnan University of Economics and Law

Agent-Based Image RestorationImage RestorationAI Agent Reliability
AskPDF
Mar 29, 2026

TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration

Vision-language agents that orchestrate specialized tools for image restoration (IR) have emerged as a promising method, yet most existing frameworks operate in a training-free manner. They rely on heuristic task scheduling and exhaustive tool traversal, resulting in sub-optimal restoration paths and prohibitive computational cost. We argue that the core bottleneck lies in the absence of a learned policy to make decision, as a vision-language model cannot efficiently handle degradation-aware task ordering and tool composition. To this end, we propose TIR-Agent, a trainable image restoration agent that performs a direct tool-calling policy through a two-stage training pipeline of supervised fine-tuning (SFT) followed by reinforcement learning (RL). Two key designs underpin effective RL training: (i) a random perturbation strategy applied to the SFT data, which broadens the policy's exploration over task schedules and tool compositions, and (ii) a multi-dimensional adaptive reward mechanism that dynamically re-weights heterogeneous image quality metrics to mitigate reward hacking. To support high-throughput, asynchronous GPU-based tool invocation during training, we further develop a globally shared model-call pool. Experiments on both in-domain and out-of-domain degradations show that TIR-Agent outperforms 12 baselines, including 6 all-in-one models, 3 training-free agents, and 3 proprietary models, and achieves over 2.5×\times inference speedup by eliminating redundant tool executions.

Guoli Jia, Yisheng Zhang, Haote Hu +11

Tsinghua University, Beijing, China · Data Science & Artificial Intelligence Research Institute, China Unicom, Beijing, China · Hunan University, Changsha, China +2

Tool-Using Vision-Language AgentsImage RestorationRL for VLMsAgent-Based Image Restoration
AskPDF
Mar 17, 2026

PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning

Image Restoration (IR) agents, leveraging multimodal large language models to perceive degradation and invoke restoration tools, have shown promise in automating IR tasks. However, existing IR agents typically lack an insight summarization mechanism for past interactions, which results in an exhaustive search for the optimal IR tool. To address this limitation, we propose a portrait-aware IR agent, dubbed PaAgent, which incorporates a self-evolving portrait bank for IR tools and Retrieval-Augmented Generation (RAG) to select a suitable IR tool for input. Specifically, to construct and evolve the portrait bank, the PaAgent continuously enriches it by summarizing the characteristics of various IR tools with restored images, selected IR tools, and degraded images. In addition, the RAG is employed to select the optimal IR tool for the input image by retrieving relevant insights from the portrait bank. Furthermore, to enhance PaAgent's ability to perceive degradation in complex scenes, we propose a subjective-objective reinforcement learning strategy that considers both image quality scores and semantic insights in reward generation, which accurately provides the degradation information even under partial and non-uniform degradation. Extensive experiments across 8 IR benchmarks, covering six single-degradation and eight mixed-degradation scenarios, validate PaAgent's superiority in addressing complex IR tasks. Our project page is PaAgent.

Yijian Wang, Qingsen Yan, Jiantao Zhou +2

School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China · Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen 518057, China · State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau 999078, China +3

Agent-Based Image RestorationImage RestorationTool-Using Vision-Language Agents
AskPDF