Organizations: School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China · Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen 518057, China · State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau 999078, China · National-Local Joint Engineering Research Center of Biodiagnosis and Biotherapy, the Second Affiliated Hospital of Xi’an Jiaotong University, Xi’an 710006, China · Xi’an Jiaotong University, Xi’an 710049, China · College of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an 710064, China
Image Restoration (IR) agents, leveraging multimodal large language models to perceive degradation and invoke restoration tools, have shown promise in automating IR tasks. However, existing IR agents typically lack an insight summarization mechanism for past interactions, which results in an exhaustive search for the optimal IR tool. To address this limitation, we propose a portrait-aware IR agent, dubbed PaAgent, which incorporates a self-evolving portrait bank for IR tools and Retrieval-Augmented Generation (RAG) to select a suitable IR tool for input. Specifically, to construct and evolve the portrait bank, the PaAgent continuously enriches it by summarizing the characteristics of various IR tools with restored images, selected IR tools, and degraded images. In addition, the RAG is employed to select the optimal IR tool for the input image by retrieving relevant insights from the portrait bank. Furthermore, to enhance PaAgent's ability to perceive degradation in complex scenes, we propose a subjective-objective reinforcement learning strategy that considers both image quality scores and semantic insights in reward generation, which accurately provides the degradation information even under partial and non-uniform degradation. Extensive experiments across 8 IR benchmarks, covering six single-degradation and eight mixed-degradation scenarios, validate PaAgent's superiority in addressing complex IR tasks. Our project page is PaAgent.
Figures & tables
Fig. 1: Execution workflows of existing agent-based IR methods [ 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 ] . (a) One-step decision methods follow a rigid sequential pipeline, which is prone to error accumulation. (b) Multi-step decision methods introduce a reflection-based iterative mechanism for evaluating and refining restoration results.
Fig. 2: The illustration of quality score ambiguity. Despite possessing similar overall quality scores (calculated following [ 34 ] ), these two images exhibit distinct spatial degradation patterns.
Fig. 3: Overview of the proposed PaAgent architecture. (a) illustrates the entire workflow of PaAgent, which leverages Qwen3.5-9B [ 47 ] for degradation perception and task recommendation, followed by an RAG [ 38 ] module that queries the constructed tool portrait bank for optimal tool invocation. (b) depicts the evolution of the tool portrait bank, where interaction insights are summarized by Qwen3.5-Plus [ 47 ] and stored for future utilization. (c) shows the SORL strategy, which integrates MLLM’s insights and NR-IQA scores [ 40 , 41 , 42 , 43 , 44 , 45 , 46 ] via Qwen3.5-Plus [ 47 ] to generate reward signals, thereby driving the GRPO [ 39 ] algorithm to fine-tune Qwen3.5-9B. [ 47 ]
Fig. 4: The illustration of the RAG [ 38 ] process. It mainly consists of two phases: (1) offline knowledge base construction (upper stream), where the tool portrait bank is chunked, embedded via an embedding model, and stored in a vector database; and (2) online retrieval and reasoning (lower stream), where the query is embedded to retrieve relevant chunks through similarity search. The retrieved chunks are then integrated into prompt templates and fed into the QWen3.5-Plus [ 47 ] to select the IR tool.
Fig. 5: Visual comparison of different methods on snow (CSD [ 1 ] ) and noise (BSD68 [ 2 ] ) images. Our PaAgent yields better visual results on different degraded images, with outputs closer to the reference images.
Fig. 6: Visual comparison of different methods on haze (SOTS [ 3 ] ) and rainy (Rain13K [ 4 ] ) images. Our PaAgent yields better visual results on different degraded images, with outputs closer to the reference images.
Fig. 7: Visual comparison of different methods on low-light (LOL-V1 [ 5 ] ) and blur (GoPro [ 52 ] ) images. Our PaAgent yields better visual results on different degraded images, with outputs closer to the reference images.
Fig. 8: Visual comparison of different methods on composite degradation (LOL_Blur [ 50 ] and CDD-11 [ 7 ] ) images. Our PaAgent yields better visual results on different degraded images, with outputs closer to the reference images.
Desnowing
Denoising (BSD68 [ 2 ] )
CSD [ 1 ]
σ =5
σ =10
σ =15
σ =25
σ =35
σ =50
Method
Venue
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PromptIR [ 20 ]
NIPS’23
20.45
0.79
27.00
0.80
33.11
0.93
31.64
0.90
28.41
0.82
26.22
0.75
23.62
0.65
GridFormer [ 21 ]
IJCV’24
15.94
0.79
24.64
0.69
28.82
0.88
27.00
0.81
23.85
0.67
21.68
0.56
19.18
0.44
RAM [ 22 ]
ECCV’24
20.08
0.79
30.60
0.86
33.16
0.93
31.94
0.91
30.06
0.86
28.60
0.82
26.74
0.76
NDR-Restore [ 23 ]
TIP’24
20.06
0.78
27.38
0.79
32.29
0.91
31.98
0.90
29.29
0.84
27.67
0.79
23.78
0.65
TABLE I: Quantitative results of different methods on desnowing and denoising datasets. σ denotes the noise level The best results are marked in red.
Dehazing (SOTS [ 3 ] )
Deraining (Rain13K [ 4 ] )
Indoor
Outdoor
Rain100L
Rain100H
Test100
Test1200
Test2800
Method
Venue
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PromptIR [ 20 ]
NIPS’23
23.52
0.93
27.12
0.93
37.34
0.98
15.66
0.53
22.37
0.73
24.25
0.76
25.46
0.83
GridFormer [ 21 ]
IJCV’24
11.96
0.70
17.30
0.84
25.40
0.85
12.22
0.39
21.80
0.69
23.12
0.71
23.87
0.77
RAM [ 22 ]
ECCV’24
13.09
0.74
28.23
0.95
28.16
0.90
26.44
0.85
25.72
0.86
31.14
0.90
31.20
0.92
NDR-Restore [ 23 ]
TIP’24
17.27
0.85
27.12
0.95
36.58
0.98
14.63
0.49
22.17
0.72
23.28
0.73
25.29
0.84
TABLE II: Quantitative results of different methods on dehazing and deraining datasets. The best results are marked in red.
LLIE
Deblurring (GoPro [ 52 ] )
LLIE+Deblurring (LOL_Blur [ 50 ] )
LOL-V1 [ 5 ]
Blur_gamma
Blur
Low-light
Low-light+Blur
Method
Venue
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PromptIR [ 20 ]
NIPS’23
7.66
0.18
22.14
0.75
22.15
0.75
8.18
0.18
8.15
0.16
GridFormer [ 21 ]
IJCV’24
7.73
0.18
24.89
0.78
25.14
0.78
8.75
0.27
8.77
0.25
RAM [ 22 ]
ECCV’24
26.24
0.90
27.08
0.86
27.63
0.86
9.10
0.29
8.79
0.26
NDR-Restore [ 23 ]
TIP’24
7.61
0.17
23.72
0.77
23.84
0.77
8.48
0.23
8.45
0.21
TABLE III: Quantitative results of different methods on low-light image enhancement (LLIE) and deblurring datasets. Blur_gamma denotes blurred images with gamma correction. The best results are marked in red.
Single Degradation
Double Degradation
Triple Degradation
Haze (H)
Rain (R)
Snow (S)
Low-light (L)
H+R
H+S
L+H
L+R
L+S
L+H+R
L+H+S
Method
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PromptIR [ 20 ]
17.52
0.76
18.76
0.74
17.95
0.73
14.49
0.63
17.43
0.72
16.83
0.73
15.27
0.65
15.12
0.61
13.80
0.57
15.67
0.64
14.96
0.62
GridFormer [ 21 ]
17.32
0.78
18.92
0.77
18.83
0.77
14.62
0.64
18.21
0.79
16.79
0.76
15.67
0.68
14.85
0.60
13.34
0.60
15.95
0.66
15.51
0.67
RAM [ 22 ]
15.73
0.72
18.49
0.76
16.90
0.69
14.41
0.64
15.89
0.72
15.15
0.68
15.25
0.65
14.54
0.66
13.87
0.57
15.66
0.65
14.89
0.61
NDR-Restore [ 23 ]
17.02
0.74
18.40
0.73
17.53
0.72
14.43
0.62
17.19
0.72
16.17
0.72
15.16
0.63
15.12
0.61
13.78
0.57
15.58
0.63
14.85
0.60
TABLE IV: Quantitative results of different methods on the CDD-11 [ 7 ] dataset. The best results are marked in red.
Double Degradation
Triple Degradation
H+S
L+S
L+H+R
L+H+S
Method
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
w/ RS
19.33
0.71
18.27
0.70
17.57
0.68
17.39
0.65
w/ MS
23.36
0.74
22.94
0.72
21.58
0.70
20.09
0.68
w/ SFT
25.31
0.86
24.84
0.78
24.10
0.75
23.59
0.71
w/ SE
26.68
0.88
26.01
0.82
25.53
0.81
24.21
0.79
TABLE V: The average PSNR and SSIM scores of ablation study of the tool portrait bank and the subjective-objective reinforcement learning strategy on the CDD-11 [ 7 ] dataset. The best results are marked in red.
Fig. 9: Visualization results of ablation study on the tool portrait bank and the subjective-objective reinforcement learning strategy. In the zoomed-in areas, our full model yields better visual results on different degraded images, with outputs closer to the reference images.
Fig. 10: Ablation study on the input context of our proposed PaAgent. Incorrect and correct restoration task suggestions are marked in purple and red , respectively. Our full model consistently generates accurate restoration strategies across various degradation scenarios.
Fig. 11: Ablation study on the multi-step decision mechanism of our PaAgent. Our full model dynamically evaluates intermediate results and triggers additional restoration steps when necessary, thereby avoiding premature termination caused by the fixed restoration sequence.
Tsinghua University, Beijing, China · Data Science & Artificial Intelligence Research Institute, China Unicom, Beijing, China · Hunan University, Changsha, China +2
Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · School of Life Sciences, Tsinghua University, Beijing, China +1
School of Intelligent Systems Engineering Shenzhen Campus of Sun Yat-sen University, Guangdong, China · Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Shenzhen, China