RecurIR: Learning When and How to Revise Restoration Policies via Recursive Self-Improvement
Organizations: Shanghai Jiao Tong University · Harbin Institute of Technology
Abstract
Agent-based image restoration has achieved promising performance by coordinating restoration tools to handle heterogeneous and coupled degradations. However, existing agents often improve their decisions through accumulated successful and failed executions, without explicitly identifying the conditions under which learned restoration strategies remain effective. Consequently, changes in degradation severity, spatial coverage, or interactions may invalidate previous decisions, while providing limited guidance on how these decisions should be revised. In this paper, we propose RecurIR, a recursive self-improvement framework that enables restoration agents to learn when and how to revise their decision rules. Specifically, we introduce a policy-hypothesis discovery mechanism to identify testable, condition-dependent rules through matched tool executions. We then design a rule-falsification-driven task evolution mechanism to generate targeted degradation variations, examine the validity of learned rules, and identify their applicability boundaries. Based on the resulting evidence, we develop a change-conditioned policy revision mechanism that learns how variations in degradation conditions should modify tool selection, execution order, operation strength, and processing regions. We further incorporate independent verification to retain effective revisions and use the resulting feedback to guide subsequent hypothesis discovery and task evolution. Through this recursive process, RecurIR progressively refines its restoration policies as degradation conditions change. Extensive experiments demonstrate the effectiveness of the proposed framework.
Figures & tables
| Type | Methods | Deraining | Desnowing | Dehazing | Deblurring | Denoising | Average | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | ||
| Models | Perceive-IR [ 64 ] | 30.93 | 0.902 | 32.99 | 0.948 | 40.05 | 0.996 | 29.79 | 0.889 | 33.81 | 0.962 | 33.51 | 0.940 |
| Defusion [ 39 ] | 30.66 | 0.905 | 33.33 | 0.952 | 40.21 | 0.995 | 29.98 | 0.889 | 33.73 | 0.962 | 33.58 | 0.940 | |
| AllRestorer [ 40 ] | 30.71 | 0.903 | 33.17 | 0.951 | 40.11 | 0.991 | 29.63 | 0.882 | 33.62 | 0.959 | 33.45 | 0.937 | |
| BaryIR [ 51 ] | 30.82 | 0.907 | 33.05 | 0.948 | 40.20 | 0.995 | 29.52 | 0.881 | 33.55 | 0.963 | 33.43 | 0.939 | |
| Agents | MAIR [ 28 ] | 31.23 | 0.921 | 33.97 | 0.957 | 41.48 | 0.996 | 34.08 | 0.965 | 34.02 | 0.971 | 34.96 | 0.962 |
| Methods | RealRain-1k-L | RTTS | SIDD | |||||
|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | LPIPS | FADE | BRISQUE | NIMA | PSNR | SSIM | |
| ECFNet [ 20 ] | 23.69 | 0.756 | 0.401 | 1.394 | 29.105 | 4.372 | 24.33 | 0.471 |
| Perceive-IR [ 64 ] | 27.31 | 0.901 | 0.372 | 1.264 | 23.694 | 4.613 | 24.53 | 0.497 |
| MAIR [ 28 ] | 27.53 | 0.907 | 0.335 | 1.190 | 23.231 | 4.874 | 24.68 | 0.500 |
| HybridAgent [ 30 ] | 27.65 | 0.909 | 0.351 | 1.198 | 23.280 | 4.811 | 24.70 | 0.509 |
| IAMAgent [ 54 ] | 27.61 | 0.912 | 0.344 | 1.203 | 23.284 | 4.887 | 24.66 | 0.498 |
| Methods | H + R + N | H + R | H + N | R + N | H | R | N |
|---|---|---|---|---|---|---|---|
| Perceive-IR [ 64 ] | 30.22/0.894 | 31.55/0.945 | 28.32/0.871 | 30.52/0.849 | 31.90/0.980 | 38.95/0.983 | 31.88/0.923 |
| DSwinIR [ 56 ] | 30.09/0.899 | 31.07/0.960 | 28.45/0.873 | 31.29/0.915 | 31.77/0.973 | 39.09/0.982 | 32.22/0.923 |
| MAIR [ 28 ] | 31.25/0.912 | 31.56/0.973 | 31.33/0.907 | 32.07/0.919 | 39.43 / 0.993 | 40.17/0.992 | 32.76/0.935 |
| AgenticIR [ 67 ] | 31.31/0.912 | 31.62/0.971 | 31.44/0.910 | 32.15/ 0.921 | 39.43/0.991 | 40.06/0.991 | 32.83/0.937 |
| HybridAgent [ 30 ] | 31.22/0.912 | 31.88 / 0.976 | 31.52/0.911 | 32.31 / 0.922 | 39.39/ 0.993 | 39.99/0.990 | 32.54/0.933 |
| IAMAgent [ 54 ] | 31.52/0.926 | 31.70/0.975 | 31.09/0.905 | 31.98/0.920 | 39.21/0.990 | 40.11/ 0.992 | 32.58/0.933 |
| Method | RR | SR | PAR |
|---|---|---|---|
| AgenticIR [ 67 ] | 0.32 | 0.84 | 0.88 |
| MAIR [ 28 ] | 0.36 | 0.90 | 0.80 |
| HybridAgent [ 30 ] | 0.38 | 0.88 | 0.82 |
| IAMAgent [ 54 ] | 0.32 | 0.90 | 0.86 |
| Restore-R1 [ 38 ] | 0.28 | 0.88 | 0.86 |
| RecurIR (Ours) | 0.12 | 0.94 | 0.91 |
| Method | PDM | RTEM | CPRM | RR | SR | PAR | PSNR |
|---|---|---|---|---|---|---|---|
| (a) | 0.35 | 0.84 | 0.79 | 34.88 | |||
| (b) | ✔ | 0.27 | 0.88 | 0.86 | 35.11 | ||
| (c) | ✔ | ✔ | 0.18 | 0.92 | 0.89 | 35.39 | |
| (d) | ✔ | ✔ | ✔ | 0.12 | 0.94 | 0.91 | 35.58 |
| Aspect | Existing Experience-Driven Agents | RecurIR (Ours) |
|---|---|---|
| Learned Knowledge | Historical restoration experiences and effective strategies | Condition-dependent relationships between degradation characteristics and restoration rules |
| Memory Content | Previously executed operations and their restoration outcomes | Conditions governing when restoration rules should be selected, retained, or revised |
| Decision Basis | Experience retrieval, restoration planning, and execution feedback | Conditional rule matching, applicability assessment, and policy revision |
| Condition Changes | Replanning based on previously acquired experiences | Learning and updating rule-selection relationships as degradation conditions change |
| Learning Objective | Improving restoration decisions through accumulated experiences | Learning when and how to select and revise restoration rules under different degradation conditions |
| Task | Tools |
|---|---|
| Derain | PPTformer [ 53 ] , EfDeRain+ [ 24 ] , MHNet [ 23 ] |
| Desnow | MSP-Former [ 5 ] , PW-FNet [ 27 ] ,PEUNet [ 25 ] |
| Dehaze | PGH 2 Net [ 49 ] , DEA-Net-CR [ 8 ] , ECFNet [ 20 ] |
| Deblur | Omni-Deblurring [ 34 ] ,ALGNet [ 18 ] , MDT [ 3 ] |
| Denoise | FSNet [ 11 ] , StarIR [ 14 ] , Restormer [ 58 ] |
| All-in-one | Perceive-IR [ 64 ] , BaryIR [ 51 ] , Defusion [ 39 ] |
| Module | Parameter | Initial Value |
|---|---|---|
| PDM | 0.5 | |
| 0.5 | ||
| RTEM | 0.10 | |
| 0.10 | ||
| 0.2 | ||
| 0.5 |
| Test100 | Test1200 | Rain100H | Rain100L | Average | ||||||
| Methods | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM |
| EfDeRain+ [ 24 ] | 31.10 | 0.911 | 33.12 | 0.925 | 34.57 | 0.957 | 39.03 | 0.972 | 34.46 | 0.941 |
| MAIR [ 28 ] | 31.23 | 0.921 | 33.28 | 0.927 | 34.40 | 0.955 | 39.99 | 0.984 | 34.73 | 0.947 |
| AgenticIR [ 67 ] | 31.10 | 0.920 | 33.31 | 0.928 | 34.52 | 0.956 | 39.71 | 0.980 | 34.66 | 0.946 |
| HybridAgent [ 30 ] | 31.56 | 0.923 | 33.39 | 0.926 | 34.28 | 0.956 | 39.72 | 0.981 | 34.74 | 0.947 |
| IAMAgent [ 54 ] | 31.55 | 0.922 | 33.43 | 0.925 | 34.55 | 0.951 | 39.41 | 0.979 | 34.74 | 0.944 |
| CSD | SRRS | Snow100K | ||||
| Methods | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM |
| PEUNet [ 25 ] | 37.28 | 0.97 | 31.89 | 0.98 | 34.11 | 0.96 |
| PW-FNet [ 27 ] | - | - | 32.74 | 0.98 | 34.50 | 0.95 |
| StarIR [ 14 ] | 38.57 | 0.99 | 32.75 | 0.98 | 34.46 | 0.96 |
| MAIR [ 28 ] | 38.41 | 0.98 | 32.70 | 0.97 | 33.97 | 0.96 |
| AgenticIR [ 67 ] | 38.55 | 0.98 | 32.73 | 0.97 | 34.17 | 0.96 |
| SOTS-Indoor | SOTS-Outdoor | |||
| Methods | PSNR | SSIM | PSNR | SSIM |
| PGH 2 Net [ 49 ] | 41.70 | 0.996 | 37.52 | 0.989 |
| MAIR [ 28 ] | 41.48 | 0.996 | 39.03 | 0.993 |
| AgenticIR [ 67 ] | 41.25 | 0.996 | 39.17 | 0.994 |
| HybridAgent [ 30 ] | 41.52 | 0.996 | 39.09 | 0.996 |
| IAMAgent [ 54 ] | 41.44 | 0.996 | 39.17 | 0.996 |
| GoPro | HIDE | |||
| Methods | PSNR | SSIM | PSNR | SSIM |
| StarIR [ 14 ] | 34.34 | 0.970 | 32.12 | 0.951 |
| MAIR [ 28 ] | 34.08 | 0.965 | 32.06 | 0.950 |
| AgenticIR [ 67 ] | 34.18 | 0.963 | 32.12 | 0.949 |
| HybridAgent [ 30 ] | 34.29 | 0.968 | 32.04 | 0.950 |
| IAMAgent [ 54 ] | 34.31 | 0.974 | 32.09 | 0.952 |
| CBSD68 | Kodak24 | Urban100 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Methods | 15 | 25 | 50 | 15 | 25 | 50 | 15 | 25 | 50 |
| SFNet [ 12 ] | 34.09 | 31.49 | 28.02 | 34.93 | 32.42 | 29.25 | 34.19 | 32.01 | 29.03 |
| FSNet [ 11 ] | 34.11 | 31.51 | 28.01 | 34.95 | 32.42 | 29.27 | 34.15 | 32.04 | 29.15 |
| Perceive-IR [ 64 ] | 34.38 | 31.74 | 28.53 | 34.84 | 32.50 | 29.16 | 34.86 | 32.55 | 29.42 |
| VLU-Net [ 60 ] | 34.35 | 31.72 | 28.46 | - | - | - | 34.92 | 32.71 | 29.61 |
| MAIR [ 28 ] | 34.02 | 31.69 | 28.51 | 34.91 | 32.45 | 29.22 | 34.87 | 32.70 | 29.49 |
| Case | Existing Experience-Guided Agent | RecurIR (Ours) |
|---|---|---|
| Case 1 Severity Change | Degradations: Noise and blur. Restoration strategy: Denoising Deblurring. Reasoning: Previous restoration experience suggests removing noise before deblurring. Therefore, the agent selects a denoising-then-deblurring sequence. | Degradation: The image contains moderate global blur and spatially varying noise. Noise is mild in most regions but severe in the dark facade and window areas . Noise and blur coexist near building edges . Restoration strategy: Strong regional denoising in severely noisy areas, mild denoising elsewhere, followed by global deblurring . Reasoning: The required denoising strength varies with local degradation severity. The agent selects different restoration settings to suppress noise while preserving details in less degraded regions. |
| Case 2 Region Change | Degradations: Rain and haze. Restoration strategy: Dehazing Deraining. Reasoning: Previous experience indicates that haze removal can improve visibility before deraining. The agent therefore follows a dehazing-then-deraining sequence. | Degradation: The image contains moderate global haze , while rain streaks are concentrated in the left and upper regions . The distant background is mainly affected by haze, whereas the left foreground contains coupled rain and haze . Restoration strategy: Global dehazing followed by regional deraining restricted to rain-affected areas. Reasoning: Haze removal is required across the scene, whereas deraining is only needed in specific regions. The agent adjusts the processing scope to avoid unnecessary restoration in unaffected areas. |
| Case 3 Mixed Degradation Change | Degradations: Haze, noise, and rain. Restoration strategy: Denoising Dehazing Deraining. Reasoning: Based on previous experience, the agent removes noise first, improves visibility through dehazing, and then removes rain streaks. | Degradation: The image contains mild global noise , moderate haze , and localized rain in the left road area . The distant background is dominated by haze, while the dark corner regions contain coupled noise and haze . The left road area contains coupled rain and haze . Restoration strategy: Mild global denoising , stronger dehazing in distant regions , and regional deraining in rain-affected areas. Reasoning: Different regions contain different degradation combinations and severities. The agent selects restoration operations and processing regions according to their local degradation conditions. |