Simpler and Stronger Baselines for All-in-One Image Restoration

Hongyu Du; Yuqi Wang; Wenlong Jiao; Wenwen Li; DongWei Ren

cs.CVDec 11, 2025

Simpler and Stronger Baselines for All-in-One Image Restoration

Authors: Hongyu Du, Yuqi Wang, Wenlong Jiao, Wenwen Li, DongWei Ren

Abstract

Current methods for all-in-one image restoration increasingly rely on complex mechanisms, such as Mixture-of-Experts or prompt learning, to handle diverse degradations.In this work, we challenge this trend by showing that structural symmetry provides a strong and effective inductive bias, which is often sufficient to support high-quality restoration in many practical settings. We propose SymUNet, a minimalist baseline that enforces strict encoder-decoder alignment via simple additive fusion, which outperforms complex state-of-the-art models on standard benchmarks while significantly reducing computational costs. Furthermore, to address the negative transfer observed in highly diverse degradation scenarios (e.g., 5-task settings), we introduce SE-SymUNet, which leverages frozen semantic priors (CLIP) to disentangle conflicting restoration tasks. Extensive experiments demonstrate that our approach establishes a simpler, stronger, and more efficient foundation for multi-degradation learning.

Figures & tables

Figure 1: An overview of the architectural philosophies under comparison. (a) The Asymmetric U-Net follows a decoder-heavy design widely used in Restormer [ 44 ] , PromptIR [ 30 ] , and DFPIR [ 34 ] . It adopts concatenation-based skip connections, leading to channel expansion and additional refinement blocks, resulting in a complex and indirect feature fusion path. (b) Our Symmetric U-Net (SymUNet) enforces a strict U-shaped design with shared structures across the encoder, decoder, and bottleneck. By using additive skip connections, it maintains consistent channel dimensions and enables a more direct and efficient flow of degradation-aware information.

Figure 2: An overview of our semantic enhanced baseline SE-SymUNet, built upon SymUNet. SE-SymUNet introduces a bidirectional semantic guidance module to incorporate CLIP priors. At each decoder stage, image features

\bm{f}

are refined by the semantic context

\bm{Z}

via the Semantic Guidance Module, followed by updating

\bm{Z}

using the refined features

\bm{f}^{\prime}

through the Semantic Refinement Module, forming an adaptive feedback loop. SE-Bottleneck shares the same structure as SE-Decoder.

Method	Dehazing	Deraining	Denoising on BSD68			Average
	SOTS-Outdoor	Rain100L	$\sigma=15$	$\sigma=25$	$\sigma=50$
Restormer * [ 44 ]	27.78/0.958	33.78/0.958	33.72/0.865	30.67/0.865	27.63/0.792	30.75/0.901
NAFNet * [ 6 ]	24.11/0.960	33.64/0.956	33.18/0.918	30.47/0.865	27.12/0.754	29.67/0.844
AirNet [ 17 ]	27.94/0.962	34.90/0.968	33.92/0.933	31.26/0.888	28.00/0.797	31.20/0.910
PromptIR [ 30 ]	30.58/0.974	36.37/0.972	33.98/0.933	31.31/0.888	28.06/0.799	32.06/0.913
InstructIR-3D [ 9 ]	30.22/0.959	37.98/0.978	34.15/0.933	31.52/0.890	28.30/0.804	32.43/0.913

Table 1: Quantitative comparison for the three-task all-in-one restoration benchmark. We report PSNR (dB) / SSIM. Best and second-best results are highlighted in bold and underline , respectively. * Results are cited from [ 12 ] , as these methods were not originally designed for the all-in-one restoration task.

Method	Dehazing	Deraining	Denoising	Deblurring	Low-Light	Average
	SOTS-Outdoor	Rain100L	BSD68 ( $\sigma=25$ )	GoPro	LOL
Restormer*	24.09/0.927	34.81/0.960	31.49/0.884	27.22/0.829	20.41/0.806	27.60/0.881
NAFNet*	25.23/0.939	35.56/0.967	31.02/0.883	26.53/0.808	20.49/0.809	27.76/0.881
AirNet*	21.04/0.884	32.98/0.951	30.91/0.882	24.35/0.781	18.18/0.735	25.49/0.846
PromptIR*	26.54/0.949	36.37/0.970	31.47/0.886	28.71/0.881	22.68/0.832	29.15/0.904
InstructIR-5D	27.10/0.956	36.84/0.973	31.40/0.887	29.40/0.886	23.00/0.836	29.55/0.907

Table 2: Quantitative comparison for the five-task all-in-one restoration benchmark. We report PSNR (dB) / SSIM. Best and second-best results are highlighted in bold and underline , respectively. * Results are cited from the survey paper [ 12 ] , as the original papers for these methods do not report on this specific all-in-one benchmark.

Figure 3: Visual comparison with state-of-the-art methods on the five-task all-in-one benchmark. From top to bottom: deraining, dehazing, and low-light enhancement. SE-SymUNet better removes complex degradations while preserving fine details, restores more natural color and contrast, and recovers details with fewer artifacts under extreme low-light conditions, demonstrating superior visual quality and robustness.

Architectural Symmetry
Configuration	PSNR	SSIM
(a) Asymmetric Baseline	32.55	0.919
(b) Symmetric Baseline (SymUNet)	32.93	0.921
Semantic Guidance
(c) Baseline (SymUNet)	32.93	0.921
(d) + Semantic Guidance (one-way)	33.02	0.921

Table 3: Ablation studies on architectural symmetry and semantic guidance.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: PSNR vs. FLOPs on the three-task benchmark. The bubble size corresponds to the total number of parameters. Our SymUNet has fewer parameters than all competing methods, with the sole exception of AirNet. In terms of performance, SymUNet resides in the optimal top-left quadrant (i.e., representing an optimal efficiency-performance balance), thus achieving state-of-the-art results while featuring substantially lower computational overhead.

Method	Params (M)	FLOPs (G)	PSNR (dB)
Restormer [ 44 ]	26.10	140.99	30.75
NAFNet [ 6 ]	17.06	15.97	29.67
AirNet [ 17 ]	5.68	301.27	31.20
PromptIR [ 30 ]	32.97	158.14	32.06
Perceive-IR [ 48 ]	-	-	32.63
VLU-Net [ 45 ]	123.00	171.07	32.70

Appendix

Table 4: Comparison of performance vs. complexity on the three-task benchmark. Our SymUNet offers unparalleled efficiency, while our SE-SymUNet achieves the highest performance with a highly competitive FLOP count.

Figure 5: Visualization of degradation-carrying features learned by our SymUNet.

Figure 6: Visual results on Rain100L dataset.

Figure 7: Visual results on SOTS dataset.

Figure 8: Visual results on CBSD68 dataset.

Figure 9: Visual results on GoPro dataset.

Figure 10: Visual results on LOL dataset.

All-in-One Image Restoration

All-in-one image restoration trains unified models to remove diverse and co-occurring degradations while preserving scene content.

See all 88 papers

Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration
Aug 7, 2026Zihao He, Yunfeng Wu, Xinchao Wang +1Image Corruption Robustness All-in-One Image Restoration
School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China · Alibaba Group, Beijing, China · School of Advanced Technology, Xi’an Jiaotong-Liverpool University, Suzhou, China +1
What to Remove, What to Preserve: Dual-Ambiguity Rectification for All-in-One Image Restoration
Jul 30, 2026Cencen Liu, Wen Yin, Dongyang Zhang +6Representation Learning Representation Disentanglement
University of Electronic Science and Technology of China · University of Electronic Science and Technology of China, Chengdu, Sichuan, China · Jiigan Technology +1
Universal Image Restoration via Internalized Chain-of-Thought Reasoning
Jun 16, 2026Yu Guo, Zhengru Fang, Shengfeng He +4All-in-One Image Restoration Image Restoration
Hong Kong JC Lab of Smart City and the Department of Computer Science, City University of Hong Kong · School of Computing and Information Systems, Singapore Management University · Computer Science and Information Engineering, National Taiwan University

Aug 7, 2026J/K move · Enter open · S save

Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration

Zihao He, Yunfeng Wu, Xinchao Wang, Songhua Liu

School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China · Alibaba Group, Beijing, China · School of Advanced Technology, Xi’an Jiaotong-Liverpool University, Suzhou, China+1

Image Corruption Robustness All-in-One Image Restoration Image Restoration Degradation-Aware Image Restoration

Open paper AskPDF arXiv

All-in-one image restoration seeks a single model that can recover images degraded by diverse and spatially non-uniform corruptions. However, many unified Transformers rely on fixed patch partitioning: task/degradation condition is injected only into the backbone blocks after tokenization, leaving the embedding and reconstruction stages insensitive to local degradation variations. In contrast to previous approaches, we present Flexible Image Transformer (FIT) that explicitly models degradation awareness across the entire pipeline, from patch sampling to pixel reconstruction. Specifically, FIT employs a lightweight Degradation Encoder to predict a global degradation vector

\mathbf{g}

and a spatial degradation map

\mathbf{M}

from local degradation severity, which jointly condition the patch embedding and unembedding through adaptive deformation. Moreover, to improve robustness across degradation types, we introduce a task-token dropout strategy that regularizes task conditioning during training. On five standard benchmarks (BSD68, Rain100L, SOTS, GoPro, and LOLv1), FIT achieves state-of-the-art performance with 30.72 dB average PSNR on the five-degradation setting and 32.83 dB on the three-degradation setting, outperforming recent unified restoration methods by +0.5

\sim

1.1 dB. Moreover, the learned offsets provide a direct handle for visualizing degradation-aware spatial adaptation.

Abstract

All-in-One Image Restoration

Explore similar work

Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration

What to Remove, What to Preserve: Dual-Ambiguity Rectification for All-in-One Image Restoration

Universal Image Restoration via Internalized Chain-of-Thought Reasoning