TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration

Yanjie Tu; Qingsen Yan; Axi Niu; Jiacong Tang

cs.CVJan 28, 2026

TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration

Authors: Yanjie Tu, Qingsen Yan, Axi Niu, Jiacong Tang

Organizations: School of Computer Science, Northwestern Polytechnical University, Xi’an, China · Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen, China.

Abstract

All-in-one image restoration aims to address diverse degradation types using a single unified model. Existing methods typically rely on degradation priors to guide restoration, yet often struggle to reconstruct content in severely degraded regions. Although recent works leverage semantic information to facilitate content generation, integrating it into the shallow layers of diffusion models often disrupts spatial structures (e.g., blurring artifacts). To address this issue, we propose a Triple-Prior Guided Diffusion (TPGDiff) network for unified image restoration. TPGDiff incorporates degradation priors throughout the diffusion trajectory, while introducing structural priors into shallow layers and semantic priors into deep layers, enabling hierarchical and complementary prior guidance for image reconstruction. Specifically, we leverage multi-source structural cues as structural priors to capture fine-grained details and guide shallow layers representations. To complement this design, we further develop a distillation-driven semantic extractor that yields robust semantic priors, ensuring reliable high-level guidance at deep layers even under severe degradations. Furthermore, a degradation extractor is employed to learn degradation-aware priors, enabling stage-adaptive control of the diffusion process across all timesteps. Extensive experiments on both single- and multi-degradation benchmarks demonstrate that TPGDiff achieves superior performance and generalization across diverse restoration scenarios. Our project page is: https://leoyjtu.github.io/tpgdiff-project.

Figures & tables

Figure 1: (a) Existing methods inject prior information uniformly into the diffusion model, whereas our approach adopts a hierarchical strategy, distributing distinct priors across specific layers of the network. (b) The generation results of diffusion models are largely governed by representations encoded in the deep layers of the network, which play a dominant role in determining the final reconstruction. (c) Visual comparison between the LQ inputs and our restoration results.

Figure 2: Overall architecture of TPGDiff. The framework explicitly models three types of priors from a low-quality input image and integrates them into a diffusion-based restoration network: (a) a semantic extractor that learns semantic representations via teacher-student distillation, (b) a degradation extractor that captures degradation-related characteristics, and (c) a structural adapter that injects structural priors into the diffusion model through an adapter module.

Figure 3: Design of the proposed structural extractor with multi-cue token aggregation. It aggregates heterogeneous tokens to learn unified structural representations for downstream image restoration tasks.

Figure 4: Visual ablation study on prior components. Each prior plays a distinct and indispensable role, where their synergy ensures both geometric integrity and semantic consistency.

Method	Low-light Enhancement (LOL-v2)				Method	NH Dehazing (NH-HAZE)				Method	Demoire (RDNet)
Method	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	Method	PSNR $\uparrow$	SSIM $\uparrow$	MUSIQ $\uparrow$	LPIPS $\downarrow$	Method	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$
Retinexformer	22.80	0.840	62.45	0.169	TransWeather	11.58	0.411	40.22	0.692	SwinIR	24.89	0.888	28.73	0.100
URetinex-Net	19.84	0.824	52.38	0.237	AutoDIR	12.71	0.477	46.29	0.612	RDNet	26.16	0.941	23.64	0.091
Restormer	20.77	0.851	57.04	0.115	DiffUIR	11.39	0.422	47.77	0.651	InstructIR	24.69	0.843	32.18	0.111
NAFNet	18.04	0.827	54.25	0.147	InstructIR	12.24	0.498	54.80	0.530	UniRestore	24.06	0.819	45.28	0.155
AirNet	19.69	0.821	55.43	0.151	AgenticIR	12.20	0.450	37.47	0.675	FoundIR	24.71	0.876	32.49	0.107

Table 1: Quantitative comparison between our method and other state-of-the-art approaches on nine different degradation-specific datasets.

\uparrow

represents the bigger the better, and

\downarrow

represents the smaller the better. Best and second-best results are highlighted in red and blue , respectively.

Method	POLED
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
HINet ( Chen et al., 2021a )	11.52	0.436	0.831
MPRNet ( Zamir et al., 2021 )	8.34	0.365	0.798
SwinIR ( Liang et al., 2021 )	6.89	0.301	0.852
DGUNet ( Mou et al., 2022 )	8.88	0.391	0.810
NAFNet ( Chen et al., 2022 )	10.83	0.416	0.794

Table 2: Comparison under unknown tasks setting on the POLED ( Zhou et al., 2021 ) dataset.

\uparrow

represents the bigger the better, and

\downarrow

represents the smaller the better. Best and second-best results are highlighted in red and blue , respectively.

Method	TOLED
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
HINet ( Chen et al., 2021a )	13.84	0.559	0.448
MPRNet ( Zamir et al., 2021 )	24.69	0.707	0.347
SwinIR ( Liang et al., 2021 )	17.72	0.661	0.419
DGUNet ( Mou et al., 2022 )	19.67	0.627	0.384
NAFNet ( Chen et al., 2022 )	26.89	0.774	0.346

Table 3: Comparison under unknown tasks setting on the TOLED ( Zhou et al., 2021 ) dataset.

\uparrow

represents the bigger the better, and

\downarrow

represents the smaller the better. Best and second-best results are highlighted in red and blue , respectively.

Method	Low-light		Deblurring		Deraining		Dehazing		Denoising		Average
Method	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
AirNet ( Li et al., 2022 )	18.18	0.735	24.35	0.781	32.98	0.951	21.04	0.884	30.91	0.882	25.49	0.846
IR-SDE ( Luo et al., 2023b )	20.07	0.780	26.34	0.800	34.12	0.951	24.56	0.940	30.89	0.865	27.20	0.867
PromptIR ( Potlapalli et al., 2023 )	22.89	0.829	27.93	0.851	36.17	0.970	30.41	0.972	31.20	0.885	29.72	0.901
DA-CLIP ( Luo et al., 2023a )	21.66	0.828	27.31	0.838	35.65	0.962	29.78	0.968	30.93	0.885	29.07	0.896
DiffUIR ( Zheng et al., 2024 )	22.32	0.826	27.50	0.845	35.98	0.968	29.47	0.965	31.02	0.885	29.25	0.898

Table 4: Quantitative comparison between our method and other state-of-the-art approaches on five restoration tasks.

\uparrow

represents the bigger the better. Best and second-best results are highlighted in red and blue , respectively.

Figure 5: Visual ablation study on structural priors. The restored patches and pixel-wise error maps show that removing different structural priors leads to distinct types of structural errors.

Figure 6: Visual comparison with other all-in-one image restoration methods on image denoising, low-light enhancement, image deraining, and image deblurring tasks. Zoom in for a better view.

Model	Method			Metrics
Model	Deg.	Sem.	Struc.	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
(a)	✗	✗	✗	31.63	0.912	0.060
(b)	✓	✗	✗	31.93	0.912	0.060
(c)	✓	✓	✗	32.61	0.920	0.049
Ours	✓	✓	✓	32.63	0.921	0.045

Table 5: Ablation study on prior types. We evaluate the impact of different prior types on guiding the recovery process.

Semantic Prior		Structural Prior		Metrics
Deep	Shallow	Deep	Shallow	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
✗	✗	✓	✓	31.73	0.912	0.061
✓	✓	✗	✗	32.38	0.918	0.051
✗	✓	✓	✗	32.18	0.915	0.052
✓	✗	✗	✓	32.63	0.921	0.045
✓	✓	✓	✓	31.51	0.910	0.061

Table 6: Ablation study on input positions. We compare the four combinations of injecting semantic and structural priors at deep versus shallow layers.

Model	Method			Metrics
Model	Depth	Seg	DoG	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
w/o Depth	✗	✓	✓	29.00	0.903	0.066
w/o Seg	✓	✗	✓	29.22	0.893	0.067
w/o DoG	✓	✓	✗	28.94	0.895	0.068
Ours	✓	✓	✓	29.46	0.905	0.066

Table 7: Ablation study on structural prior components. We evaluate the contribution of Depth, Seg, and DoG priors by removing each component separately.

Geometry-Guided Image Restoration

See all 16 papers

PixRestore: Unified Image Restoration via Pixel Diffusion Transformer
Aug 17, 2026Lingchen Sun, Rongyuan Wu, Xiangtao Kong +6All-in-One Image Restoration Degradation-Aware Image Restoration
The Hong Kong Polytechnic University · OPPO Research Institute
Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
Jul 28, 2026Jaeha Kim, Kyoung Mu LeeDiffusion Models for Image Restoration Low-Rank Adaptation
Dept. of ECE & ASRI, Seoul National University, Seoul, Korea · IPAI, Seoul National University, Seoul, Korea
DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration
Jun 17, 2026Yanjie Tu, Qingsen Yan, Axi Niu +3All-in-One Image Restoration Image Restoration
School of Computer Science, Northwestern Polytechnical University, Xi’an 710000, China · Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen 518057, China · State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau 999078, China

Aug 17, 2026J/K move · Enter open · S save

PixRestore: Unified Image Restoration via Pixel Diffusion Transformer

Lingchen Sun, Rongyuan Wu, Xiangtao Kong, Jixin Zhao+5

The Hong Kong Polytechnic University · OPPO Research Institute

All-in-One Image Restoration Degradation-Aware Image Restoration Pixel-Space Diffusion Models Diffusion Transformer Image Corruption Robustness

Open paper AskPDF arXiv

Unified image restoration (UIR) aims to recover high-quality (HQ) content from low-quality (LQ) images with different degradations using a single model. Most recent methods adapt large pretrained text-to-image (T2I) latent diffusion models for their strong capacity and generative priors. However, the variational autoencoder (VAE) in latent T2I models may discard restoration-sensitive details, while the open-ended synthesis prior can introduce content-inconsistent artifacts. We present PixRestore, a VAE-free pixel-space Diffusion Transformer (DiT) for UIR, where the diffusion backbone is trained entirely from scratch, without relying on T2I pretraining. PixRestore performs flow matching directly on patchified pixels, preserving fine-grained details while keeping the token sequence tractable. To adapt to different degradations, PixRestore learns to predict the reliability of layer features using LQ--HQ DINO feature similarity. Features from more reliable layers are fused as dense conditioning, while less reliable layers receive stronger HQ-feature supervision to encourage degradation removal. We train PixRestore on a large-scale corpus of diverse scenes and degradations, and further finetune it into a one-step generator using DINO-based adversarial objectives for efficient inference. Experiments on public benchmarks and real-world test sets show that, with only about 50M parameters and single-step inference, PixRestore achieves the best overall fidelity, perceptual quality, and robustness to degradations among competing UIR models while being far more efficient. Larger PixRestore variants can further boost performance, demonstrating the scalability of our pixel-space design. Code and the curated benchmark can be found at https://github.com/csslc/PixRestore.

Abstract

Geometry-Guided Image Restoration

Explore similar work

PixRestore: Unified Image Restoration via Pixel Diffusion Transformer

Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors

DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration