Ultra-High-Definition Restoration Transformers with Correlation Matching Transformation

Cong Wang; Liyan Wang; Jinshan Pan; Wei Wang; Wenqi Ren; Jun Liu; Xiaochun Cao

cs.CVAug 18, 2026

Ultra-High-Definition Restoration Transformers with Correlation Matching Transformation

Authors: Cong Wang, Liyan Wang, Jinshan Pan, Wei Wang, Wenqi Ren, Jun Liu, Xiaochun Cao

Organizations: School of Computing and Communications, Lancaster University

Abstract

We propose UHDformer++, a general Transformer-based framework to solve numerous Ultra-High-Definition (UHD) image restoration tasks. UHDformer++ operates across $4$ coordinated learning spaces: 1) a high-resolution space (HR) for multi-level feature extraction, 2) a low-resolution space (LR) for learning compact, representative features, 3) a super-resolution space (SR) for upsampling low-resolution features from SR, and 4) a low-high fusion and reconstruction space (LHFR) for final image restoration. Specifically, HR extracts multi-scale high-resolution features and fuses them with low-resolution cues to produce residual images, while LR distills complementary representations from HR to improve restoration quality. To supply LHFR with richer features, SR super-resolves LR outputs before fusion. We further introduce two modules to bridge the high- and low-resolution spaces. The Feature-Refined Correlation Matching Transformation (FR-CMT) module selects the top $C/r~(C~\text{denotes the number of channels;~}r\geq1~\text{controls the squeezing level})$ , from the fusion between max- and mean-pooled high-resolution features to replace less informative channels in the low-resolution Transformer. The Adaptive Channel Modulator (ACM) adaptively recalibrates multi-scale high-resolution features, ensuring that only task-relevant information propagates to LR. Extensive experiments demonstrate that UHDformer++ reduces model parameters by at least 86% compared with recent state-of-the-art methods while achieving substantial performance gains across $5$ UHD restoration tasks, including low-light image enhancement, dehazing, deblurring, deraining, and desnowing. Code will be released at https://github.com/supersupercong/uhdformerplus.

Figures & tables

Panel of Figure 1 — Fig. 1: Challenging Ultra-High-Definition image restoration examples. Top : Low-light image enhancement; Middle : Image dehazing; Bottom : Image deblurring. Compared with recent state-of-the-art approaches UHDFour [ 2 ] for low-light image enhancement, DehazeFormer [ 3 ] for image dehazing, and FFTformer [ 4 ] for image deblurring, our proposed UHDformer++ is able to recover clear results with finer structures and more natural colors while consuming only $0.3962$ M parameters with about $50\times$ fewer parameters than state-of-the-art methods (see Tabs. II - IV ) due to the simple and effective designs introduced in this paper.

Panel of Figure 2 — Fig. 2: Parameters-performance trade-off. Compared with recent state-of-the-art methods UHDFour [ 2 ] for low-light image enhancement, DehazeFormer-B [ 3 ] for image dehazing, and FFTformer [ 4 ] for image deblurring, our UHDformer++ respectively reduces at least $98\%$ , $86\%$ , and $97\%$ model sizes while obtaining state-of-the-art UHD image restoration performance under different training sets, significantly improving the trade-off on these $3$ UHD image restoration tasks.

Panel of Figure 3 — Fig. 3: Overall framework of the proposed UHDformer++ . It mainly contains $4$ spaces: (a) learning in high-resolution space (HR), (b) learning in low-resolution space (LR), (c) super-resolution space (SR), and (d) Low-high fusion and reconstruction space (LHFR). The HR explores multi-level high-resolution features and fuses low-high features and reconstructs the residual images, while the LR learns the low-resolution features via the Feature-Refined Correlation Matching Transformation Transformer Block, as shown in Fig. 17 . Before transforming high-resolution features to low-resolution space, we use an Adaptive Channel Modulator to adaptively modulate multi-level high-resolution features to provide more representative content to low-resolution space. Finally, we super-resolve the low-resolution features from LR by SR, which is further fused and reconstructed for the final reconstruction via LHFR.

Panel of Figure 4 — Fig. 4: (a) Feature-Refined Correlation Matching Transformation (FR-CMT) and (b) Correlation Matching Transformation (CMT) . FR-CMT contains a fusion of max-pooling and mean-pooling operations, followed by CMT, convolutions, and a Gated Feature Refinement (GFR) module. CMT selects the top $C/r\penalty\ (C\penalty\ \text{denotes the number of channels;\penalty\ }r\geq 1\penalty\ \text{is the squeezing factor which controls the squeezing level})$ channels from the pooling high-resolution features to replace low-resolution features. GFR is used to refine the features for better representation.

Fig. 5: Correlation Matching Transformation Attention .

Fig. 6: Correlation Matching Transformation Feed-Forward Network.

Panel of Figure 7 — Fig. 7: Adaptive Channel Modulator .

Dataset	Training samples	Testing samples	Resolution
UHD-LL	2,000	150	3,840 $\times$ 2,160
UHD-Haze	2,290	231	3,840 $\times$ 2,160
UHD-Blur	1,964	300	3,840 $\times$ 2,160
UHD-Rain	3,000	200	3,840 $\times$ 2,160
UHD-Snow	3,000	200	3,840 $\times$ 2,160

TABLE I: Datasets Statistics .

Method	Metrics						Computational Complexity			Full Size
	Full Reference			No-Reference			Computational Complexity
	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PI $\downarrow$	NIQE $\downarrow$	MIUSIQ $\uparrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$	Time (s) $\downarrow$
Training Set on LOL
SwinIR [ 15 ]	17.900	0.7379	0.5217	6.5902	8.5836	28.3047	11.5	11992.17	1.41	✗
Restormer [ 49 ]	19.728	0.7703	0.4566	6.5099	8.3787	30.2802	26.1	2255.85	1.27	✗
Uformer [ 16 ]	18.168	0.7201	0.5593	7.1781	9.8494	29.5157	20.6	657.45	0.43	✗

TABLE II: Low-light image enhancement . The best and the second results are marked in bold and underline , respectively. Full Size means that if the method can directly infer a full UHD image on one 3090 GPU.

Panel of Figure 10 — Fig. 8: Low-light image enhancement on UHD-LL trained on the LOL dataset [ 62 ] .

Panel of Figure 11 — Fig. 9: Low-light image enhancement on UHD-LL trained on the UHD-LL dataset .

Method	Metrics						Computational Complexity			Full Size
	Full Reference			No-Reference			Computational Complexity
	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PI $\downarrow$	NIQE $\downarrow$	MIUSIQ $\uparrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$	Time (s) $\downarrow$
Training Set on SOTS-ITS
GridNet [ 66 ]	14.783	0.8466	0.2518	4.4077	4.0874	35.6738	0.96	343.92	0.10	✔
MSBDN [ 67 ]	15.043	0.8570	0.2709	4.5582	4.4640	35.5302	31.4	664.87	0.09	✔
Restormer [ 49 ]	13.875	0.6405	0.4477	5.2136	6.1129	35.4127	26.1	2255.85	1.27	✗

TABLE III: Image dehazing .

Panel of Figure 13 — Fig. 10: Image dehazing on UHD-Haze trained on the SOTS-ITS dataset [ 63 ] .

Panel of Figure 14 — Fig. 11: Image dehazing on UHD-Haze trained on the UHD-Haze dataset .

Method	Metrics						Computational Complexity			Full Size
	Full Reference			No-Reference			Computational Complexity
	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PI $\downarrow$	NIQE $\downarrow$	MIUSIQ $\uparrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$	Time (s) $\downarrow$
Training Set on GoPro
DMPHN [ 78 ]	26.490	0.7985	0.2696	6.1592	6.0393	26.1362	21.7	1251.76	0.15	✔
MIMO-Unet++ [ 48 ]	24.290	0.7354	0.3723	6.0395	6.1247	25.3302	16.1	2470.52	0.31	✔
MPRNet [ 14 ]	24.571	0.7426	0.3597	6.9655	7.9493	27.6944	20.1	27317.71	1.13	✔

TABLE IV: Image deblurring .

Panel of Figure 16 — Fig. 12: Image deblurring on UHD-Blur trained on the GoPro dataset [ 64 ] .

Panel of Figure 17 — Fig. 13: Image deblurring on UHD-Blur trained on the UHD-Blur dataset .

Method	Metrics						Computational Complexity			Full Size
	Full Reference			No-Reference			Computational Complexity
	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PI $\downarrow$	NIQE $\downarrow$	MIUSIQ $\uparrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$	Time (s) $\downarrow$
Restormer [ 49 ]	19.408	0.7105	0.4775	8.8409	11.2731	24.9277	26.1	2255.85	1.27	✗
Uformer [ 16 ]	19.494	0.7163	0.4598	8.0579	9.9422	25.0728	20.6	657.45	0.43	✗
SFNet [ 79 ]	20.091	0.7092	0.4768	7.1914	8.0167	22.3553	13.23	1991.03	0.42	✗
UHDformer [ 32 ]	37.348	0.9748	0.0554	5.0464	4.9256	30.4390	0.3393	51.63	0.16	✔

TABLE V: Image deraining on UHD-Rain dataset .

Panel of Figure 19 — Fig. 14: Image deraining on UHD-Rain trained on the UHD-Rain dataset .

Method	Metrics						Computational Complexity			Full Size
	Full Reference			No-Reference			Computational Complexity
	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PI $\downarrow$	NIQE $\downarrow$	MIUSIQ $\uparrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$	Time (s) $\downarrow$
Restormer [ 49 ]	24.142	0.8691	0.3190	8.0490	9.4711	26.0491	26.1	2255.85	1.27	✗
Uformer [ 16 ]	23.717	0.8711	0.3095	7.5036	8.5964	26.1323	20.6	657.45	0.43	✗
SFNet [ 79 ]	23.638	0.8456	0.3528	7.1527	7.3665	22.5295	13.23	1991.03	0.42	✗
UHDformer [ 32 ]	36.614	0.9881	0.0245	5.0046	4.7680	31.1335	0.3393	51.63	0.16	✔

TABLE VI: Image desnowing on UHD-Snow dataset .

Panel of Figure 21 — Fig. 15: Image desnowing on UHD-Snow trained on the UHD-Snow dataset .

ID	Experiment	Metrics			Computational Complexity
ID	Experiment	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$
(a)	w/o LN	26.812	0.9279	0.2369	0.3955	49.65
(b)	LN in UHDformer [ 32 ]	27.119	0.9277	0.2493	0.3962	49.65
(c)	DyT in UHDformer++	27.256	0.9308	0.2304	0.3962	49.65
(d)	GDFN in Restormer [ 49 ]	26.499	0.9270	0.2366	0.3064	38.06
(e)	CMTN in UHDformer [ 32 ]	27.090	0.9290	0.2394	0.3379	48.70

TABLE VII: Effect on the Normalization Layer (DyT), Feed-Forward Neural Network (CMTFN), and FR-CMT in the UHDformer++ . Each component used in this paper is more effective than our previous version used in UHDformer [ 32 ] .

Panel of Figure 23 — Fig. 16: Effect on different basic components.

ID	Experiment	Metrics			Computational Complexity
ID	Experiment	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$
(a)	w/o FR-CMT in CMTA & CMTN	26.933	0.9279	0.2433	0.3060	27.94
(b)	w/o FR-CMT in CMTA	26.879	0.9278	0.2406	0.3511	38.8
(c)	w/o FR-CMT in CMTFN	26.650	0.9256	0.2347	0.3511	38.8
(d)	w/o Using Max-Pooling in FR-CMT	27.079	0.9270	0.2406	0.3962	49.45
(e)	w/o Using Mean-Pooling in FR-CMT	26.911	0.9281	0.2469	0.3962	49.45

TABLE VIII: Ablation study on Feature-Refined Correlation Matching Transformation (FR-CMT) .

Panel of Figure 25 — Fig. 17: Visual effect on Feature-Refined Correlation Matching Transformation (FR-CMT).

Panel of Figure 26 — Fig. 18: Feature visualization on correlation matching transformation. Please zoom in on a high-resolution display.

$r$	1	2	4 ( Default )	8
PSNR (dB) $\uparrow$	26.7669	27.3854	27.2567	27.1235
SSIM $\uparrow$	0.9271	0.9288	0.9308	0.9278
LPIPS $\downarrow$	0.2376	0.2383	0.2304	0.2345
Params. (M) $\downarrow$	0.3962	0.3962	0.3962	0.3962
FLOPs (G) $\downarrow$	49.65	49.65	49.65	49.65

TABLE IX: Effect on Squeezing Factor

r

in Feature-Refined Correlation Matching Transformation.

ID	Experiment	Metrics			Computational Complexity
ID	Experiment	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$
(a)	w/o Adaptive Channel Modulator (ACM)	26.953	0.9289	0.2357	0.3953	48.65
(b)	w/ Efficient Channel Attention (ECA) [ 82 ]	26.750	0.9278	0.2407	0.3953	48.7
(c)	w/ Squeeze-and-Excitation Attention (SE) [ 83 ]	27.008	0.9286	0.2355	0.3969	48.7
(d)	w/ Adaptive Channel Modulator (ACM) ( Ours )	27.256	0.9308	0.2304	0.3962	49.65

TABLE X: Ablation study on Adaptive Channel Modulator (ACM).

Panel of Figure 29 — Fig. 19: Visual effect on Adaptive Channel Modulator (ACM).

ID	Experiment	Metrics			Computational Complexity
ID	Experiment	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$
(a)	Individual Adaptive Channel Modulator	26.981	0.9281	0.2404	0.4062	60.07
(b)	Shared Adaptive Channel Modulator ( Ours )	27.256	0.9308	0.2304	0.3962	49.65

TABLE XI: Ablation study on the learning manner of Adaptive Channel Modulator .

Panel of Figure 31 — Fig. 20: Visual effect on learning manner of Adaptive Channel Modulator (ACM) .

ID	Experiment	Metrics			Computational Complexity
ID	Experiment	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$
(a)	w/o GFR	27.013	0.9270	0.2442	0.3705	49.23
(b)	w/ Shared GFR ( Ours )	27.256	0.9308	0.2304	0.3962	49.65

TABLE XII: Effect on Gated Feature Refinement (GFR) .

Panel of Figure 33 — Fig. 21: Visual effect on Gated Feature Refinement (GFR).

ID	Experiment	Metrics			Computational Complexity
ID	Experiment	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Params. (M) $\downarrow$	FLOPs (G) $\downarrow$
(a)	w/o SR Space	26.449	0.9274	0.2355	0.3929	46.21
(b)	w/ SR Space ( Ours )	27.256	0.9308	0.2304	0.3962	49.65

TABLE XIII: Effect on the use of SR Space .

Panel of Figure 35 — Fig. 22: Visual effect on SR space .

$C$	4	8	16 ( Default )	24	32
PSNR (dB) $\uparrow$	22.789	24.773	27.256	26.690	27.158
SSIM $\uparrow$	0.9086	0.9162	0.9308	0.9298	0.9312
LPIPS $\downarrow$	0.306	0.2756	0.2304	0.2305	0.2250
Params. (M) $\downarrow$	0.0399	0.1193	0.3962	0.8310M	1.4237
FLOPs (G) $\downarrow$	4.07	13.69	49.65	107.89	188.39

TABLE XIV: Effect on the number of channels (

C

) .

$N$	8	12	16 ( Default )	20	24
PSNR (dB) $\uparrow$	27.188	27.452	27.256	26.858	26.583
SSIM $\uparrow$	0.9284	0.9299	0.9308	0.9274	0.9275
LPIPS $\downarrow$	0.2438	0.2378	0.2304	0.2401	0.2350
Params. (M) $\downarrow$	0.2913	0.3438	0.3962	0.4487	0.5011
FLOPs (G) $\downarrow$	37.83	43.74	49.65	55.57	61.48

TABLE XV: Effect on Number of Correlation Matching Transformation Transformer Blocks (

N

) .

$S$	2	4	8 ( Default )	16	32
PSNR (dB) $\uparrow$	27.602	27.109	27.256	27.051	26.682
SSIM $\uparrow$	0.9315	0.9295	0.9308	0.9280	0.9253
LPIPS $\downarrow$	0.2335	0.2370	0.2304	0.2375	0.2426
Params. (M) $\downarrow$	0.2426	0.2733	0.3962	0.8878	2.8538
FLOPs (G) $\downarrow$	96.27	58.98	49.65	47.32	46.74

TABLE XVI: Effect on Shuffle Down Factor (

S

) .

$\lambda_{freq}$	0	0.01	0.05 ( Default )	0.1	0.2	0.5
PSNR (dB) $\uparrow$	26.828	27.3703	27.256	26.634	27.215	27.005
SSIM $\uparrow$	0.9118	0.9257	0.9308	0.9281	0.9291	0.9305
LPIPS $\downarrow$	0.2545	0.2464	0.2304	0.2392	0.2360	0.2351
$\lambda_{re\_sr}$	0	0.1	0.25	0.5 ( Default )	0.75	1
PSNR (dB) $\uparrow$	26.683	26.421	26.734	27.256	26.864	26.897
SSIM $\uparrow$	0.9284	0.9283	0.9280	0.9308	0.9275	0.9290

TABLE XVII: Effect on weight of loss function .

Method	PSNR (dB) $\uparrow$	SSIM $\uparrow$	Params. $\downarrow$
Retinex-Net [ 62 ]	16.77	0.54	0.45M
Zero-DCE [ 84 ]	16.79	0.67	79.416K
AGLLNet [ 85 ]	17.52	0.77	3.438K
Zhao et. al. [ 76 ]	21.67	0.87	11.560M
RUAS [ 86 ]	16.44	0.70	3K
SCI [ 87 ]	14.78	0.62	0.258K

TABLE XVIII: Comparisons on general low-light image enhancement on the LOL [ 62 ] . UHDFour-2/8 means that the UHDFour version uses the shuffle-down factor with 2/8.

Panel of Figure 41 — Fig. 23: Low-light image enhancement example on LOL [ 62 ] . Although ourUHDformer++ consumes only 0.3962M parameters, it achieves comparable or superior performance on the low-light image enhancement problem.

Method	PSNR (dB) $\uparrow$	SSIM $\uparrow$	Params. $\downarrow$
SRN [ 11 ]	31.751	0.9144	6.8M
DMPHN [ 78 ]	31.734	0.9130	21.7M
MIMO-Unet+ [ 48 ]	33.277	0.9348	16.1M
MPRNet [ 14 ]	34.195	0.9454	20.1M
Restormer [ 49 ]	34.483	0.9488	26.1M
Uformer [ 16 ]	34.668	0.9502	20.6M

TABLE XIX: Results on general image deblurring on the GoPro [ 64 ] benchmarks as well as the limitations of our UHDformer++. Note that the PSNR and SSIM may be different from existing papers as we use the commonly used IQA PyTorch Toolbox (https://github.com/chaofengc/IQA-PyTorch) to recompute the results.

Explore similar work

Neural Discrimination-Prompted Transformers for Efficient UHD Image Restoration and Enhancement
Mar 1, 2026Cong Wang, Jinshan Pan, Liyan Wang +2All-in-One Image Restoration Efficient ViTs
Shenzhen Campus of Sun Yat-Sen University, China · The Hong Kong Polytechnic University, Hong Kong · Nanjing University of Science and Technology, China +2
From Zero to Detail: A Progressive Spectral Decoupling Paradigm for UHD Image Restoration with New Benchmark
Apr 17, 2026Chen Zhao, Yunzhe Xu, Zhizhou Chen +5Frequency-Domain Image Processing Image Restoration
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Computer Science and Technology, Dalian University of Technology, Dalian 116023, China · Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, U.S.A.
UHD Image Deblurring via Autoregressive Flow with Ill-conditioned Constraints
Mar 11, 2026Yucheng Xin, Dawei Zhao, Xiang Chen +6Image Deblurring Flow-Based Generative Modeling
Shandong Normal University · Qilu University of Technology · Nanjing University of Science and Technology +2

Mar 1, 2026J/K move · Enter open · S save

Neural Discrimination-Prompted Transformers for Efficient UHD Image Restoration and Enhancement

Cong Wang, Jinshan Pan, Liyan Wang, Wei Wang+1

Shenzhen Campus of Sun Yat-Sen University, China · The Hong Kong Polytechnic University, Hong Kong · Nanjing University of Science and Technology, China+2

All-in-One Image Restoration Efficient ViTs Image Deblurring Image Dehazing Low-Light Image Enhancement

Open paper AskPDF arXiv

We propose a simple yet effective UHDPromer, a neural discrimination-prompted Transformer, for Ultra-High-Definition (UHD) image restoration and enhancement. Our UHDPromer is inspired by an interesting observation that there implicitly exist neural differences between high-resolution and low-resolution features, and exploring such differences can facilitate low-resolution feature representation. To this end, we first introduce Neural Discrimination Priors (NDP) to measure the differences and then integrate NDP into the proposed Neural Discrimination-Prompted Attention (NDPA) and Neural Discrimination-Prompted Network (NDPN). The proposed NDPA re-formulates the attention by incorporating NDP to globally perceive useful discrimination information, while the NDPN explores a continuous gating mechanism guided by NDP to selectively permit the passage of beneficial content. To enhance the quality of restored images, we propose a super-resolution-guided reconstruction approach, which is guided by super-resolving low-resolution features to facilitate final UHD image restoration. Experiments show that UHDPromer achieves the best computational efficiency while still maintaining state-of-the-art performance on

3

UHD image restoration and enhancement tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes and pre-trained models will be made available at https://github.com/supersupercong/uhdpromer.

Abstract

Explore similar work

Neural Discrimination-Prompted Transformers for Efficient UHD Image Restoration and Enhancement

From Zero to Detail: A Progressive Spectral Decoupling Paradigm for UHD Image Restoration with New Benchmark

UHD Image Deblurring via Autoregressive Flow with Ill-conditioned Constraints