Deep LoRA-Unfolding Networks for Image Restoration

Xiangming Wang; Haijin Zeng; Benteng Sun; Jiezhang Cao; Kai Zhang; Qiangqiang Shen; Yongyong Chen

cs.CVFeb 21, 2026

Deep LoRA-Unfolding Networks for Image Restoration

Authors: Xiangming Wang, Haijin Zeng, Benteng Sun, Jiezhang Cao, Kai Zhang, Qiangqiang Shen, Yongyong Chen

Organizations: School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen 518055, Guangdong, China · Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China · School of Electronics and Information Engineering, Harbin Institute of Technology (Shenzhen), Shenzhen 518055, Guangdong, China

Abstract

Deep unfolding networks (DUNs), combining conventional iterative optimization algorithms and deep neural networks into a multi-stage framework, have achieved remarkable accomplishments in Image Restoration (IR), such as spectral imaging reconstruction, compressive sensing and super-resolution.It unfolds the iterative optimization steps into a stack of sequentially linked blocks.Each block consists of a Gradient Descent Module (GDM) and a Proximal Mapping Module (PMM) which is equivalent to a denoiser from a Bayesian perspective, operating on Gaussian noise with a known level.However, existing DUNs suffer from two critical limitations: (i) their PMMs share identical architectures and denoising objectives across stages, ignoring the need for stage-specific adaptation to varying noise levels; and (ii) their chain of structurally repetitive blocks results in severe parameter redundancy and high memory consumption, hindering deployment in large-scale or resource-constrained scenarios.To address these challenges, we introduce generalized Deep Low-rank Adaptation (LoRA) Unfolding Networks for image restoration, named LoRun, harmonizing denoising objectives and adapting different denoising levels between stages with compressed memory usage for more efficient DUN.LoRun introduces a novel paradigm where a single pretrained base denoiser is shared across all stages, while lightweight, stage-specific LoRA adapters are injected into the PMMs to dynamically modulate denoising behavior according to the noise level at each unfolding step.This design decouples the core restoration capability from task-specific adaptation, enabling precise control over denoising intensity without duplicating full network parameters and achieving up to $N$ times parameter reduction for an $N$ -stage DUN with on-par or better performance.Extensive experiments conducted on three IR tasks validate the efficiency of our method.

Figures & tables

Fig. 1: PSNR-Params.-FLOPs comparison of our LoRun and SOTAs in CASSI task. Our LoRun reduces the parameters of traditional DUN methods by nearly 70% while maintaining comparable performance.

Fig. 2: Structure comparison between the previous DUNs and our hierarchical deep LoRA-unfolding networks. Previous DUNs adopt independent parameters to all stages while our LoRun first leverages pre-trained denoiser for each stage then is trained with different LoRA layers for different denoising abilities.

Fig. 3: Representation for LoRA adopted in our LoRun framework.

Fig. 4: Illustration of the proposed LoRun method. (a) The gradient descent module. (a-1) The gradient descent process based on the PGD algorithm. (a-2) The gradient descent process based on the HQS algorithm. (b) The proximal mapping module (denoiser). (c) Framework of the proposed deep LoRA-Unfolding network. Our LoRun first adopts the same most frozen-shared backbone denoiser among all stages, followed by fine-tuning the denoising level and direction of each stage with small learnable-separate low-rank modules. This strategy mostly reduces the parameters and accelerates convergence.

Datasets	CS Ratios	AMP-Net	CASNet	DGU-Net +	TransCS	AutoBCS	CSformer	NesTD-Net	Block-9	LoRun-9
		TIP’21	TIP’22	CVPR’22	TIP’22	TCYB’23	TIP’23	TIP’24	Ours	Ours
-	-	0.94M	16.90M	6.81M	1.49M	2.01M	6.65M	5.93M	15.96M	5.23M
General100	0.01	22.71/ 0.68	23.48/0.65	22.86/0.62	21.66/0.54	22.24/0.62	23.35/ 0.69	23.14/0.62	23.89 /0.65	23.92 /0.65
	0.04	26.96/0.77	28.50/ 0.82	27.92/ 0.81	27.25/0.74	27.91/0.80	28.61 /0.80	28.58/ 0.82	28.60/ 0.82	28.70 / 0.82
	0.10	30.62/0.88	32.78/ 0.91	32.74/ 0.91	32.17/0.87	32.70/ 0.89	32.97 /0.88	32.85 / 0.91	32.82/ 0.91	32.78/ 0.91
	0.25	36.01/0.95	38.07/ 0.97	37.55/ 0.96	36.95/ 0.96	36.71/0.96	36.51/ 0.96	38.42 / 0.97	38.17 / 0.96	38.16/ 0.97

TABLE I: Average PSNR (dB)/SSIM performance comparisons of recent CS methods on various datasets at different CS ratios.

Panel of Figure 6 — Fig. 5: CS results under ratio $\beta=25\%$ of different methods and datasets, from top to bottom General100 , Set11 and Set14 .

Model	Method	$\beta$ =1%	$\beta$ =4%	$\beta$ =10%	$\beta$ =25%	$\beta$ =30%	Params.
Transformer	Block-9	20.73 0.54	24.50 0.74	28.44 0.88	33.63 0.95	34.72 0.95	0.10M
Transformer	LoRun-9 $\gamma$ =10	21.02 0.55	25.17 0.75	29.14 0.89	34.16 0.95	35.54 0.95	0.06M
U-Net	Block-9	21.14 0.56	25.60 0.79	29.57 0.89	35.00 0.96	36.36 0.96	4.65M
U-Net	LoRun-9 $\gamma$ =10	21.13 0.56	25.45 0.79	29.29 0.89	34.75 0.95	36.17 0.96	1.42M

TABLE II: Comparisons on Set11 under different CS ratios for CS based on Transformer and U-Net denoiser. PSNR (upper entry), SSIM (lower entry) and parameters are reported.

DeSCI

\lambda

-Net

TSA-Net

PnP-CASSI

DIP-HSI

BIRNAT

GAP-Net

DAUHST

DWMT

PSRSCI

RCUMP

Block-9

LoRun-9

Category

Model

CNN

PnP

RNN

DUN

Transformer

Diffusion

DUN

Supervised

Zero-Shot

Supervised

Reference

TPAMI’19

ICCV’19

ECCV’20

PR’21

ICCV’21

TPAMI’23

IJCV’23

NeurIPS’22

AAAI’24

ICLR’25

TIP’24

Ours

Params.

62.64M

44.25M

33.85M

4.40M

4.27M

6.15M

15.47M

1312M

9.03M

5.18M

1.62M

scene01

28.38/0.80

30.10/0.85

32.31/0.89

29.09/0.80

31.32/0.86

36.79/0.95

33.63/0.91

37.25/ 0.96

36.46/ 0.96

37.18/ 0.96

37.63/ 0.96

38.88 / 0.97

38.78 / 0.97

scene02

26.00/0.70

28.49/0.81

31.03/0.86

28.05/0.71

25.89/0.70

37.89/0.96

33.19/0.90

39.02/ 0.97

37.75/0.96

38.74/ 0.97

39.98/ 0.97

41.70 / 0.98

41.68 / 0.98

TABLE III: Comparisons of PSNR (dB)/SSIM between LoRun (

\gamma=10

) and SOTA methods on 10 simulation scenes in KAIST for CASSI .

Panel of Figure 9 — Fig. 6: CASSI results of scene01 in different bands, from top to bottom $\lambda\in\{$ 481.5\text{,}\mathrm{nm} $,$ 492.5\text{,}\mathrm{nm} $,$ 522.5\text{,}\mathrm{nm} $,$ 567.5\text{,}\mathrm{nm} $,$ 604.5\text{,}\mathrm{nm} $\}$ .

Panel of Figure 10 — Fig. 7: CASSI results of different scenes in different bands, from top to bottom $\lambda\in\{$ 487.0\text{,}\mathrm{nm} $,$ 498.0\text{,}\mathrm{nm} $,$ 584.5\text{,}\mathrm{nm} $,$ 492.5\text{,}\mathrm{nm} $,$ 476.5\text{,}\mathrm{nm} $\}$ .

Panel of Figure 11 — Fig. 8: CASSI measurements of different scenes, each scene has the size of $256\times 310$ .

Method	scene01	scene02	scene03	scene04	scene05	Avg	Params.
Block-9	34.87 0.94	35.32 0.94	35.48 0.95	40.55 0.97	32.74 0.95	35.79 0.95	4.80M
LoRun-9 $\gamma$ =10	34.20 0.93	34.40 0.92	35.23 0.95	41.26 0.98	32.15 0.94	35.45 0.94	1.43M

TABLE IV: Comparisons on CASSI datasets with U-Net denoiser. PSNR (upper entry), SSIM (lower entry) and parameters are reported.

Panel of Figure 13 — Fig. 9: Presentation of 4 isotropic Gaussian kernels with different widths (0.7, 1.2, 1.6, 2.0), 4 anisotropic Gaussian kernels and 4 motion blur kernels.

Method	SF	K1	K2	K3	K4	K5	K6	K7	K8	K9	K10	K11	K12	Params.
Block-9	$\times 2$	29.90	30.31	29.79	28.70	28.41	28.29	28.49	27.63	29.57	29.49	29.54	29.52	5.3M
Block-9	$\times 3$	26.70	27.37	27.51	27.46	27.27	27.21	27.30	27.10	26.89	26.78	26.68	26.52	5.3M
	$\times 4$	24.75	25.60	25.84	25.94	25.86	25.75	25.79	25.95	25.18	24.92	24.97	24.96
LoRun-9 $\gamma=10$	$\times 2$	29.88	30.17	29.73	28.72	28.45	28.25	28.42	27.60	29.03	29.31	29.43	29.43	1.96M
LoRun-9 $\gamma=10$	$\times 3$	26.73	27.38	27.52	27.47	27.29	27.25	27.22	27.02	26.65	26.68	26.43	26.33	1.96M
	$\times 4$	24.74	25.65	25.89	25.99	25.90	25.80	25.75	25.99	25.05	24.82	24.72	24.83

TABLE V: Comparisons between our LoRun (

\gamma=10

) and Block-9 for BSD68 datasets SR in different SFs and different blur kernels.

Panel of Figure 15 — Fig. 10: SR results of different blur kernels and SFs. (Up: K5 , SF= $\times 2$ . Down: K9 , SF= $\times 4$ .)

Method	$\beta$ =1%	$\beta$ =4%	$\beta$ =10%	$\beta$ =25%	$\beta$ =30%	$\beta$ =40%	$\beta$ =50%
LoRun-9 $\beta$ -pretrained=1%	21.07 0.55	25.41 0.79	29.15 0.89	34.47 0.95	35.72 0.96	38.09 0.97	40.26 0.98
LoRun-9 $\beta$ -pretrained=50%	21.08 0.55	25.66 0.79	29.57 0.89	35.07 0.96	36.31 0.96	38.65 0.98	40.58 0.98

TABLE VI: Ablation experiments on

\beta

-pretrained based on the U-Net denoiser in CS .

Method	$\beta$ =1%	$\beta$ =4%	$\beta$ =10%	$\beta$ =25%	$\beta$ =30%	$\beta$ =40%	$\beta$ =50%
LoRun-9 $\gamma$ =10	21.02 0.55	25.17 0.75	29.14 0.89	34.16 0.95	35.54 0.95	37.88 0.97	40.10 0.98
LoRun-9 $\gamma$ =20	21.08 0.55	25.17 0.75	29.13 0.89	34.24 0.95	35.56 0.95	37.95 0.97	40.14 0.98
LoRun-9 $\gamma$ =80	20.97 0.55	25.01 0.74	28.83 0.88	33.93 0.95	35.30 0.95	37.74 0.97	39.98 0.98

TABLE VII: Ablation on

\gamma

based on the Transformer denoiser in CS .

Method	baby	bird	head	Average	Params.
Block-1	29.62	27.55	28.55	28.57	0.59M
Block-1 w/o GDM	29.14	27.28	25.80	27.41	0.59M

TABLE VIII: Ablation analysis for GDM in SR with SF =

\times 3

on Set5 .

Fig. 11: Loss curves with training iterations among different training strategies.

	$K=1$	$K=4$	$K=9$	$K=12$
Original	31.08 0.9032	32.28 0.9085	32.84 0.9128	32.91 0.9133
LoRun (Ours)	31.28 0.9062	32.24 0.9090	32.78 0.9125	32.81 0.9130

TABLE IX: Ablation analysis for the effect of the number

K

of blocks.

Panel of Figure 21 — Fig. 12: Visualization of the selected LoRA modules.

Panel of Figure 22 — Fig. 13: Structures of the Transformer denoiser and its components. (a) The Transformer denoiser which aims to demonstrate the performance of our LoRun in simple but classical modules. (b) The feed forward network of the Transformer. (c) The attention network induced with LoRA of the Transformer.

Panel of Figure 23 — Fig. 14: Structures of the U-Net denoiser and its components. (a) The U-Net denoiser which aims to demonstrate the performance of our LoRun in simple but classical modules. (b) The double-convolution module induced with LoRA.

Panel of Figure 24 — Fig. 15: Structures of the components of the denoiser for CASSI task. (a) The convolution modulated block of the denoiser. (b) The spatial alignment block. (c) The window-based spectra wise self-attention with LoRA.

Panel of Figure 25 — Fig. 16: Structures of the denoiser and its components for CS task. (a) The depth-wise channel attention block. (b) The multi-scale transformer module with LoRA. (c) The denoiser which is the dual path fusion block for CS task.

Fig. 17: Structures of the denoiser and its components for SR task. This architecture is based on the convolution-LoRA module and upsample block.

Deep Unfolding

Deep unfolding turns iterative algorithms into trainable networks, combining model-based structure with learned components for efficient inference and optimization.

See all 37 papers

Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
Jul 28, 2026Jaeha Kim, Kyoung Mu LeeDiffusion Models for Image Restoration Low-Rank Adaptation
Dept. of ECE & ASRI, Seoul National University, Seoul, Korea · IPAI, Seoul National University, Seoul, Korea
DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration
Jun 17, 2026Yanjie Tu, Qingsen Yan, Axi Niu +3All-in-One Image Restoration Image Restoration
School of Computer Science, Northwestern Polytechnical University, Xi’an 710000, China · Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen 518057, China · State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau 999078, China
Newton Deep Unfolding for Compressed Sensing
Sep 13, 2026Changhua He, Xianchao XiuCompressed Sensing Deep Unfolding
School of Mechatronic Engineering and Automation, Shanghai University

Jul 28, 2026J/K move · Enter open · S save

Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors

Jaeha Kim, Kyoung Mu Lee

Dept. of ECE & ASRI, Seoul National University, Seoul, Korea · IPAI, Seoul National University, Seoul, Korea

Diffusion Models for Image Restoration Low-Rank Adaptation Image Restoration Diffusion-Based Inverse Problems Diffusion Model Inference Acceleration Diffusion-Based Denoising

Open paper AskPDF arXiv

Degraded images not only reduce visual quality but also impair downstream high-level vision tasks. Task-driven image restoration (TDIR) addresses this issue by jointly optimizing restoration quality and task performance. Recent works show that pretrained diffusion priors benefit TDIR, yet diffusion-based restoration is inherently stochastic, as the sampling process depends on a random noise term, which can undermine task consistency. In this paper, we show that a deterministic, noise-free one-step forward pass with pretrained diffusion priors can substantially improve TDIR, but the benefit critically depends on the adaptation module: LoRA yields consistent gains, whereas ControlNet-style conditioning does not. This enables one-step forwarding that surpasses conventional multi-step diffusion TDIR baselines. Furthermore, we introduce a task-preserving GAN training strategy that improves perceptual quality without sacrificing task performance. Extensive experiments on classification, segmentation, and detection demonstrate consistent gains over prior TDIR methods, and we further validate generalization on real-world degraded images and OCR.

Abstract

Deep Unfolding

Explore similar work

Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors

DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration

Newton Deep Unfolding for Compressed Sensing