Organizations: School of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, Henan, China. · Shanghai Artificial Intelligence Laboratory, Shanghai 200233, China. · School of Automation, Central South University, Changsha 410083, China. · Intelligent Manufacturing Engineering at Machinery Technology Development Co.,Ltd., China Academy of Machinery Science and Technology Group Co.,Ltd., Beijing 100801, China.
Multimodal large models have shown excellent ability in addressing image super-resolution in real-world scenarios by leveraging language class as condition information, yet their abilities in degraded images remain limited. In this paper, we first revisit the capabilities of the Recognize Anything Model (RAM) for degraded images by calculating text similarity. We find that directly using contrastive learning to fine-tune RAM in the degraded space is difficult to achieve acceptable results. To address this issue, we employ a degradation selection strategy to propose a Real Embedding Extractor (REE), which achieves significant recognition performance gain on degraded image content through contrastive learning. Furthermore, we use a Conditional Feature Modulator (CFM) to incorporate the high-level information of REE for a powerful Mamba-based network, which can leverage effective pixel information to restore image textures and produce visually pleasing results. Extensive experiments demonstrate that the REE can effectively help image super-resolution networks balance fidelity and perceptual quality, highlighting the great potential of Mamba in real-world applications. The source code of this work will be made publicly available at: https://github.com/nathan66666/DACESR.git
Figures & tables
Fig. 1 : The tag representations of RAM on clean images and images with varying levels of degradation.“Similarity” refers to the Jaccard similarity (Eq. ( 1 )).
Fig. 2 : Comparison of text output accuracy across RAM under different types and intensities of degradation. (a) Blur: The x-axis represents isotropic Gaussian blur, where larger values indicate stronger blurring. (b) JPEG: The x-axis denotes JPEG compression levels, with lower values indicating higher compression. (c) Noise: The x-axis represents the intensity of additive Gaussian noise, where higher values correspond to increased noise levels. In (d), the classification is based on the text similarity values of RAM for different degraded outputs, which are evenly divided into four categories in descending order. Each category contains multiple types/levels of degradation.
Fig. 3 : The overview of DACESR.
Fig. 4 : The training pipeline of the Real Embedding Extractor (REE).
Datasets
Metric
ESRGAN
BSRGAN
Real-ESRGAN
SwinIR
DASR
HAT
KDSR
DCLS
CDFormer
DACESR
[ 13 ]
[ 14 ]
[ 15 ]
[ 10 ]
[ 16 ]
[ 11 ]
[ 50 ]
[ 51 ]
[ 52 ]
(Ours)
Bicubic
PSNR(↑)
28.17
27.32
26.65
27.21
28.55
27.44
27.48
30.73
30.87
28.34
LPIPS(↓)
0.115
0.236
0.228
0.214
0.170
0.203
0.295
0.291
0.286
0.139
Level-1
PSNR(↑)
21.16
26.78
26.17
26.45
27.84
26.85
26.69
27.32
26.90
28.16
LPIPS(↓)
0.473
0.241
0.231
0.216
0.171
0.202
0.307
0.368
0.363
0.142
Level-2
PSNR(↑)
22.77
26.75
26.16
26.39
27.58
26.81
26.76
27.24
26.92
27.68
TABLE I : Quantitative comparison with state-of-the-art methods for real-world image super-resolution on benchmark datasets. For the compared methods, we employ their officially released pre-trained models. The best and second best results are highlighted in red and blue respectively.
Fig. 5 : The LAM results of different model architectures across various types of degradation. LAM attribution indicates the significance of each pixel in the input LR image during the reconstruction process of the patch highlighted by a box. The Diffusion Index (DI) denotes the extent of pixel involvement. A higher DI indicates a broader range of utilized pixels.
Fig. 6 : Real-world image super-resolution results on SR × 4.
Network
Params(M)
FLOPS(G)
Time(ms)
Bicubic
Level-I
Level-II
Level-III
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
SRResNet
1.52
166
113
28.05/0.1747
27.60/0.1772
27.34/0.2228
23.71/0.4419
EDSR
1.52
130
105
28.26/0.1807
27.79/0.1834
27.53/0.2284
23.87/0.4351
SwinIR
11.72
539
1719
28.28/0.1488
27.78/0.1531
27.45/0.1854
23.60/0.3869
RRDB
16.70
1176
460
27.92/0.1473
27.84/0.1569
27.29/0.1886
23.54/0.3847
DASR
8.07
184
142
28.55 /0.1696
27.84/0.1707
27.58/0.2126
23.93/0.4144
TABLE II : Quantitative comparison of different backbone networks for real-world image super-resolution, all retrained on the same degradation space. FLOPS are measured with a 256×256 input, and all data are obtained using an NVIDIA V100 GPU. “Mamba-based” is a Mamba-based method, derived from [ 30 ] . The best results are highlighted in bold.
Method
DrealSR [ 58 ]
RealSR [ 53 ]
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
StableSR [ 41 ]
28.13/0.3315
24.70/0.3018
PASD [ 20 ]
27.00/0.3931
24.29/0.3435
DiffBIR [ 21 ]
26.76/0.4599
24.77/0.3658
SeeSR [ 19 ]
28.17/0.3189
25.18/0.3009
Ours
29.74 / 0.2803
27.31 / 0.2911
TABLE III : Quantitative comparison with diffusion-based methods. The data for the methods we compared is sourced from [ 19 ] . The best results are highlighted in bold.
Backbone
Condition
Bicubic
Level-I
Level-II
Level-III
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
Mamba-based
None
28.05 /0.1811
27.57 /0.1840
27.39 /0.2209
23.80 / 0.3938
Mamba-based
RE
27.79/ 0.1546
27.36/ 0.1676
27.08/ 0.1989
23.73/0.3975
Mamba-based
LE
27.56/0.1945
27.23/0.1903
27.07/0.2227
23.67/0.4078
TABLE IV : Quantitative Comparison of Different Conditional Information Prompts. “Mamba-based” refers to the backbone composed of RSSB [ 30 ] . “RE” represents the representation embedding of the low-resolution image, and “LE” represents the logits embedding of the low-resolution image.
Fig. 7 : Visual comparison of results with and without conditional information under different degradations using the same backbone network.
Backbone
Condition
Bicubic
Level-I
Level-II
Level-III
Network
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
PSNR(↑)/LPIPS(↓)
Mamba-based
RAM
27.72/0.1761
27.27/0.1770
27.08/0.2067
23.69/ 0.3860
Mamba-based
DAPE
27.45/0.1720
27.25/0.1740
26.96/0.2062
23.68/0.3931
Mamba-based
REE
27.79 / 0.1546
27.36 / 0.1676
27.08 / 0.1989
23.73 /0.3975
TABLE V : Quantitative Comparison of Different Conditional Networks. “Mamba-based” refers to the RSSB [ 30 ] .
Fig. 8 : Using images with different degradations as input, the average activation maps of the intermediate layers of each network.
×
+
Bicubic
Level-I
Level-II
Level-III
PSNR/LPIPS
PSNR/LPIPS
PSNR/LPIPS
PSNR/LPIPS
✔
✘
27.80 /0.1877
27.33/0.1858
27.10/0.2203
23.52/0.4011
✘
✔
27.71/0.1789
27.38 /0.1775
27.12 /0.2123
23.58/0.3982
✔
✔
27.79/ 0.1546
27.36/ 0.1676
27.08/ 0.1989
23.73 / 0.3975
TABLE VI : Quantitative Comparison of Conditioned Information Modulator Designs.
In an image latent space, the embeddings of high-resolution, natural, and sharp images form a manifold. Degradation of high-resolution images pushes their embeddings off this manifold. Real-world super-resolution (SR) then becomes the task of mapping the degraded embedding back onto this manifold --- not anywhere on the manifold, but to the point that preserves what the input still carries, both its semantics and pixel details. Every published method implements this mapping in a reconstruction-oriented latent space or pixel space. We claim these spaces are the wrong substrates for SR. Low-resolution and degraded images are embedded far from the manifold, making the mapping difficult and expensive. The lack of semantic information in these substrates also makes it difficult to navigate to the faithful point on the manifold, causing severe hallucination when degradation is heavy. Thus, restoring in a suitable latent space is crucial to the SR task. We show that the latent space of 23 fused layers of a frozen DINOv3-L is one such space that makes the SR task easier. Degraded images are embedded near the manifold. Moreover, this substrate contains a hierarchy of information, from pixel record to degradation robust semantics, guiding the model to find the faithful point on the manifold. On this substrate, a 415M decoder is trained under reconstruction and adversarial objectives to map the degraded embeddings back and decode to pixel space in one pass. The resulting model, RAESR, attains the best fidelity--perception trade-off among state-of-the-art adversarial and diffusion-based restorers on RealSR, DRealSR, LSDIR and DIV2K-Val, at 37 ms per 512 by 512 image on a single H20 GPU. Swapping the substrate for a VAE latent under an identical recipe loses on every metric.
Wanzhou Lei, Cuifeng Shen, Yanjin He +6
University of California, Berkeley · Alibaba Group · University of Michigan, Ann Arbor +2
Real-world image super-resolution is particularly challenging for diffusion models because real degradations are complex, heterogeneous, and rarely modeled explicitly. We propose a degradation-aware and structure-preserving diffusion framework for real-world SR. Specifically, we introduce Degradation-aware Token Injection, which encodes lightweight degradation statistics from low-resolution inputs and fuses them with semantic conditioning features, enabling explicit degradation-aware restoration. We further propose Spatially Asymmetric Noise Injection, which modulates diffusion noise with local edge strength to better preserve structural regions during training. Both modules are lightweight add-ons to the adopted diffusion SR framework, requiring only minor modifications to the conditioning pipeline. Experiments on DIV2K and RealSR show that our method delivers competitive no-reference perceptual quality and visually more realistic restoration results than recent baselines, while maintaining a favorable perception--distortion trade-off. Ablations confirm the effectiveness of each module and their complementary gains when combined. The code and model are publicly available at https://github.com/jiyang0315/DASP-SR.git.
Yang Ji, Zonghao Chen, Zhihao Xue +1
KUNBYTE Hangzhou, Zhejiang Province, China · GOLDMYE Xuhui District, Shanghai, China
Real-world image super-resolution (Real-ISR) requires balancing structural fidelity to degraded observations with realistic detail synthesis. However, existing generative Real-ISR methods often rely on entangled conditioning mechanisms, leading to structural drift or semantically inconsistent details. To address this issue, we propose Visual In-Context Restoration (VICR), a Diffusion Transformer (DiT)-based framework that formulates Real-ISR as image completion. Specifically, we introduce a decoupled visual prior injection mechanism that derives local and global cues from the low-quality (LQ) image: local cues help recover image structures and support high-frequency detail synthesis, while global cues guide overall generation and promote semantic consistency. For ambiguous regions under severe degradation, VICR employs an inference-time agent to refine semantic prompts using visual evidence from the LQ input while keeping model parameters fixed. Experiments show that VICR achieves state-of-the-art performance across multiple Real-ISR benchmarks with only 127M trainable parameters.
Qichang Zhang, Hailong Wang, Baiang Li +4
Faculty of Science and Technology, University of Macau · 2Nullmax · 3Hefei University of Technology +1