In an image latent space, the embeddings of high-resolution, natural, and sharp images form a manifold. Degradation of high-resolution images pushes their embeddings off this manifold. Real-world super-resolution (SR) then becomes the task of mapping the degraded embedding back onto this manifold --- not anywhere on the manifold, but to the point that preserves what the input still carries, both its semantics and pixel details. Every published method implements this mapping in a reconstruction-oriented latent space or pixel space. We claim these spaces are the wrong substrates for SR. Low-resolution and degraded images are embedded far from the manifold, making the mapping difficult and expensive. The lack of semantic information in these substrates also makes it difficult to navigate to the faithful point on the manifold, causing severe hallucination when degradation is heavy. Thus, restoring in a suitable latent space is crucial to the SR task. We show that the latent space of 23 fused layers of a frozen DINOv3-L is one such space that makes the SR task easier. Degraded images are embedded near the manifold. Moreover, this substrate contains a hierarchy of information, from pixel record to degradation robust semantics, guiding the model to find the faithful point on the manifold. On this substrate, a 415M decoder is trained under reconstruction and adversarial objectives to map the degraded embeddings back and decode to pixel space in one pass. The resulting model, RAESR, attains the best fidelity--perception trade-off among state-of-the-art adversarial and diffusion-based restorers on RealSR, DRealSR, LSDIR and DIV2K-Val, at 37 ms per 512 by 512 image on a single H20 GPU. Swapping the substrate for a VAE latent under an identical recipe loses on every metric.
Figures & tables
Figure 1: The choice of latent space decides the task. (a) A reconstruction-oriented latent. (b) The fused DINOv3-L latent (ours). Strips: decoded images along the straight latent line zα=(1−α)zLR+αzHR in each space.
Figure 2: Perceptual-Fidelity and Perceptual-Efficiency tradeoffs. Circle area scales with inference parameter count. The vertical axis fuses the six perceptual metrics, each min–max normalized and re-oriented, then averaged with equal weight. (a) against LPIPS; (b) against measured latency (one H20, batch 1 ). Construction and robustness checks of the fused score: App. 8.4 .
Figure 3: (a) Per layer, clean inputs: PSNR of a small decoder overfit to a few images (blue) and held-out ImageNet linear-probe accuracy (red). (b, c) R2 of a linear map from the fused latent z to each layer, on clean and training-degraded inputs.
Figure 4: The critic. The restored image and its ground truth pass through the same frozen DINOv3-B, and each of its 12 layers is read by its own trainable head that emits one logit per token.
Method
NFE
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
NIQE ↓
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
CLIP ↑
Params (M)
Lat. (ms)
Mem (GiB)
RAESR
1
24.08
0.6269
0.3007
0.1970
4.319
70.51
0.6713
0.6815
0.7049
719
37.2
1.57
RAESR D
1
23.39
0.6245
0.3127
0.2097
4.485
70.44
0.6652
0.6755
0.7033
502
30.0
1.18
Adversarial restorers
Real-ESRGAN
1
24.46
0.6648
0.3205
0.2208
5.282
60.50
0.5388
0.5270
0.5068
17
20.0
0.17
Real-HAT-GAN
1
25.56
0.6829
0.3110
0.2270
5.837
55.56
0.5310
0.4433
0.4087
21
156.7
0.73
Diffusion trained from scratch
Table 1: ×4 real-world SR, macro-averaged over the four benchmarks . NFE: network evaluations per image. RAESR D : distilled. Bold best, underline second; per-benchmark tables in App. 8 .
Figure 5: Qualitative ×4 comparison, all sixteen panels. Full frames with the region boxed; right: that region across every method at 4× nearest-neighbour magnification, ours in red . Blocks from top: LSDIR, RealSR, DIV2K-Val. Best viewed zoomed in. App. 16 shows other examples.
Comparison
RAESR votes
Baseline votes
Ties
RAESR preference
RAESR vs. Real-HAT-GAN
163
26
26
86.2%
RAESR vs. DiffBIR
133
55
30
70.7%
RAESR vs. SUPIR
109
80
22
57.7%
RAESR vs. TSD-SR
101
91
27
52.6%
Table 2: Human preference, two-alternative forced choice with input and ground truth shown.
Figure 6: Raw metric curves along the latent line zα=(1−α)zLR+αzHR , decoded at eleven points. Curves are means over 2,000 images.
Figure 7: Left: relative change in each metric when one eight-layer band of the average is removed; negative means the metric improves. Right: the same interventions on one LSDIR image.
Figure 8: Swapping the substrate, keeping everything else. Bold marks the better score of the two. Two further examples in Fig. 11 .
Method
Reconstruction
Ref. perceptual
Adversarial
NR-IQA
Real-ESRGAN
L1
VGG feature
GAN
none
StableSR
latent ϵ -MSE; CFW: L1
CFW: LPIPS
CFW: GAN
none
ResShift (4-step)
latent MSE
LPIPS
—
none
OSEDiff
pixel MSE + VSD
LPIPS
—
none
DiffBIR
MSE / latent ϵ -MSE
—
—
none
SUPIR
(training code unreleased)
—
—
none stated
Table 3: Training signals of the baselines (coarse; official paper/code). Ref. perc. = reference-based perceptual loss; NR-IQA = a no-reference quality model as a training signal.
Method
NFE
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
NIQE ↓
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
CLIP ↑
RAESR
1
19.77
0.4354
0.3485
0.1923
3.312
73.58
0.6987
0.7192
0.7108
RAESR D
1
19.92
0.4409
0.3629
0.2098
3.416
73.13
0.6859
0.7062
0.7039
Adversarial restorers
Real-ESRGAN
1
19.95
0.4694
0.4135
0.2526
3.942
66.22
0.5663
0.5997
0.5899
Real-HAT-GAN
1
20.55
0.4791
0.4289
0.2769
4.345
59.37
0.5433
0.4852
0.4570
Diffusion trained from scratch
Table 4: Per-benchmark results: LSDIR (250 images). Marks and protocol as in Table 1 .
Method
NFE
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
NIQE ↓
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
CLIP ↑
RAESR
1
25.00
0.7195
0.2703
0.2012
4.587
70.49
0.6757
0.6705
0.6841
RAESR D
1
23.45
0.7045
0.2864
0.2141
4.856
70.77
0.6748
0.6718
0.6852
Adversarial restorers
Real-ESRGAN
1
25.00
0.7473
0.2753
0.2076
5.807
60.43
0.5500
0.5163
0.4581
Real-HAT-GAN
1
26.55
0.7655
0.2480
0.1969
6.348
57.11
0.5513
0.4458
0.3737
Diffusion trained from scratch
Table 5: Per-benchmark results: RealSR (100 images). Marks and protocol as in Table 1 .
Method
NFE
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
NIQE ↓
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
CLIP ↑
RAESR
1
27.85
0.7613
0.3030
0.2120
5.348
67.41
0.6449
0.6517
0.7091
RAESR D
1
26.52
0.7567
0.3102
0.2207
5.531
67.40
0.6405
0.6477
0.7132
Adversarial restorers
Real-ESRGAN
1
28.61
0.8051
0.2819
0.2089
6.698
54.27
0.4903
0.4624
0.4516
Real-HAT-GAN
1
30.02
0.8324
0.2605
0.2063
7.294
50.56
0.4915
0.4053
0.3916
Diffusion trained from scratch
Table 6: Per-benchmark results: DRealSR (93 images). Marks and protocol as in Table 1 .
Method
NFE
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
NIQE ↓
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
CLIP ↑
RAESR
1
23.71
0.5915
0.2810
0.1827
4.027
70.55
0.6658
0.6845
0.7158
RAESR D
1
23.66
0.5958
0.2913
0.1943
4.136
70.46
0.6595
0.6764
0.7108
Adversarial restorers
Real-ESRGAN
1
24.29
0.6372
0.3112
0.2141
4.679
61.06
0.5485
0.5297
0.5277
Real-HAT-GAN
1
25.10
0.6547
0.3066
0.2279
5.361
55.20
0.5378
0.4370
0.4124
Diffusion trained from scratch
Table 7: Per-benchmark results: DIV2K-Val (3000 images). Marks and protocol as in Table 1 .
Figure 9: Layer-pair geometry on training-degraded inputs: chance-corrected, variance-matched containment Ω (left) and linear CKA against layer distance (right).
Threshold
RAE
SD3-VAE
SD2.1-VAE
NIQE ≤5
0.47
0.65
0.68
NIQE ≤6
0.39
0.54
0.54
NIQE ≤7
0.33
0.43
0.43
LPIPS ≤0.25
0.59
0.71
0.73
LPIPS ≤0.35
0.50
0.63
0.63
LPIPS ≤0.50
0.39
0.48
0.46
Table 8: Path fraction α at which the mean raw curve first crosses each threshold (smaller: the decoded line turns natural, or reaches the ground truth, earlier). The RAE column is shared by both studies, which use the same RAE rows.
Figure 10: The latent line with the SD2.1 VAE , on the same images and with the same protocol and thresholds as Fig. 6 . The number above each arrow is the gap between the two threshold crossings.
Condition
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
NIQE ↓
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
CLIP ↑
A unmodified
23.72
0.6265
0.3012
0.1971
4.320
70.45
0.6717
0.6820
0.7059
B no-shallow (layers 2 – 9 )
16.43
0.5179
0.5194
0.3807
5.021
70.66
0.6585
0.6454
0.5994
C no-deep (layers 17 – 24 )
24.60
0.6565
0.3079
0.2088
4.687
66.75
0.6134
0.6177
0.6622
D random band (3-seed mean)
20.48
0.6069
0.3728
0.2733
4.884
69.59
0.6402
0.6497
0.6482
seed 0
22.67
0.6356
0.3270
0.2308
4.814
68.25
0.6230
0.6432
0.6541
seed 1
19.90
0.6030
0.3777
0.2820
4.931
70.19
0.6500
0.6531
0.6572
Table 9: Decoder channel ablation , macro-averaged over the four benchmarks. B–D replace eight of the 23 averaged layers with training-set mean tokens at inference; E additionally replaces the broadcast term of layer 24 by its training-set mean. D is the mean of three random bands. Bold marks the cells the dissociation turns on: fidelity collapsing without the shallow band and improving without the deep one, further without the broadcast.
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
NIQE ↓
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
CLIP ↑
Released recipe (no quality metric in the loss)
RAESR (released)
23.72
0.6265
0.3012
0.1971
4.320
70.45
0.6717
0.6820
0.7059
SD3-VAE substrate
23.61
0.6033
0.3734
0.2495
4.344
69.76
0.6300
0.6632
0.7011
on LSDIR
19.73
0.4242
0.4291
0.2481
3.274
73.33
0.6492
0.7023
0.7210
on RealSR
24.25
0.6831
0.3390
0.2521
4.694
69.26
0.6313
0.6481
0.6801
on DRealSR
27.15
0.7320
0.3711
0.2618
5.528
65.82
0.6052
0.6286
0.6776
Table 10: Encoder-substrate ablation under the released recipe; the DINOv3 substrate leads on all nine metrics.
Figure 11: Swapping the substrate: two further examples. Format as in Fig. 8 . The tractor becomes a pond and the cathedral’s tracery is redrawn as generic stonework.
Figure 12: LSDIR. The first sample is the example of Fig. 5 . Best viewed zoomed in.
Figure 13: RealSR. The first sample is the example of Fig. 5 . Best viewed zoomed in.
Figure 14: DIV2K-Val. The first sample is the example of Fig. 5 . Best viewed zoomed in.
Real-world image super-resolution is particularly challenging for diffusion models because real degradations are complex, heterogeneous, and rarely modeled explicitly. We propose a degradation-aware and structure-preserving diffusion framework for real-world SR. Specifically, we introduce Degradation-aware Token Injection, which encodes lightweight degradation statistics from low-resolution inputs and fuses them with semantic conditioning features, enabling explicit degradation-aware restoration. We further propose Spatially Asymmetric Noise Injection, which modulates diffusion noise with local edge strength to better preserve structural regions during training. Both modules are lightweight add-ons to the adopted diffusion SR framework, requiring only minor modifications to the conditioning pipeline. Experiments on DIV2K and RealSR show that our method delivers competitive no-reference perceptual quality and visually more realistic restoration results than recent baselines, while maintaining a favorable perception--distortion trade-off. Ablations confirm the effectiveness of each module and their complementary gains when combined. The code and model are publicly available at https://github.com/jiyang0315/DASP-SR.git.
Yang Ji, Zonghao Chen, Zhihao Xue +1
KUNBYTE Hangzhou, Zhejiang Province, China · GOLDMYE Xuhui District, Shanghai, China
Image super-resolution (SR) with large generative models has recently achieved remarkable perceptual quality, yet maintaining fidelity to the LR observation remains challenging. In particular, we observe that diffusion transformers (DiTs) built on latent representations suffer from a critical limitation: the compression bottleneck of the VAE weakens fine-grained spatial information, leading to hallucinated details that are weakly grounded in the input image. In this work, we revisit generative SR from a representation perspective and propose a pixel-grounded super-resolution (PGSR) framework that preserves LR-observed pixel evidence before VAE compression and reuses it throughout restoration. Instead of relying solely on the compressed latent condition, PGSR extracts pre-VAE pixel evidence from the upsampled LR image and reuses it at two stages. First, Condition-Side Trajectory Guidance fuses LR-derived pixel evidence with the latent LR condition to guide the latent restoration trajectory. Second, Decoder-Side Pixel Grounding injects multi-scale pixel features into the frozen VAE decoder to ground the final rendering with LR-observed cues. To efficiently adapt large pretrained DiT models, we keep the latent autoencoder and main flow-matching backbone frozen, and train only lightweight restoration modules. We further study an efficient local-window attention variant for improved high-resolution efficiency and scalability. Extensive experiments demonstrate that PGSR improves the realism--fidelity trade-off and produces more faithful, visually convincing results than existing latent generative SR approaches.
Multimodal large models have shown excellent ability in addressing image super-resolution in real-world scenarios by leveraging language class as condition information, yet their abilities in degraded images remain limited. In this paper, we first revisit the capabilities of the Recognize Anything Model (RAM) for degraded images by calculating text similarity. We find that directly using contrastive learning to fine-tune RAM in the degraded space is difficult to achieve acceptable results. To address this issue, we employ a degradation selection strategy to propose a Real Embedding Extractor (REE), which achieves significant recognition performance gain on degraded image content through contrastive learning. Furthermore, we use a Conditional Feature Modulator (CFM) to incorporate the high-level information of REE for a powerful Mamba-based network, which can leverage effective pixel information to restore image textures and produce visually pleasing results. Extensive experiments demonstrate that the REE can effectively help image super-resolution networks balance fidelity and perceptual quality, highlighting the great potential of Mamba in real-world applications. The source code of this work will be made publicly available at: https://github.com/nathan66666/DACESR.git
Xiaoyan Lei, Wenlong Zhang, Biao Luo +3
School of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, Henan, China. · Shanghai Artificial Intelligence Laboratory, Shanghai 200233, China. · School of Automation, Central South University, Changsha 410083, China. +1