Image Super-Resolution (SR) aims to reconstruct high-resolution images from degraded low-resolution inputs. While diffusion-based SR methods offer powerful generative capabilities, their performance heavily depends on how semantic priors are structured and integrated into the generation process. Existing approaches often rely on entangled or coarse-grained priors that mix global layout with local details, or conflate structural and textural cues, thereby limiting semantic controllability and interpretability. In this work, we propose DTPSR, a novel diffusion-based SR framework that introduces disentangled textual priors along two complementary dimensions: spatial hierarchy (global vs. local) and frequency semantics (low- vs. high-frequency). By explicitly separating these priors, DTPSR enables the model to simultaneously capture scene-level structure and object-specific details with frequency-aware semantic guidance. The corresponding embeddings are injected via specialized cross-attention modules, forming a progressive generation pipeline that reflects the semantic granularity of visual content, from global layout to fine-grained textures. To support this paradigm, we construct DisText-SR, a large-scale dataset containing approximately 95,000 image-text pairs with carefully disentangled global, low-frequency, and high-frequency descriptions. To further enhance controllability and consistency, we adopt a multi-branch classifier-free guidance strategy with frequency-aware negative prompts to suppress hallucinations and semantic drift. Extensive experiments on synthetic and real-world benchmarks show that DTPSR achieves high perceptual quality, competitive fidelity, and strong generalization across diverse degradation scenarios.
Figures & tables
Figure 1: Comparison under severe degradation. Without textual priors, the diffusion model suffers from hallucinations, generating human-like artifacts or misinterpreting walls as ocean textures. Incorporating our disentangled textual priors suppresses such errors and enhances semantic consistency.
Figure 2: The overall architecture of DTPSR. Given an LR image, a global prior and local object priors are extracted, where local priors are disentangled into Low-Frequency (LF) descriptions (shape, layout, color) and High-Frequency (HF) descriptions (texture, edges, details). They are encoded into eg , Elf , and Ehf via CLIP, while the LR image is encoded into flr by an image encoder. The diffusion process sequentially updates the latent zt through GTCA, LFCA, and HFCA—yielding intermediate representations ztg , ztlf , and zthf —and then fuses flr via LRCA to produce zt−1 , progressively restoring structures and details.
Figure 3: DisText-SR dataset construction. Given an LR image, we use a segmentation model to extract regions S1,S2,…,Sn , and query a frozen MLLM to generate a global description cg and per-segment low-frequency ( clf(i) ) and high-frequency ( chf(i) ) texts. They are encoded via CLIP into Global, Low-Frequency (LF) and High-Frequency (HF) embeddings for semantic-guided SR.
Dataset
Metrics
Real-ESRGAN [ 36 ]
BSRGAN [ 44 ]
StableSR [ 34 ]
DiffBIR [ 21 ]
PASD [ 42 ]
SeeSR [ 40 ]
SUPIR [ 43 ]
FaithDiff [ 8 ]
Ours
DIV2K-Val [ 1 ]
PSNR↑
24.12
24.62
22.90
22.87
24.41
23.67
23.16
23.49
22.03
SSIM↑
0.6303
0.6311
0.5609
0.5548
0.6252
0.6043
0.5441
0.5817
0.5471
LPIPS↓
0.3168
0.3390
0.3158
0.3645
0.3794
0.3194
0.3625
0.3118
0.3806
MUSIQ↑
61.29
60.52
61.06
69.29
61.27
68.67
62.59
69.18
71.24
MANIQA↑
0.3803
0.3502
0.3586
0.5699
0.4042
0.5036
0.5224
0.4309
0.5866
CLIPIQA↑
0.5212
0.5311
0.6174
0.7156
0.5568
0.6935
0.7040
0.6463
0.7549
Table 1: Quantitative comparison with state-of-the-art methods on DIV2K-Val, RealSR, and DRealSR datasets. The best and second-best results for each metric are highlighted in bold and underlined , respectively.
Figure 4: Qualitative comparison with representative SR methods. Our DTPSR reconstructs sharper textures and more semantically aligned details, especially under complex degradations, compared to both GAN-based ( e.g . , BSRGAN, Real-ESRGAN) and diffusion-based ( e.g . , FaithDiff, SUPIR) approaches. Zoom in for better visual comparison. More qualitative comparisons can be found in Sec. 4 of the supplementary material.
Method
Params
Steps
Inference Time
SUPIR [ 43 ]
17.8B
100
17.25s
FaithDiff [ 8 ]
15.6B
20
13.20s
DTPSR (Ours)
10.5B
50
14.94s
Table 2: Comparison of model complexity and runtime efficiency on the DRealSR dataset. Inference time is reported as the average per-image runtime.
Exp
Global
Local
MANIQA ↑
CLIP-IQA ↑
MUSIQ ↑
3-1
×
×
0.5271
0.7064
67.48
3-2
×
✓
0.5851
0.7471
68.86
3-3
✓
×
0.5394
0.7211
67.80
3-4
✓
✓
0.6011
0.7640
69.24
Table 3: Ablation study on the effect of global and local textual priors on the DRealSR dataset.
Exp
HF
LF
MANIQA ↑
CLIP-IQA ↑
MUSIQ ↑
4-1
×
✓
0.5850
0.7544
68.96
4-2
✓
×
0.5495
0.7301
68.08
4-3
✓
✓
0.6011
0.7640
69.24
Table 4: Ablation study on the effect of frequency-aware textual priors on the DRealSR dataset.
Exp
Type
MANIQA ↑
CLIP-IQA ↑
MUSIQ ↑
5-1
Mixed
0.5947
0.7527
69.05
5-2
Disentangled
0.6011
0.7640
69.24
Table 5: Comparison of frequency-mixed and frequency-disentangled learning on the DRealSR dataset.
Exp
CFG Strategy
MANIQA ↑
CLIP-IQA ↑
MUSIQ ↑
6-1
None
0.5995
0.6992
66.73
6-2
Single
0.5776
0.7499
67.87
6-3
Multi (Ours)
0.6011
0.7640
69.24
Table 6: Ablation study on multi-branch classifier-free guidance (CFG) on the DRealSR dataset.
Method
MUSIQ↑
MANIQA↑
CLIP-IQA↑
DiffBIR [ 21 ]
68.86
0.5589
0.7030
PASD [ 42 ]
59.99
0.3939
0.4855
SeeSR [ 40 ]
69.82
0.5437
0.6701
SUPIR [ 43 ]
58.51
0.4429
0.6357
FaithDiff [ 8 ]
68.99
0.4644
0.6126
DTPSR-C
71.64
0.5855
0.7061
Table 7: Robustness comparison under textual corruption on the DRealSR dataset. DTPSR-C denotes the variant with randomly corrupted textual descriptions (“None” substitution). Despite artificial textual errors, DTPSR remains superior across perceptual metrics.
Despite recent advances, single-image super-resolution (SR) remains challenging, especially in real-world scenarios with complex degradations. Diffusion-based SR methods, particularly those built on Stable Diffusion, leverage strong generative priors but commonly rely on text conditioning derived from semantic captioning. Such textual descriptions provide only high-level semantics and lack the spatially aligned visual information required for faithful restoration, leading to a representation gap between abstract semantics and spatially aligned visual details. To address this limitation, we propose GramSR, a one-step diffusion-based SR framework that replaces text conditioning with dense visual features extracted from the low-resolution input using a pre-trained DINOv3 encoder. GramSR adopts a three-stage LoRA architecture, where pixel-level, semantic-level, and texture-level LoRA modules are trained sequentially. The pixel-level module focuses on degradation removal using ℓ2 loss, the semantic-level module enhances perceptual details via LPIPS and CSD losses, and the texture-level module enforces feature correlation consistency through a Gram matrix loss computed from DINOv3 features. At inference, independent guidance scales enable flexible control over degradation removal, semantic enhancement, and texture preservation. Extensive experiments on standard SR benchmarks demonstrate that GramSR consistently outperforms existing one-step diffusion-based methods, achieving superior structural fidelity and texture realism. The code for this work is available at: https://github.com/aimagelab/GramSR.
Diffusion models have achieved remarkable success in image super-resolution (SR) by leveraging strong generative priors to enhance perceptual quality. However, for textual content, these priors often produce text hallucinations: non-existent or distorted text, which alter scene semantics and compromise model reliability. While recent methods focus on enhancing text generation, they do so indiscriminately, leaving hallucinations fundamentally unresolved. We argue the core problem is a lack of selective control. Therefore, we reformulate text SR as a selective generation problem: text is resolved only when reliable generation is expected, while other scene content is fully super-resolved. We introduce TGSR (Text-Grounded Super-Resolution), a test-time framework that progressively identifies and suppresses likely hallucinated text regions during generation. To enable this, we train dedicated detectors on a large-scale dataset curated from diverse diffusion-based SR outputs. TGSR consistently improves text fidelity across multiple backbones, increasing recognition precision by up to 17.68 points without degrading perceptual quality, underscoring the value of selective control in generative SR.
Zhiming Hu, Ran Zhang, Angela Ye +7
AIC-Toronto, Samsung Electronics · AMD · University of Toronto +2
Large-scale pre-trained diffusion models have been extensively adopted for real-world image Super-Resolution because of their powerful generative priors through textual guidance. However, when super-resolving high-resolution images with patch-wise inference strategy, most existing diffusion-based SR methods tend to suffer from over-generation, due to the misalignment between the global prompt from LR image and the incomplete semantic information of local patches during each inference step. On the other hand, most existing methods also failed to generate detailed texture in local patches due to the overemphasis on global generation capabilities in network designs and training strategies. To address this issue, we present DreamSR, a novel SR model that suppresses local over-generation and improves fine-detail synthesis, thereby achieving visually faithful results with ultra-high-quality details. Specifically, we propose a dual-branch MM-ControlNet, where the ControlNet generates local textual feature with patch-level prompts while the pre-trained DiT provides global textual feature with global prompts, thereby mitigating over-generation and ensuring semantic consistency across patches. We also design a comprehensive training strategy with stage-specific data processing pipelines and a Receptive-Field Enhancement strategy, enhancing the model's capability to capture patch information and effectively restore local textures. Extensive experiments demonstrate that DreamSR outperforms state-of-the-art methods, providing high-quality SR results. Code and model are available at https://github.com/jerrydong0219/DreamSR.