Diffusion-based real-world image super-resolution (SR) achieves impressive perceptual quality but inherently suffers from severe texture hallucination. To overcome this limitation, we propose GraftSR, a texture-reference-guided generative SR framework that leverages reference images of the identical instance to anchor the restoration of authentic textures. However, severe spatial misalignment between low-quality inputs and their references poses significant challenges, often leading to ambiguous transfer targets and background feature leakage. To address these issues, GraftSR employs a novel dual-mask reference guidance mechanism that systematically decouples the cross-view texture injection process. By explicitly isolating what authentic textures to extract from the reference and precisely localizing where to apply them within the target, GraftSR achieves robust texture transfer without relying on brittle spatial alignment. Furthermore, to bridge the critical gap in appropriate training data, we construct TexRefSR-141K, the first large-scale dataset providing high-quality reference tuples equipped with complementary spatial masks. Extensive experiments on our newly established benchmark, TexRefSR-Eval, demonstrate that GraftSR sets a new state-of-the-art. Notably, it reduces LPIPS by 20.2% over top-performing baselines, achieving superior reference-faithful restoration.
Figures & tables
Figure 1: GraftSR achieves reference-faithful image SR of fine-grained details ( e.g. , material textures and delicate prints), whereas existing methods exhibit distinct limitations: (a) Diffusion-based SR generates hallucinated textures ; (b) Reference-based SR introduces artifacts beyond matched regions ; and (c) Generative editing model suffers from poor content fidelity .
Figure 2: Overview architecture of GraftSR. Driven by a dual-mask reference guidance mechanism, it extracts mask-modulated reference tokens and region-aware semantic tokens to condition the MMDiT blocks for one-step reference-faithful restoration.
Figure 3: The proposed VLM-driven data curation pipeline. It retrieves identical-instance triplets from multi-view galleries and extracts the complementary masks and unified caption to support authentic texture transfer.
Figure 4: Dataset statistics. (a) Reference distribution over the top-15 leaf categories, with long-tail categories merged as "Others". (b) Number of HQ images per category (bars) and the average number of HQ images per reference (line).
Type
Method
Full-Reference
No-Reference
LPIPS ↓
DISTS ↓
PSNR ↑
SSIM ↑
MUSIQ ↑
NIQE ↓
CLIPIQA ↑
MANIQA ↑
Open-source Generative SR
OSEDiff
0.2326
0.1444
28.3752
0.8471
65.2201
4.8256
0.6631
0.5250
SUPIR
0.2208
0.1321
29.3151
0.8382
61.5577
5.2317
0.5922
0.5225
DiT4SR
0.3816
0.2147
25.6121
0.8071
33.1845
7.3649
0.5769
0.4253
ODTSR
0.1983
0.1048
29.5098
0.8431
69.2601
4.7789
0.6176
0.5129
PiSA-SR
0.2247
0.1343
28.5478
0.8458
66.7754
4.6765
0.6831
0.5338
Table 1: Quantitative comparison on the TexRefSR-Eval-Syn benchmark, where ground-truth references are available. Both full-reference and no-reference metrics are reported. Red bold and blue underline denote the best and the second-best results.
Type
Method
No-Reference
VLM Evaluation (GPT-4.1-global)
NIQE ↓
MUSIQ ↑
CLIPIQA ↑
MANIQA ↑
PQ ↑
TC ↑
Overall ↑
Open-source Generative SR
OSEDiff
3.9329
70.5668
0.6395
0.6292
3.320
1.740
2.530
SUPIR
4.2105
59.6691
0.4778
0.5741
3.380
1.960
2.670
DiT4SR
3.7190
72.5782
0.6428
0.6485
3.720
2.040
2.880
ODTSR
4.2809
71.5261
0.6140
0.6580
2.760
1.560
2.160
PiSA-SR
4.2523
65.5855
0.4913
0.5789
2.840
1.660
2.250
Table 2: Quantitative comparison on the TexRefSR-Eval-Real benchmark with no-reference metrics and VLM-based metrics.
Figure 5: Qualitative comparison on TexRefSR-Eval, including one synthetic case with GT (top) and two real cases (bottom). Our method achieves reference-faithful restoration with superior fidelity and texture consistency. ( Zoom in for best view )
Dataset
Method
PSNR ↑
SSIM ↑
LPIPS ↓
MUSIQ ↑
MANIQA ↑
RealSR
SUPIR
23.65
0.6620
0.3541
62.09
0.5780
DiT4SR
23.50
0.6657
0.3215
67.76
0.6564
OSEDiff
25.15
0.7341
0.2920
69.08
0.6335
PiSA-SR
25.40
0.7418
0.2672
70.14
0.6551
VOSR-1.4B
25.23
0.7175
0.2732
70.58
0.6443
ODTSR
25.07
0.7361
0.2398
68.29
0.6622
Table 3: Quantitative comparison on geneal real-world SR. our GraftSR still achieves strong generalization in both fidelity and perceptual quality even without reference images.
Figure 6: Robustness analysis of GraftSR with absent or mismatched references during inference.
Figure 7: Visual comparison of the ablation studies.
Variant
NIQE ↓
MUSIQ ↑
CLIPIQA ↑
MANIQA ↑
PQ ↑
TC ↑
Ablation on Texture Reference Injection
Cross-attn
4.2210
70.55
0.5236
0.5874
2.660
1.840
Cross-attn + Dual-Mask
3.9818
71.83
0.5974
0.6168
3.000
2.000
Joint-attn
4.4200
71.07
0.5835
0.6278
3.400
2.100
Ablation on Dual-Mask Reference Guidance
w/o zref
4.2911
72.60
0.6222
0.6469
3.500
2.220
Table 4: Ablation studies on the texture reference injection strategy and the dual-mask reference guidance module.
Figure 8: The user study results. Our method is consistently preferred over SOTA ODTSR and Gemini-3-Pro.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: User Study Interface.
Figure 2: More qualitative comparisons on the TexRefSR-Eval-Syn dataset, where low-quality inputs are produced by the synthetic degradation model. Guided by the reference, our method restores textures that are more faithful to the ground truth, recovering fine patterns and structural details that other methods either over-smooth or hallucinate incorrectly.
Figure 3: More qualitative comparisons on the TexRefSR-Eval-Real dataset, which contains real-world low-quality images without ground truth. Our method produces perceptually sharper results whose textures remain consistent with the reference, whereas other methods tend to yield blurry or unfaithful reconstructions.
Figure 4: More qualitative comparisons on general real-world SR benchmarks (RealSR and DRealSR). Although no reference is available in this setting, our method still generalizes well to natural scenes and recovers clean, realistic details that are competitive with or superior to state-of-the-art general SR methods.
Despite recent advances, single-image super-resolution (SR) remains challenging, especially in real-world scenarios with complex degradations. Diffusion-based SR methods, particularly those built on Stable Diffusion, leverage strong generative priors but commonly rely on text conditioning derived from semantic captioning. Such textual descriptions provide only high-level semantics and lack the spatially aligned visual information required for faithful restoration, leading to a representation gap between abstract semantics and spatially aligned visual details. To address this limitation, we propose GramSR, a one-step diffusion-based SR framework that replaces text conditioning with dense visual features extracted from the low-resolution input using a pre-trained DINOv3 encoder. GramSR adopts a three-stage LoRA architecture, where pixel-level, semantic-level, and texture-level LoRA modules are trained sequentially. The pixel-level module focuses on degradation removal using ℓ2 loss, the semantic-level module enhances perceptual details via LPIPS and CSD losses, and the texture-level module enforces feature correlation consistency through a Gram matrix loss computed from DINOv3 features. At inference, independent guidance scales enable flexible control over degradation removal, semantic enhancement, and texture preservation. Extensive experiments on standard SR benchmarks demonstrate that GramSR consistently outperforms existing one-step diffusion-based methods, achieving superior structural fidelity and texture realism. The code for this work is available at: https://github.com/aimagelab/GramSR.
Diffusion-based methods demonstrate significant potential for remote sensing image super-resolution at large scaling factors, particularly in reference-based super-resolution (RefSR), where high-resolution reference images provide critical fine-grained texture priors. However, existing methods often suffer from a trade-off between over-reliance on reference information, which leads to texture artifacts, and under-utilization of such information, which results in insufficient detail recovery. To address these issues, we propose DS-DiT, a Decoupled Siamese Diffusion Transformer that decouples the interaction between low-resolution (LR) and reference (Ref) conditions within the attention mechanism. By allowing LR structural priors and Ref texture information to independently interact with the noisy latent, the framework effectively mitigates competition between the two conditional sources. To further compensate for the limited local modeling ability of global attention, we introduce a Patch-Level Weighting (PLW) module that adaptively modulates the fusion of conditional sources. In addition, the siamese architecture enables an inference-time autoguidance strategy that exploits the prediction discrepancy between strong and weak Ref conditions to improve generation quality without additional training. Experimental results across multiple datasets and scaling factors show that DS-DiT outperforms existing methods in both quantitative metrics and visual fidelity.
Bin Luo, Runmin Dong, Zhaoyang Luo +4
Tsinghua Shenzhen International Graduate School, Shenzhen, China · Sun Yat-sen University, Zhuhai, China · Tsinghua University, Beijing, China +1
Recent real-world image super-resolution (SR) methods often adapt text-to-image (T2I) backbones with ControlNet-style branches or spatial conditioning tokens, which increases memory and computes with resolution and often constrains training to a fixed scale. We propose Fill2SR, which repurposes a masked-inpainting Diffusion Transformer for SR without extra spatial branches. Our Inpainting-Interface Evidence Adapter (IIEA) writes the low-quality (LQ) observation into the native masked-image slot under a full-image mask, turning inpainting into a reverse-degradation conditional rectified flow trained with LoRA-only tuning. We further introduce RCDT, an offline pipeline that distills degradation descriptors from unpaired real images and transfers them onto clean targets using frozen open-source models. Fill2SR supports mixed-resolution training up to QHD and yields stable performance across 512/1024/2048 outputs. On synthetic benchmarks, our base model with IIEA achieves the best LPIPS on DIV2K and LSDIR; adding RCDT trades a small LPIPS drop for consistently stronger no-reference quality on RealLQ250 and RealPhoto60. Fill2SR remains memory-predictable, running 15362 inference on a single 32GB GPU and extending to multi-megapixel outputs via tiled restoration.
Xingfu Yi, Xiaoxue Yu
Independent Researcher, Hangzhou, China · Zhejiang University, Hangzhou, China