Existing real-world super-resolution (RSR) methods based on generative priors have achieved remarkable progress in producing high-quality and globally consistent reconstructions. However, they often struggle to recover fine-grained details of diverse object instances in complex real-world scenes. This limitation primarily arises because commonly adopted denoising losses (e.g., MSE) inherently favor global consistency while neglecting instance-level perception and restoration. To address this issue, we propose InstanceRSR, a novel RSR framework that jointly models semantic information and introduces instance-level feature alignment. Specifically, we employ low-resolution (LR) images as global consistency guidance while jointly modeling image data and semantic segmentation maps to enforce semantic relevance during sampling. Moreover, we design an instance representation learning module to align the diffusion latent space with the instance latent space, enabling instance-aware feature alignment, and further incorporate a scale alignment mechanism to enhance fine-grained perception and detail recovery. Benefiting from these designs, our approach not only generates photorealistic details but also preserves semantic consistency at the instance level. Extensive experiments on multiple real-world benchmarks demonstrate that InstanceRSR significantly outperforms existing methods in both quantitative metrics and visual quality, achieving new state-of-the-art (SOTA) performance.
Figures & tables
Figure 1: t-SNE visualization of intermediate feature representations comparing StableSR and our method. Each point corresponds to a sample, color-coded by its semantic category.
Figure 2: Overview of the proposed InstanceRSR framework. The model integrates instance masks and representation alignment into a DiT-based pipeline, where frozen visual encoders, backbone and semantic guidance jointly enhance instance awareness.
Figure 3: Visual comparison on the RealSR dataset. Competing methods tend to produce geometry distortions, over-smoothing, or noisy artifacts. In contrast, our InstanceRSR restores sharp structures and fine textures with clear boundaries and artifact-free details.
Datasets
Metrics
Real-ESRGAN
ResShift
StableSR
SeeSR
DiffBIR
OSEDiff
DiT4SR
Ours
DrealSR
LPIPS ↓
0.282
0.353
0.273
0.317
0.452
0.297
0.365
0.265
MUSIQ ↑
54.267
52.392
58.512
65.077
65.665
64.692
64.950
66.120
MANIQA ↑
0.490
0.476
0.559
0.605
0.629
0.590
0.627
0.635
ClipIQA ↑
0.409
0.379
0.438
0.543
0.572
0.519
0.548
0.560
LIQE ↑
2.927
2.798
3.243
4.126
3.894
3.942
3.964
4.150
RealSR
LPIPS ↓
0.271
0.316
0.306
0.299
0.347
0.292
0.319
0.250
Table 1: Quantitative comparison of RSR methods on four real-world benchmarks. Our method achieves SOTA performance across four benchmarks.
Figure 4: Ablation study analysis. (a) Representation learning trained with intermediate features f extracted from different layers. (b) Effect of varying sampling steps under the default setting. (c) Impact of representation learning on pre-training efficiency and reconstruction quality.
Case
Quality ↑
Efficiency ↑
Instance Awareness ↑
Case 0 (Default)
1.00
1.00
1.00
Case 1 (w/o Rep. Learning)
0.92
0.65
0.93
Case 2 (w/o Mask Modeling)
0.94
0.90
0.85
Table 2: Ablation study results normalized to the default setting (Case 0 = 1).
Real-world image super-resolution (Real-ISR) requires balancing structural fidelity to degraded observations with realistic detail synthesis. However, existing generative Real-ISR methods often rely on entangled conditioning mechanisms, leading to structural drift or semantically inconsistent details. To address this issue, we propose Visual In-Context Restoration (VICR), a Diffusion Transformer (DiT)-based framework that formulates Real-ISR as image completion. Specifically, we introduce a decoupled visual prior injection mechanism that derives local and global cues from the low-quality (LQ) image: local cues help recover image structures and support high-frequency detail synthesis, while global cues guide overall generation and promote semantic consistency. For ambiguous regions under severe degradation, VICR employs an inference-time agent to refine semantic prompts using visual evidence from the LQ input while keeping model parameters fixed. Experiments show that VICR achieves state-of-the-art performance across multiple Real-ISR benchmarks with only 127M trainable parameters.
Qichang Zhang, Hailong Wang, Baiang Li +4
Faculty of Science and Technology, University of Macau · 2Nullmax · 3Hefei University of Technology +1
Latent diffusion models have recently achieved remarkable progress in Real-world Image Super-Resolution (Real-SR). However, existing methods consistently exhibit a persistent seesaw effect: enhancing perceptual realism compromises structural fidelity. This effect, we argue, is not intrinsic to diffusion models, but largely stems from the architectural constraints of prevailing two-stage frameworks. To address this challenge, we propose PixelSR, a novel single-stage Pixel Diffusion Transformer for Real-SR that operates directly in pixel space. Specifically, PixelSR consists of two core designs: Pixel-space Conditional Diffusion and Orthogonal Flow Decoupling. The former establishes an effective pixel-space restoration framework by integrating spatially aligned LR conditioning, visual semantic guidance, and lightweight texture refinement, thereby bypassing VAE compression and enabling end-to-end optimization directly in RGB space. Complementarily, the latter decomposes the asymmetric flow target into a low-rank semantic component and a noise-free orthogonal texture component. By routing these components to specialized branches, it achieves decoupled optimization with reduced mutual interference. Benefiting from these designs, our PixelSR restores perceptually realistic details while faithfully preserving structural content. Extensive experiments across multiple benchmarks demonstrate that PixelSR not only achieves competitive performance, but also effectively mitigates the fidelity--realism seesaw effect.
Xiang Lv, Kai Zhao, Qiang Wang +3
China University of Petroleum (East China) · Kuaishou Technology · Shenzhen University of Advanced Technology
Generative-prior-based super-resolution (SR) methods leverage implicit knowledge learned from large-scale models to enhance low-resolution inputs. However, such priors are inherently stochastic and optimized for diverse generation, conflicting with the deterministic, high-fidelity reconstruction required for SR. It is also tied to the base model, costly to update, and prone to hallucinations as real-world data evolves. To address these limitations, we introduce WorldSR, a novel framework that augments SR with explicit, entity-aligned visual world knowledge. WorldSR automatically decomposes the input into semantic entities and performs entity-wise retrieval of structurally and semantically aligned images from a large-scale external corpus. For efficiency and reliability, we incorporate a memory mechanism to reuse previously retrieved results and a multi-step filtering process to refine candidate references. The selected references as world knowledge are then integrated through a novel multi-reference super-resolution model, which provides fine-grained constraints for detail synthesis and structural recovery. Experiments on real-world SR benchmarks show that WorldSR consistently improves structural fidelity and visual quality, outperforming existing generative super-resolution methods. Our results suggest that entity-level grounding in external visual world knowledge provides a simple and effective complement to model-bound generative priors.
Junyuan Deng, Xinyi Wu, Zhenyao Wu +3
Honor Device Co., Ltd. · Shenzhen University of Advanced Technology