VOSR: A Vision-Only Generative Model for Image Super-Resolution
Organizations: The Hong Kong Polytechnic University · OPPO Research Institute
Abstract
Most of the recent generative image super-resolution (SR) methods rely on adapting large text-to-image (T2I) diffusion models pretrained on web-scale text-image data. While effective, this paradigm starts from a generic T2I generator, despite that SR is fundamentally a low-resolution (LR) input-conditioned image restoration task. In this work, we investigate whether an SR model trained purely on visual data can rival T2I-based ones. To this end, we propose VOSR, a Vision-Only generative framework for SR. We first extract semantically rich and spatially grounded features from the LR input using a pretrained vision encoder as visual semantic guidance. We then revisit classifier-free guidance for training generative models and show that the standard unconditional branch is ill-suited to restoration models trained from scratch. We therefore replace it with a restoration-oriented guidance strategy that preserves weak LR anchors. Built upon these designs, we first train a multi-step VOSR model from scratch and then distill it into a one-step model for efficient inference. VOSR requires less than one-tenth of the training cost of representative T2I-based SR methods, yet in both multi-step and one-step settings, it achieves competitive or even better perceptual quality and efficiency, while producing more faithful structures with fewer hallucinations on both synthetic and real-world benchmarks. Our results, for the first time, show that high-quality generative SR can be achieved without multimodal pretraining. The code and models can be found at https://github.com/cswry/VOSR.
Figures & tables
| Dataset | Setting | Method | Type | PSNR | SSIM | LPIPS | DISTS | AFINE-FR | NIQE | MUSIQ | MANIQA | AFINE-NR | TOPIQ-NR |
| LSDIR | Multi-step | StableSR | T2I | 19.1120 | 0.3751 | 0.5169 | 0.3068 | 0.5477 | 5.4703 | 56.2263 | 0.5345 | -0.7529 | 0.5627 |
| PASD | 19.0817 | 0.4186 | 0.4898 | 0.2433 | -0.1689 | 4.4446 | 65.6496 | 0.5780 | -0.9574 | 0.5827 | |||
| SeeSR | 19.1408 | 0.4198 | 0.4180 | 0.2147 | -0.7413 | 4.2406 | 72.3981 | 0.6616 | -1.0766 | 0.7316 | |||
| DiT4SR | 17.7220 | 0.3833 | 0.4458 | 0.2246 | -0.7300 | 4.2398 | 73.3560 | 0.6854 | -1.1859 | 0.6951 | |||
| ResShift | VO | 19.8723 | 0.4300 | 0.4784 | 0.2699 | 1.0205 | 6.0833 | 59.1250 | 0.5243 | -0.7036 | 0.4966 | ||
| VOSR-0.5B-ms | 19.5677 | 0.4312 | 0.3984 | 0.2126 | -0.8310 | 4.0466 | 73.1033 | 0.6679 | -1.0844 | 0.7231 |
| Multi-step Vision-only | Multi-step T2I-based | One-step Vision-only | One-step T2I-based | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ResShift | VOSR-0.5B-ms | VOSR-1.4B-ms | StableSR | PASD | SeeSR | DiT4SR | SinSR | VOSR-0.5B-os | VOSR-1.4B-os | OSEDiff | PiSA-SR | |
| Params (M) | 174.0 | 619.4 | 1,742.9 | 1,540 | 1,680 | 2,510 | 21,380 | 174.0 | 620.7 | 1,745.7 | 1,760 | 1,290 |
| Time (s) | 0.534 | 1.226 | 2.114 | 10.036 | 2.751 | 3.943 | 12.550 | 0.116 | 0.094 | 0.095 | 0.104 | 0.091 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | LPIPS | MUSIQ |
|---|---|---|
| Teacher (VOSR-0.5B-ms) | 0.3069 | 68.93 |
| Shortcut-based distillation | 0.2913 | 68.21 |
| RC-based distillation | 0.2856 | 69.78 |
| Benchmark | NIQE | MUSIQ | MANIQA | AFINE-NR | TOPIQ-NR |
|---|---|---|---|---|---|
| ScreenSR | 3.7719 | 72.1500 | 0.7187 | -1.2093 | 0.7363 |
| RealSR [ 2 ] | 6.1167 | 57.4564 | 0.6016 | -0.9088 | 0.4140 |
| DRealSR [ 42 ] | 6.7909 | 50.5644 | 0.5588 | -0.7731 | 0.3932 |
| PT | PT | Distill to One-step | |
| Learning rate | |||
| LR scheduler | Constant | Constant | Constant |
| Warm-up steps | 0 | 0 | 0 |
| Training steps | 400K | 400K | 50K |
| Global batch size | 1024 | 256 | 32 |
| Weight decay | 0.01 | 0.01 | 0.01 |
| Method | LPIPS | MUSIQ |
|---|---|---|
| w/o SVE | 0.3011 | 63.74 |
| w/ SVE (CLIP) | 0.2788 | 63.82 |
| w/ SVE (SigLIPv2) | 0.2817 | 64.81 |
| w/ SVE (DINOv3) | 0.2858 | 67.51 |
| w/ SVE (DINOv2) | 0.2872 | 68.23 |
| Guidance design | LPIPS | MUSIQ |
|---|---|---|
| Full condition only | 0.3752 | 67.29 |
| Standard CFG | 0.4053 | 50.78 |
| Ours (partial conditioning) | 0.3772 | 69.26 |
Image Super-Resolution
Image super-resolution reconstructs high-resolution images from lower-resolution observations, balancing detail, fidelity, and computational cost.
See all 247 papers