Organizations: Department of Electrical Engineering, National Chung Hsing University, Taichung 402202, Taiwan · College of Computing and Informatics, Providence University, Taichung 433303, Taiwan · ASUSTeK Computer Inc., Taipei 112019, Taiwan
Super-resolution networks are trained for one magnification factor, so several output resolutions require several models. We propose PMS-SR, a progressive single-input multiple-output network that produces x2, x4, and x8 results in one forward pass, with a Transformer backbone on the low-resolution grid. In Stage 1, each scale is predicted as a residual over the detached bicubic upsampling of the previous scale's prediction. With Dirac and zero initialization, training starts exactly from the bicubic cascade, and each head adds only the band the previous scale lacks. In Stage 2, inter-scale mutual refinement couples the scales in both directions through space-to-depth feedback, deformable alignment, and per-pixel fusion, so x8 features refine the x2 output. Trained on DF2K, the 7.95M-parameter model achieves a higher PSNR than every compared multi-scale method at x4 and x8 on all five standard benchmarks. At x8, it outperforms the x8-specific DRN-L on all of them, leads all compared methods on Urban100, and is within 0.13 dB of the best result elsewhere. Against three single-scale SwinIR-M models, PMS-SR uses 22% of the parameters, runs 2.9x faster on BSD100 inputs, and its outputs agree 6.7-12.4 dB more closely. Code and pretrained models will be released at https://github.com/julian135707/PMS-SR upon acceptance.
Figures & tables
Fig. 1: Overall architecture of PMS-SR. All attention runs on the LR grid (blue). Stage 1 (orange) expands the features ×2→×4→×8 with decreasing widths; the ×2 output is a residual over the bicubically upsampled input, and each higher scale is a residual over the detached, bicubically upsampled prediction of the previous scale (dashed cascade). Stage 2, the IMR module (red, Fig. 3 ), refines all scales jointly; the heads of both stages are green. The bottom row shows the actual input and outputs of one forward pass at their true relative sizes.
Fig. 2: One CSWin block of PMS-SR (Eqs. ( 3 )–( 5 )): two parallel stripe attentions (horizontal and vertical, half the channels each) plus a channel attention block (CAB) scaled by a learnable α , followed by a LeFF feed-forward network. Blue boxes are adopted from prior work without structural change: stripe attention with LePE [ 22 ] , the CAB of HAT [ 8 ] , and LeFF [ 54 ] .
Fig. 3: The inter-scale mutual refinement (IMR) module, Stage 2 (Eqs. ( 10 )–( 12 )). Fine features are folded back by space-to-depth ( PU↓r ), coarse features are aligned to the finer grid by deformable alignment ( DAs ), and the streams are fused per pixel ( SCFs ). The ×8 row refines entirely on the ×4 grid and expands only the correction (refine-then-upsample). All refinements are residual.
Method
Scale
Params
Set5
Set14
BSD100
Urban100
Manga109
(M)
PSNR / SSIM
PSNR / SSIM
PSNR / SSIM
PSNR / SSIM
PSNR / SSIM
Scale ×2
LapSRN [ 11 ]
×2
1.31
37.52 / 0.959
33.08 / 0.913
31.80 / 0.895
30.41 / 0.910
37.27 / 0.974
MSWSR [ 13 ]
×2
1.23
37.49 / 0.9582 †
33.23 / 0.9123
31.88 / 0.8929
31.14 / 0.9169
37.32 / 0.9733
MLWAN [ 14 ]
×2
1.23
37.68 / 0.9589
33.47 / 0.9227
31.96 / 0.8974
31.34 / 0.9309
37.39 / 0.9733
MS-LapSRN [ 12 ]
×2
0.22
37.78 / 0.960
33.28 / 0.915
32.05 / 0.898
31.15 / 0.919
37.78 / 0.976
TABLE I: Comparison with multi-scale super-resolution methods (PSNR / SSIM, Y channel). Best in red , second best in blue ; PMS-SR+ (self-ensemble) and the DIV2K-only control are not ranked. † = measured by us from the authors’ released weights (Sec. IV-C ). CGSSR LTE is an arbitrary-scale model, listed with its published PSNR.
Method
Scale
Params
Set5
Set14
BSD100
Urban100
Manga109
(M)
PSNR / SSIM
PSNR / SSIM
PSNR / SSIM
PSNR / SSIM
PSNR / SSIM
EDSR [ 5 ] (2017)
×2
40.7
38.11 / 0.9601
33.92 / 0.9195
32.32 / 0.9013
32.93 / 0.9351
39.10 / 0.9773
RCAN [ 6 ] (2018)
×2
15.4
38.27 / 0.9614
34.12 / 0.9216
32.41 / 0.9027
33.34 / 0.9384
39.44 / 0.9786
SAN [ 71 ] (2019)
×2
15.7
38.31 / 0.9620
34.07 / 0.9213
32.42 / 0.9028
33.10 / 0.9370
39.32 / 0.9792
SwinIR-M [ 7 ] (2021)
×2
11.8
38.42 / 0.9623
34.46 / 0.9250
32.53 / 0.9041
33.81 / 0.9427
39.92 / 0.9797
EDT [ 30 ] (2023)
×2
11.5
38.45 / 0.9624
34.57 / 0.9258
32.52 / 0.9041
33.80 / 0.9425
39.93 / 0.9800
TABLE II: Reference comparison with single-scale (SISO) models, each trained per factor (Params is the per-factor size), against one 7.95M PMS-SR covering all three factors in one pass ( ∗ ). Best in red , second best in blue ; PMS-SR+ (self-ensemble) is not ranked. † = measured by us from the authors’ released weights (Sec. IV-C ); ≈ = size reported for the ×4 model.
Fig. 4: Visual comparison at ×8 : (a) BSD100 violinist, (b) Urban100 img_043, (c) Urban100 img_025, and (d) Manga109 GinNoChimera . Full-image PSNR (dB) / SSIM is given below each panel, and the best value of each metric is shown in bold red .
Fig. 5: Visual comparison at (a) ×8 on Manga109 ( YukiNoFuruMachi ), (b) ×4 on Urban100 img_011, and (c) ×4 on Urban100 img_052. Protocol and highlighting are the same as in Fig. 4 .
Scale
Variant
Set5
Set14
BSD100
Urban100
×2
w/o IMR
38.14 / 0.9611
33.93 / 0.9198
32.30 / 0.9001
32.95 / 0.9361
Full
38.16 / 0.9611
33.95 / 0.9200
32.31 / 0.9002
33.03 / 0.9366
×4
w/o IMR
32.64 / 0.9006
28.96 / 0.7906
27.84 / 0.7458
27.15 / 0.8154
Full
32.72 / 0.9012
29.00 / 0.7915
27.87 / 0.7464
27.25 / 0.8177
×8
w/o IMR
27.43 / 0.7900
25.34 / 0.6535
25.05 / 0.6083
23.27 / 0.6549
Full
27.49 / 0.7922
25.37 / 0.6551
25.07 / 0.6094
23.36 / 0.6593
TABLE III: Test-time ablation of the IMR module: Stage-1 outputs (w/o IMR) vs. the full model, identical trained weights (PSNR / SSIM, Y channel; better in red , ties unmarked).
Fig. 6: Effect of IMR at ×8 : (a) HR image with the zoomed region, (b) Stage-1 output (without IMR), (c) full model (with IMR), and (d) the correction ∣I^8(2)−I^8(1)∣ on the Y channel, which concentrates along edges and texture transitions.
BSD100
Set5
Variant
×2
×4
×8
×2
×4
×8
Full model (50k reference)
32.19
27.75
24.98
38.00
32.47
27.26
(a) w/o Stage 2 (IMR), from scratch
32.20
27.74
24.96
38.01
32.46
27.24
(b) w/o channel attention branch
32.19
27.73
24.96
38.01
32.47
27.23
(c) direct bic↑s anchor
32.21
26.76
22.60
38.03
29.86
22.80
(d) cascade without detachment
32.21
27.75
24.99
38.01
32.55
27.29
TABLE IV: Further ablation axes, trained from scratch for 50k iterations (Y-PSNR, dB). Drops of at least 0.2 dB vs. the 50k reference in bold ; all others are within ±0.1 dB.
Method (ladder)
Params
FLOPs
Runtime
SwinIR-M ×2 [ 7 ]
11.75M
48.05G
30.3 ms
SwinIR-M ×4
11.90M
50.55G
31.5 ms
SwinIR-M ×8
12.05M
60.55G
32.6 ms
SwinIR-M ladder (3 models, 3 passes)
35.70M
159.15G
94.4 ms
Derived ladder (SwinIR-M ×8+ bicubic ↓ )
12.05M
60.55G
32.6 ms
PMS-SR (1 model, 1 pass, whole image)
7.95M
142.17G
32.8 ms
TABLE V: Cost of producing the full ×2/×4/×8 ladder for one input. FLOPs are counted by thop for one 64×64 LR input; runtimes are averaged over the 100 BSD100 ×8 LR inputs (about 60×40 ) on an RTX 4090. Tiled: 48×48 tiles with overlap 16.
Fig. 7: Mean ×8 PSNR over the five benchmarks vs. the total parameters of the models that deliver ×2 , ×4 , and ×8 (Tables I , II , and V ). Single-scale methods count as three models; where one size is reported, it is tripled. DRN-L has no ×2 model. The hollow marker is the derived ladder.
Agreement (dB) ↑
Fidelity (dB) ↑
Benchmark
System
×8→×4
×4→×2
×2
×4
×8
Set5
Bicubic ladder (control)
62.48
55.16
27.72
25.36
24.40
Derived ladder (control)
∞
∞
33.24
29.32
27.59
SwinIR-M (3 models)
40.52
42.54
32.89
29.33
27.59
PMS-SR (1 model, 1 pass)
52.36
50.21
33.09
29.19
27.49
Set14
Bicubic ladder (control)
62.10
55.32
27.73
24.74
23.19
TABLE VI: Single-input deployment: each system receives the same ×8 LR input and delivers all three renditions (whole-image inference). Italic rows are degenerate controls; the best fidelity per column (controls excluded) is in red .
Chinese Academy of Sciences Changchun Institute of Optics Fine Mechanics and Physics, Changchun, Jilin 130033, China · University of the Chinese Academy of Sciences, Beijing 100049, China