Learned real-time supersampling accumulates jittered low-resolution samples along motion vectors. Many methods use a temporal consistency loss to reduce flicker. We show that with accurate registration, this loss can suppress accumulation: on static content, the network copies its warped history at almost no temporal cost. Camera jitter in motion vectors raises this cost and can hide the effect. We identify four registration errors that training can conceal: jitter-carrying motion vectors, one motion field for both resolution lattices, jittered depth, and an unregistered skip path. A model-free oracle shows that correct registration makes freezing two orders of magnitude cheaper on static pixels. Controlled training shows that registration reverses the temporal weight's effect on PSNR. Under exact registration, increasing the weight reduces pixel variance at the expense of accuracy and perceived quality, with a 0.7 to 0.9 dB loss on static frames at weight 0.25. This reversal recurs across three network families and two scales; the loss also reduces accuracy on unjittered STSS data. With a small temporal weight and a normalised sample accumulator, a plain registered network improves the best published QRISP PSNR by 0.34 to 0.62 dB at ×2, ×3 and ×4 upscaling to 1080p. At ×2, its static pixel standard deviation is one quarter that of DLSS 2. Without retraining, it outperforms classical temporal upsampling by at least 4.4 dB on Arm's jittered Bistro capture. Our most accurate ×4 model runs in 2.03 ms per 1080p frame on an H200. Code: https://github.com/CV-AC/Frozen_history
Figures & tables
Figure 1: Registered supersampling. (a) The low-resolution trunk predicts colour, reliability and retention for the normalised output accumulator; the frozen-output limit assumes locally constant accumulated weight. (b) A static reference has small copy cost under R; a large temporal weight then suppresses new samples. (c) The output and sample lattices require different motion fields. (d) Model-free copy cost over chained reprojections. Section 4 details the registration corrections; Section 5.1 derives the copy cost.
Figure 2: Registration and temporal weight. (a,b) Test-set PSNR and static JOD against static pixel STD for λT∈{0,0.05,0.25,1} ; arrows indicate increasing weight. Filled markers: our × 4 network, 45k iterations; hollow diamonds: QRISP-style network, 15k. (c,d) PSNR change and static copy fraction when the weight rises from 0 to 0.25, across architectures, scales and loss forms. R uses full registration; J uses the original misregistered conventions. Under R, the warped ℓ1 term increases copying while reducing accuracy and static JOD. Grey rows test unwarped and squared terms, including larger weights. Section 5.4 defines copy fraction; Supplementary Tables S2 – S5 give full results.
J: Δ PSNR-Y
R: Δ PSNR-Y
Setting
static
moving
all
static
moving
all
Ours, × 4
+0.11
+0.10
+0.09
− 0.86
− 0.24
− 0.27
Ours, × 2
+0.23
+0.20
+0.20
− 0.68
− 0.05
− 0.09
QRISP-style a
+0.31
+0.23
+0.23
− 0.85
− 0.30
− 0.32
Direct prediction
+0.12
+0.12
+0.11
− 0.70
− 0.29
− 0.31
Ours, λT=0.05
+0.05
+0.07
+0.07
− 0.19
0.00
− 0.02
Table 1: PSNR-Y change from λT=0 to 0.25 (last row: 0.05), relative to the same network and pipeline. The warped ℓ1 term costs more accuracy on static than moving frames under R.
Method
Iter.
ms ↓
PSNR-Y ↑
SSIM-Y ↑
LPIPS ↓
FLIP ↓
JOD ↑
tPSNR 60↑
Static PSNR ↑
Static FLIP ↓
Static std
JOD s
× 2 (960 × 540 → 1920 × 1080)
Ours (192 × 8)
100k
—
32.69
0.9246
0.091
0.0570
8.61
33.57
37.57
0.0458
0.096
8.26
DLSS 2 (shipped frames)
—
—
30.12
0.8821
0.151
0.0732
8.22
31.55
36.25
0.0546
0.41
7.96
TAAU (classical)
—
—
29.99
0.8740
0.162
0.0743
7.99
31.09
35.43
0.0568
0.67
7.61
FSR 1.2 [ 1 ] (shipped, spatial) ¶
—
—
29.44
0.8668
0.160
0.0708
7.88
28.49
32.80
0.0585
0.010
7.08
Bicubic (our harness)
—
—
29.31
0.8649
0.217
0.0719
7.81
28.40
32.63
0.0593
0.011
7.01
Table 2: QRISP quality and run time. † Published QRISP/NSRR scores; other metrics use our harness. Static PSNR and STD use the 18-frame School stop; static FLIP uses 97 static-camera frames; JOD s averages three stops. ms: one recurrent × 4 step on an H200, excluding rendering. Metrics and evaluation populations are defined in Section 7.1 . Dashes indicate unavailable metrics or timings.
Figure 3: Reconstructions at × 2, × 3 and × 4, with crop PSNR-Y and F LIP. Crops maximise our PSNR gain within detailed regions ( × 2, × 4 School: luma ≥ 100), against DLSS 2 at × 2, the best baseline at × 3 and × 4 School, and bicubic at × 4 SeaPort. Table 2 reports test-set scores. At × 3 and × 4 the third column is the nearest-neighbour LR input because no DLSS frames are supplied.
Figure 4: Temporal behaviour at × 2. Top: School frame 283 and static STD maps over frames 275–292; right: full-frame STD distributions. Our mean STD is 0.096 versus DLSS 2’s 0.41, with static PSNR 37.57 versus 36.25 dB. Our 90th-percentile STD is 0.24 versus 0.97; 75% of pixels are constant, versus 24% for DLSS 2 and 99.5% for the reference. Bottom: a SeaPort x–t slice across a pan–stop–pan interval. The plotted trajectories show no visible lag; Figure 5 c gives per-frame scores.
Figure 5: Whole-sequence and per-frame evidence at × 2. (a) PSNR-Y and (b) JOD on each test sequence; labels give our margin over the strongest plotted baseline. (c) SeaPort PSNR-Y, with camera stops shaded. Our model leads in PSNR on every SeaPort frame and on 1455 of the 1460 scored QRISP frames. School and SpaceShip abbreviate AbandonedSchool and SpaceShipDemo.
(a) Arm Bistro, jittered, zero-shot
Method
PSNR-Y ↑
SSIM-Y ↑
LPIPS ↓
FLIP ↓
JOD ↑
Bicubic
22.57
0.7743
0.269
0.1576
7.22
Bicubic, registered
24.21
0.8283
0.259
0.1241
7.70
Lanczos-3
22.41
0.7662
0.291
0.1575
7.17
TAAU (classical)
27.16
0.9075
0.111
0.0922
8.58
TAAU, clamp (classical)
27.23
0.9103
0.107
0.0922
8.61
Table 3: Other benchmarks at × 2. (a) Arm Bistro, zero-shot from QRISP, with matched 45k history controls. (b) STSS, retrained on one Lewis file and scored with the authors’ code. Temporal weight reduces PSNR on all four STSS scenes.
Variant
Seed
PSNR-Y ↑
LPIPS ↓
STD s
(a) Ours, × 4, 45k, λT=0.05
Normalised, 30 frames
0
29.74
0.196
0.43
Single predicted blend
0
29.71
0.195
0.53
Normalised, 16 frames
0
29.60
0.208
0.33
(b) History, 45k, λT=0.05 , 30 frames
× 2, recurrent
—
32.58
0.095
0.11
Table 4: Accumulation, history, registration and temporal-weight ablations. Panel (c) removes individual registration corrections; the other panels use full registration. Panel (d) averages two seeds.
Figure S1: Model-free copy cost. PSNR after carrying a frame forward by k chained reprojections: the high-resolution reference against the target reference on the 1080p output lattice (a), and the low-resolution render against the target render on the sample lattice (b), with registered (blue) and misregistered (orange) motion (jitter-carrying on the output lattice, unjittered on the sample lattice), for moving (solid) and static (dashed) camera frames. All hold lengths use a common set of valid pixels. No network is involved. On the output lattice, a one-frame copy reaches 48.7 dB on the static-camera frames with registered motion, compared with 24.2 dB with jitter-carrying motion. On the sample lattice the shipped jitter-carrying motion is the correct one.
Requirement
Measurement
Value
(1) history on the output lattice
copy cost ( k=1 , ℓ1 ) on static pixels, m vs. m~
3.2×10−4 vs. 3.76×10−2 (118 × )
same on the longest camera stop / on moving frames
1304 × / 2.2 ×
held-copy PSNR ( k=1 , all frames), m vs. m~ , × 4 / × 3 / × 2
35.71/25.45, 35.89/26.51, 36.09/28.30 dB
(2) state on the input lattice
previous LR frame carried with m~ (Equation 5 ) vs. with m
28.35 vs. 26.52 dB
(3) unjittered depth
mean ∣ reprojected depth change ∣ , static segment, jittered vs. unjittered
about 103×
(4) registered skip
gain of the registered bicubic skip, moving SeaPort frames (5 frames)
+0.86 to +1.53 dB
Table S1: Model-free measurements quoted in Sections 4 and 5.2 of the paper (QRISP test set, × 4 unless noted; no network involved).
λT
PSNR-Y ↑
tPSNR 60↑
Std s
JOD s↑
α~s
FLIP ↓
R: registered pipeline
0
29.75
30.59
0.77
7.64
0.034
0.0801
0.05
29.74
30.95
0.43
7.55
2.3×10−3
0.0829
0.25
29.48
31.21
0.17
7.26
3.2×10−5
0.0881
1
28.96
31.32
0.086
6.70
9.6×10−7
0.0980
J: misregistered pipeline
Table S2: Registration × temporal weight: our 192×8 network at × 4, trained for 45k iterations. Only the registration pipeline and λT differ. Static standard deviation and sample share use the AbandonedSchool stop; static JOD averages the three static segments. Temporal PSNR uses a shared unjittered motion field. Metric definitions and averaging are given below.
Figure S2: The static AbandonedSchool stop ( × 4) for the R models with λT=0 , 0.25 and 1 and the J model with λT=0.25 : output crops at frame 283, temporal standard-deviation maps over frames 275–292 (8-bit intensity units), and per-pixel median deployed new-sample shares α over the same frames. Numbers below the output crops are crop PSNR-Y and F LIP; static STD (float output) and segment median α are full-frame segment statistics as in Table S2 .
Figure S3: Training dynamics on the grid of Table S2 . Top: mean new-sample share α on training crops. Bottom: fraction with ρ≥0.999 . Left: R; right: J. These training averages are distinct from the deployed static-segment medians.
Δ PSNR-Y
Δ JOD s
copy fraction
Setting
J
R
J
R
J
R
Ours
+0.09
− 0.27
+0.03
− 0.38
0.20 → 0.23
0.85 → 0.99
Ours, × 2
+0.20
− 0.09
+0.01
− 0.14
0.32 → 0.45
0.87 → 1.00
QRISP-style a
+0.23
− 0.32
+0.10
− 0.69
0.11 → 0.16
0.70 → 0.98
Direct prediction
+0.11
− 0.31
+0.01
− 0.34
0.21 → 0.31
0.21 → 0.77
Unwarped term
+0.08
+0.09
− 0.11
− 0.10
0.20 → 0.55
0.85 → 0.96
Table S3: Architecture, scale and loss comparisons corresponding to Figure 2 c,d of the paper. Changes are from λT=0 to 0.25 under J and R. The first four rows use the warped ℓ1 term; the remaining rows test other loss forms and unjittered STSS input.
Run
PSNR-Y
SSIM-Y
LPIPS
tPSNR 60
Std s
JOD
JOD s
α~s
copy
Our network, × 4
R
λT=0
29.75
0.8560
0.177
30.59
0.769
7.71
7.64
0.034
0.855
R
λT=0.25
29.48
0.8432
0.221
31.21
0.174
7.60
7.26
3.2×10−5
0.986
J
λT=0
29.72
0.8528
0.193
30.15
1.361
7.70
7.58
0.61
0.199
J
λT=0.25
29.81
0.8571
0.179
30.28
1.332
7.71
7.61
0.46
0.234
Our network, × 2
Table S4: Every run behind Figure 2 c,d of the paper (R: registered pipeline, J: misregistered pipeline). For the warped ℓ1 term, raising λT from 0 to 0.25 increases the copy fraction under R in all tested architectures and at both scales; the increase is smaller under J. The final block compares alternative loss forms.
J
R
Setting
λT
static (97)
moving (1363)
all
static (97)
moving (1363)
all
Ours, × 4
0→0.25
+0.11
+0.10
+0.09
− 0.86
− 0.24
− 0.27
Ours, × 4
0→0.05
+0.05
+0.07
+0.07
− 0.19
0.00
− 0.02
Ours, × 4
0→1
+0.02
+0.04
+0.02
− 1.98
− 0.74
− 0.80
Ours, × 2
0→0.25
+0.23
+0.20
+0.20
− 0.68
− 0.05
− 0.09
Ours, × 2 a
0→0.05
—
—
—
− 0.20
+0.04
+0.02
Table S5: Change in PSNR-Y from zero temporal weight to the indicated weight under J and R, reported separately for static-camera frames, moving frames and the complete test set.
(a) QRISP-style architecture, QRISP, × 4, 15k
Pipeline
λT
PSNR-Y ↑
tPSNR 60↑
Std s
JOD s↑
α~s
R
0
28.73
29.61
1.25
7.28
0.031
R
0.25
28.41
30.34
0.22
6.59
1.1×10−5
J
0
28.32
28.82
1.98
6.92
0.42
J
0.25
28.54
29.03
1.86
7.02
0.31
(b) STSS Arena, × 2, no jitter, 45k
Table S6: QRISP-style architecture and STSS Arena. (a) Single- α blend, × 4, 15k iterations, with our pipeline and schedule. (b) Arena at × 2 without jitter. These are mechanism comparisons; the QRISP-style scores do not reproduce the published QRISP protocol.
V1. Jitter: ∣ LR − reference at the sample positions ∣
Table S7: Model-free checks of jitter sign, motion convention and the two-lattice correction on the Arm capture. The convention used gives the lowest error among the tested alternatives in each check.
Run
PSNR
SSIM
LPIPS
FLIP
JOD
tPSNR
R, λT=0 ( × 2 grid)
31.12
0.9476
0.041
0.0755
9.22
32.94
R, λT=0.25
30.96
0.9490
0.044
0.0803
9.20
33.78
J, λT=0 , own conv.
31.06
0.9491
0.040
0.0845
9.12
32.10
J, λT=0.25 , own conv.
31.32
0.9517
0.038
0.0814
8.99
32.84
J, λT=0 , jitter-free motion
23.93
0.8081
0.170
0.1503
7.29
24.28
Ours, sRGB space
30.53
0.9565
0.042
—
9.10
32.35
Table S8: Additional Arm Bistro results at × 2, zero-shot. R and J use their own conventions unless noted. Wrong-convention runs lose 6.1 to 12.0 dB relative to the same checkpoint with its own conventions. The checkpoint and input changes are specified below.
Figure S4: Selected × 2 crops on the Arm Bistro capture, zero-shot (top), and STSS Lewis with our STSS-trained model at λT=0.05 (bottom): bicubic, TAAU, our model and the reference. Full reference frames are shown at the left; labels give crop PSNR-Y and F LIP.
Arena
Lewis
SunTemple
Subway
PSNR-Y
FLIP
JOD
PSNR-Y
FLIP
JOD
PSNR-Y
FLIP
JOD
PSNR-Y
FLIP
JOD
Bicubic
33.24
0.0561
8.93
32.83
0.0665
8.25
31.42
0.0804
7.93
29.87
0.0700
7.36
TAAU
32.18
0.0714
8.83
32.90
0.0698
8.31
31.12
0.0862
7.86
30.39
0.0712
7.54
TAAU, clamp
32.11
0.0724
8.81
32.81
0.0704
8.29
31.02
0.0873
7.83
30.37
0.0719
7.54
Ours, λT=0
34.16
0.0710
8.99
37.53
0.0466
9.16
31.64
0.0866
7.98
31.08
0.0703
7.65
Ours, λT=0.05
33.61
0.0875
8.86
37.62
0.0476
9.17
31.61
0.0865
7.93
31.09
0.0710
7.68
Table S9: STSS ( × 2, no jitter) on our protocol, per test scene. Compared with λT=0 , λT=0.25 lowers PSNR on all four scenes, with the largest drop of 3.70 dB on Arena.
Variant
PSNR-Y ↑
Δ
LPIPS ↓
tPSNR 60↑
Std s
(a) R, λT=0.05 , 192 × 8, × 4, 45k
final recipe
29.74
—
0.196
30.95
0.43
single- α blend accumulator
29.71
− 0.02
0.195
30.95
0.53
16-frame training clips
29.60
− 0.14
0.208
30.95
0.33
(b) R minus one piece, λT=0 , 192 × 8, × 4, 15k
control, all pieces
29.31
—
0.203
30.19
0.83
Table S10: Ablations. (a) Components of the final recipe at 45k iterations. (b) One registration component removed at a time, with λT=0 and 15k iterations. Each difference uses the control with the same seed; the failed seed-0 depth run is retained in Table S11 .
Variant
PSNR-Y ↑
Δ
LPIPS ↓
tPSNR 60↑
Std s
control, all pieces
29.31
—
0.203
30.19
0.83
control, all pieces, seed 1
29.38
—
0.223
30.25
0.74
− sample phase
29.22
− 0.09
0.208
30.08
0.81
− registered skip
29.28
− 0.03
0.205
30.14
0.80
− LR re-jitter
29.19
− 0.12
0.210
30.12
0.81
− unjittered depth
div.
div.
div.
div.
div.
Table S11: All registration leave-one-out runs: R, λT=0 , 192×8 , × 4, 15k iterations. This includes the failed seed-0 depth run and complements Table S10 .
Figure S5: Selected crops at × 3 and × 4: bicubic, our classical TAAU, our model and the reference. The full reference frames mark the crop locations, and labels give crop PSNR-Y and FLIP. The × 4 crop lies in the static segment. Selection is described in the text and in Section S16 .
Model
Size
Iter.
Params
GFLOP
H200 ms
PSNR-Y ↑
Measurements from main Table 2
QRISP-M †
f32 / m3
500k
71k
18
0.443
29.06
QRISP-L †
f64 / m5
500k
0.27M
70
0.698
29.42
Ours, shift-MLP
64 × 6
200k
0.17M
38
0.201
29.32
Ours, shift-MLP
64 × 8
200k
0.20M
47
0.301
29.35
Ours
64 × 4
200k
0.36M
89
0.332
29.32
Table S12: Recurrent-step latency at × 4, 1080p output and batch 1, excluding rendering. H200 measurements are identical to the main paper. Daggered PSNR values are published QRISP results.
Figure S6: PSNR-Y and H200 recurrent-step latency at × 4, from Table S12 . Circles: our convolutional trunks; squares: our shift-MLP trunks, both trained for 200k iterations. Diamonds: published QRISP PSNR paired with our re-implementation’s latency. Lines connect configurations within each method.
Δ PSNR-Y (dB)
Model
GPU
Iter.
fp16
int8
int8, worst scene
64 × 4
A100
15k
0.000
− 0.128
− 0.214
64 × 4
H200
15k
0.000
− 0.127
− 0.214
192 × 8
A100
45k
0.000
− 0.352
− 0.489
192 × 8
H200
45k
0.000
− 0.341
− 0.480
Table S13: PSNR cost of TensorRT int8 inference ( × 4): change in PSNR-Y against the float32 model over one unbroken 300-frame pass of each scene.
Model
H200 ms
60 Hz
120 Hz
Break-even ∗
QRISP-M
0.443
3%
5%
2 116 fps
QRISP-L
0.698
4%
8%
1 343 fps
Ours, shift-MLP 64×6
0.201
1%
2%
4 664 fps
Ours, shift-MLP 64×8
0.301
2%
4%
3 115 fps
Ours 64×4
0.332
2%
4%
2 824 fps
Ours 96×4
0.524
3%
6%
1 789 fps
Table S14: Current H200 run time as a share of a frame interval ( × 4, 1080p output). Percentages and break-even thresholds are derived from Table S12 , with exact intervals 1000/60 and 1000/120 ms.
Iter.
λT
Seed
PSNR-Y ↑
LPIPS ↓
Static PSNR ↑
Static std
Med. α
15k
0
0
28.73
0.241
33.59
1.30
0.031
15k
0.05
0
28.69
0.261
33.53
0.64
1.2 ⋅10−3
15k
0.25
0
28.41
0.313
32.66
0.23
1.1 ⋅10−5
45k
0
0
29.05
0.210
33.89
1.16
0.024
45k
0
1
29.04
0.213
33.89
1.12
0.024
45k
0.05
0
28.97
0.233
33.71
0.59
6.9 ⋅10−4
Table S15: The QRISP-style network of Section 5.4 of the paper (QRISP-L’s architecture) trained with our registered pipeline. A small temporal weight nearly halves the static pixel standard deviation at 0.06 dB lower PSNR (45k iterations, two seeds); a large one freezes the accumulator and costs 0.9 dB of static PSNR (15k). The scores are not comparable with the published ones (500k iterations, their pipeline).
Tier
Trunk
Size
lr
PSNR-Y ↑
Δ PSNR
(a) Conventional trunks
S
plain conv (control)
64 × 4
6.0e-4
28.75
—
ECB re-parameterised
64 × 4
6.0e-4
28.98
+0.24
plain conv
64 × 4
2.0e-3
28.98
+0.23
ECB re-parameterised
64 × 4
2.0e-3
28.88
+0.13
U-Net 1-2-1
64/128
6.0e-4
28.76
+0.01
Table S16: Trunk alternatives and learning rates: × 4, 15k iterations, registered pipeline and the same training recipe. Differences use the first control of the same tier, with the medium-tier seed-1 ECB compared against its seed-1 control. Only reconstruction accuracy is reported.
Figure S7: Failure case: particle sprites without motion vectors (SpaceShipDemo/0001, × 4; the R and J grid models at λT=0.05 , Table S19 gives both weights). Both models smear the particle sprites. The R model has larger error in the shown crop (22.1 versus 23.2 dB crop PSNR-Y; the labels give crop PSNR-Y and F LIP).
Method
PSNR-Y ↑
Δ
SSIM-Y ↑
LPIPS ↓
FLIP ↓
JOD ↑
Static std
(a) Stateless and classical temporal upsamplers, QRISP
× 2
Bicubic
29.31
0.8649
0.217
0.0719
7.81
0.011
Lanczos-3
29.17
− 0.14
0.8622
0.233
0.0718
7.78
0.011
Lanczos-3, jittered input
27.95
− 1.36
0.8271
0.237
—
—
—
TAAU (variance clipping)
29.99
0.8740
0.162
0.0743
7.99
0.67
Table S17: Additional rows. (a) Stateless and classical temporal upsamplers on QRISP ( Δ : against bicubic upsampling; for the clamp rows, against the TAAU with variance clipping). (b) Our network with and without history at equal training length (45k iterations), on QRISP and zero-shot on the Arm capture.
PSNR-Y ↑
F LIP ↓
JOD ↑
pixel std (8-bit)
Sequence / segment
Ours
DLSS 2
Ours
DLSS 2
Ours
DLSS 2
Ours
DLSS 2
Ref.
whole sequences (frames 8–299)
AbandonedSchool/0000
25.87
24.58
0.1137
0.1282
7.77
7.35
—
—
—
AbandonedSchool/0001
37.92
35.69
0.0384
0.0486
9.14
8.80
—
—
—
SeaPort/0000
30.04
27.39
0.0525
0.0694
8.40
7.92
—
—
—
SpaceShipDemo/0000
36.83
33.27
0.0400
0.0533
9.23
8.80
—
—
—
Table S18: Ours and DLSS 2 at × 2 on every test sequence and every static segment. Ours is higher in PSNR-Y and JOD and lower in F LIP on all five sequences, and higher in JOD, lower in F LIP and lower in pixel standard deviation on all three static segments.
Abandoned-
SpaceShipDemo
λT
School
SeaPort
all
0000
0001
Mean
0
R
28.95
27.59
32.72
33.96
31.47
29.75
J
28.86
27.44
32.85
33.87
31.83
29.72
R − J
+0.10
+0.15
− 0.13
+0.09
− 0.35
+0.04
0.05
R
28.94
27.55
32.72
33.81
31.63
29.74
J
28.91
27.49
32.95
33.99
31.91
29.78
Table S19: Per-scene PSNR-Y of the R and J × 4 grid models (192 × 8, 45k iterations), with the two SpaceShipDemo sequences.
Method
static (97)
moving (1363)
all
Ours (192 × 8, 100k)
0.0458
0.0587
0.0570
DLSS 2 (shipped frames)
0.0546
0.0754
0.0732
TAAU (classical)
0.0568
0.0760
0.0743
Bicubic
0.0593
0.0731
0.0719
Table S20: F LIP on static-camera and moving frames ( × 2, QRISP test set).
Trunk
λT
Clip
Seed 0
Seed 1
∣Δ∣
96 × 8
0.05
30
29.164
29.168
0.003
192 × 8
0.25
16
28.818
28.827
0.009
192 × 8
0.05
16
29.277
29.266
0.011
192 × 8 b
0.05
30
29.424
29.398
0.026
96 × 8, ECB
0.05
30
29.375
29.346
0.028
192 × 8
0
16
29.312
29.376
0.064
Table S21: Seed pairs: every pair of 15k-iteration runs ( × 4) whose configurations differ only in the seed.
Figure S8: Per-frame PSNR-Y (top) and FLIP (middle) on SeaPort/0000 at × 2, scored frames 8–299: ours, DLSS 2, TAAU and bicubic. Grey bands mark camera stops; the bottom strip shows median camera motion. The legend gives sequence means.
Figure S9: Gallery at × 2 (selected crops). Each row shows the full reference frame with the crop box, bicubic upsampling, our classical TAAU, DLSS 2 (shipped frames), our model and the reference; below each crop is its F LIP error map (0 to 1), labelled with crop PSNR-Y and F LIP. Our model recovers thin and periodic structure (branches, lettering, floor gratings) where the baselines alias, blur or break it up; in the SeaPort frame-129 crop, TAAU produces fragmented lettering and scores 14.7 dB PSNR-Y.
Figure S10: Gallery at × 3 (selected crops), as in Figure S9 ; the QRISP dataset provides DLSS 2 frames only at × 2.
Figure S11: Gallery at × 4 (selected crops), as in Figure S9 . At × 4, the input has one low-resolution pixel per 16 output pixels. In these selected crops, our model reconstructs grass blades, branches and floor gratings with less blocking and moiré than bicubic upsampling and TAAU.
Latent world models that integrate a flow in a frozen self supervised latent space train stably and cheaply, yet silently lose the property manipulation depends on most: motion. The pretrained flow never moves the manipulated object; retraining it with latent-only losses only trades stillness for teleport-like motion. We trace the failure to the training signal, not the representation: anchor-sparse, latent-only supervision never says where along the horizon change belongs. Decode-augmented rollout training (DART) repairs this while keeping the representation frozen, retraining only the flow with decode-path supervision. DART outperforms its latent only parent on the full protocol, restores the temporal structure of motion, and re-couples predicted motion to the scene; at larger scale it further improves prediction quality, closing nearly half the remaining gap to an oracle-informed interpolation reference. Finally, we report an unexpected finding about evaluation: pixel error alone rewards frozen predictions.
Xiwen Chen, Rigaudiere Z. Li, Zhiruo Zhou +2
Shenzhen International Graduate School, Tsinghua University · Shanghai Jiaotong University · Wuhan University of Technology
Large generative models can recover realistic detail in real-world video super-resolution (VSR), but processing an entire video with them is computationally expensive. In this work, we present RelayVSR, a streaming VSR framework built on the Sparse Generative Relay mechanism. A large generative model generates reference latents for sparse keyframes, while a lightweight VSR network uses these references and low-resolution video to super-resolve every frame. The lightweight VSR network, implemented as a Dual-Memory Video Transformer, reuses keyframe information across frames and updates recent video context, supporting first-keyframe conditioning and dual-endpoint conditioning with bounded lookahead. However, errors in shared keyframes can propagate and accumulate across output frames, making keyframe quality alone an insufficient optimization target. We address this collaboration gap with Video-Aware Reference Optimization (VARO), which uses reinforcement learning to update the large generative model with two reward levels: a system-level reward evaluates videos produced by the fixed lightweight VSR network, while a reference-level reward evaluates decoded keyframe quality. VARO improves final video quality over direct joint training, and its dual-level rewards outperform a system-level reward alone. At 1080p on a single NVIDIA A100 80GB, dual-endpoint RelayVSR with a 15-frame keyframe interval reaches 29.29 FPS, 13.82 GB peak GPU memory, and 0.327 s first-frame model latency, compared with 7.80 FPS, 24.447 GB, and 2.83 s for FlashVSR-Tiny. The code is available at https://github.com/kopperx/RelayVSR.
To fit diverse display and bandwidth constraints, high-frame-rate videos are temporally downscaled to low-frame-rate (LFR) and later upscaled, requiring joint optimization for effective frame-rate rescaling. However, existing methods typically link the two operations via training objectives, without fully exploiting their reciprocal nature, which may cause high-frequency information loss. Moreover, they overlook the impact of lossy codecs on LFR videos, limiting real-world applicability. In this work, we propose an end-to-end framework for compression-aware frame-rate rescaling, named TVRN. To regularize high-frequency information lost during frame-rate downscaling, TVRN adopts an invertible architecture that combines a Multi-Input Multi-Output Temporal Wavelet Transform with a high-frequency reconstruction module. To enable end-to-end training through non-differentiable lossy codecs, we design a surrogate network that approximates their gradients. Finally, to improve robustness under various compression levels, we extend TVRN to an asymmetric architecture by incorporating compression-aware features learned via a learning-to-rank strategy. Extensive experiments show that TVRN outperforms existing methods in reconstruction quality under industrial video compression settings. Source code is publicly available at https://github.com/fengxinmin/TVRN_public.
Xinmin Feng, Li Li, Dong Liu +1
MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei 230093, China