Pretrained image diffusion models can enhance low-light video without low-light training data and without retraining for a new camera. ZVRD, the only published training-free method for low-light video, is too slow to use: our implementation needs over a minute per 256×256 frame. We present a causal, training-free pipeline that adds two temporal components to a frozen MR. Illuminate image prior. Temporal input fusion recursively averages motion-compensated low-light frames before diffusion inversion. Late-start offset propagation runs the full sampling trajectory only on key frames. Each non-key frame inverts the low-noise part of its own trajectory and adds an enhancement offset warped from the latest key frame. At ZVRD's own resolution the pipeline is 261× faster than ZVRD and scores better on all five metrics we report. On 13 held-out SDSD clips at 1280×720, it improves on framewise eight-step processing by 0.55 dB in PSNR and 0.034 in LPIPS and halves the warping error, in 10% less time. Measurements inside the model show why, and a linear-Gaussian model of inversion explains them. Fusion removes frame-to-frame noise before inversion amplifies it. A flow error perturbs the transported offset ten times less than it perturbs a transported state. The perceptual gain does not transfer to low resolution: at 346×260, LPIPS is worse by 0.028. At 0.65 frames per second, the pipeline is an offline method.
Figures & tables
Figure 2: Pipeline overview on real SDSD frames (low-light images shown with a display gain). (a) Temporal input fusion averages the current frame xi with the flow-warped previous fused input, using a per-pixel weight wi (dark: flow unreliable or content changed). (b) A key frame (orange) runs the full trajectory and stores the latent offset Δk at time ts ; a non-key frame (purple) inverts only to ts , adds the warped offset, and denoises from there. Trajectory thumbnails are illustrative, and Δk is drawn in image space for display. (c) A causal exposure gain gives the output. Key frames cost 30 U-Net evaluations and non-key frames 10.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Warp ↓
Flicker ↓
NFE
s/frame ↓
Speedup
ZVRD [ 2 ] (our impl.)
15.81 ∘
0.512
0.312
1.51
0.280
950
90.78
1 ×
MR E15
19.08 ∘
0.681 ∘
0.156
1.00
0.107
30
0.67
135 ×
MR E8
18.91 ∘
0.672
0.163
0.94
0.103
16
0.38
239 ×
Ours
19.18
0.688
0.178
0.51
0.087
12
0.35
261 ×
Table 1: Comparison with ZVRD at 256×256 , six SDSD clips of 30 frames, all methods training-free. MR E15 and MR E8 are framewise MR with 15 and 8 sampling steps. Warp is warping error in units of 10−2 ; NFE is the number of network evaluations per frame; s/frame is measured on one NVIDIA L40. Speedup is the geometric mean over clips of the time ratio to ZVRD. Bold: best in column. ∘ : the 95% paired-bootstrap interval of the difference from our pipeline includes zero. The six clips are design clips ( Sec. 4.1 ); no setting was chosen at this resolution.
Method
Low-light training data
PSNR ↑
SSIM ↑
LPIPS ↓
Warp ↓
Flicker ↓
NFE
s/frame ↓
ReDDiT-2 [ 23 ]
LOLv2 pairs
17.25 ∘
0.617
0.286
1.99
0.166
2
0.47
ReDDiT-4 [ 23 ]
LOLv2 pairs
18.04 ∘
0.582
0.302
2.48
0.184
4
0.87
ReDDiT-8 [ 23 ]
LOLv2 pairs
18.51 ∘
0.609
0.295
2.33
0.187
8
1.67
Zero-TIG [ 24 ]
BVI-RLV videos
19.09 ∘
0.739
0.296
1.10
0.131
n/a
0.12
AGLLDiff [ 27 ]
none
13.35
0.589
0.378
1.39
0.134
49.9
67.45
MR A25 [ 5 ]
none
17.47
0.672
0.233 ∘
1.23
0.136
50
3.81
Table 2: Held-out SDSD clips , 13 clips of 30 frames at 1280×720 . The first four rows use low-light training data and are references; they are not ranked. MR A25, E15, and E8 are framewise MR with 25, 15, and 8 sampling steps; MR state reuse warps the key frame’s inversion state to the other frames ( Sec. 4.2 ). Warp is warping error in units of 10−2 . NFE is the number of network evaluations per frame, averaged over frames (Zero-TIG is not a diffusion model). Bold: best among training-free methods. ∘ : the 95% paired-bootstrap interval of the difference from our pipeline includes zero. All MR rows and ours include the exposure calibration of Sec. 3.5 ; the external methods run as released. Times are single runs on one NVIDIA L40 and indicate magnitude only, except that E8 was timed in the same process as our pipeline. The E15 run does not skip replaced attention layers; with the skip E15 takes 2.44 s (Supp. Tab. 6 ).
Figure 3: Qualitative comparison on frame 15, a non-key frame, of the held-out indoor and outdoor clips whose LPIPS change relative to E8 is the median of their group. Inputs are shown without brightening; insets enlarge the most textured window of the reference (yellow box). † marks methods that require training. Our pipeline shows less residual grain than framewise MR E8 on the shelves and walls; the trained references render brighter images with higher contrast.
wmax
Trajectory
Calib.
PSNR ↑
SSIM ↑
LPIPS ↓
Warp ↓
Flicker ↓
Time ↓
–
E8
yes
17.21
0.674
0.252
1.01
0.126
1.00
–
E15
yes
17.35
0.673
0.235 ∘
1.15
0.132
1.45 ‡
0.8
E8
yes
17.49 ∘
0.707 ∘
0.228
0.50
0.109
1.03
0.8
E15
yes
17.71 ∘
0.710 ∘
0.213
0.54
0.110
1.49 ∗
–
late start, offset
yes
17.34
0.673
0.235 ∘
1.12
0.131
0.87
0.8
late start, key state
yes
17.57
0.696
0.249
0.56
0.112
0.91 ∗
Table 3: Ablation on the 13 held-out clips at 1280×720 . wmax : fusion ceiling (– means no fusion). “Key state” starts non-key frames from the warped key-frame state instead of Eq. 6 . E8 and E15 are framewise schedules with 8 and 15 steps; late start is Sec. 3.4 . Calib.: with exposure calibration. Warp is in units of 10−2 . Time is relative to E8 timed in the same process, except ∗ , a ratio of separate single runs, and ‡ , from the repeated timing of Supp. Tab. 6 . Bold: best in column. ∘ : the 95% paired-bootstrap interval of the difference from our pipeline (last row) includes zero.
Figure 4: Temporal behavior. (a) Temporal residual ∣yi−W(yi−1)∣ on a held-out clip, warped with the reference flow, at a non-key transition (top) and at the first key-frame refresh (bottom), for E8, late start without fusion, our pipeline, and the reference; numbers are mean residuals in units of 10−2 . (b) Warping error per frame index, averaged over the 13 held-out clips, for E8, each component alone, and our pipeline; dotted lines mark key frames. (c) LPIPS of our pipeline minus E8 per frame index on the six scene-cut sequences (solid) and on the same frames without a cut (dashed); the red line marks the cut.
Set
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Warp ↓
Flicker ↓
SDE
MR E15
16.44
0.530 ∘
0.353
2.52
0.202
MR E8
15.98
0.537 ∘
0.344
2.27
0.185
MR state reuse
14.85
0.481
0.416
3.76
0.278
Fusion only
16.01
0.537 ∘
0.371 ∘
1.50
0.153
Late start only
16.55
0.536 ∘
0.351
2.46
0.199
Ours, wmax=0.5
16.61 ∘
0.541 ∘
0.353
2.14
0.188
Table 4: Transfer and stress tests. SDE: 15 scenes of 30 frames at 346×260 . Long: 300 frames at 1280×720 . Stride 2 and 4: the long windows subsampled, 30 frames each. Long: four SDSD scenes outside the design and held-out splits. Scene cut: six sequences built from held-out clips, with a cut between frames 4 and 5; temporal metrics include the cut transition. All rows include exposure calibration. Fusion only: wmax=0.8 on E8. Late start only: Sec. 3.4 without fusion. MR state reuse: Sec. 4.2 . Warp is in units of 10−2 . Bold: best in column within each set. ∘ : the 95% paired-bootstrap interval of the difference from our pipeline includes zero.
Table 5: Roles of the 25 SDSD clips : which clips were used to choose the settings of the temporal components and of the exposure calibration. The temporal components never ran on a held-out clip before their settings were fixed. The E8 schedule was chosen from framewise measurements on the SDSD test scenes.
Method
s/frame ↓
P95 (s)
NFE
Memory (GiB)
MR E8
1.69
1.70
16
6.1
MR E15
2.44
2.46
30
6.4
Ours
1.54
2.49
12
6.4
Table 6: Repeated timing at 1280×720 : the six design clips of Tab. 1 , 30 frames each, three processes with rotated method order on one NVIDIA L40. NFE is the number of network evaluations per frame. P95 is the 95th-percentile frame time. Memory is the peak allocated GPU memory. The three process means differ by at most 0.013 s for each method. Bold: best in column.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Warp ↓
Flicker ↓
NFE
s/frame ↓
MR E8
18.91 ∘
0.672
0.163
0.94
0.103
16
0.38
Fusion only
19.10 ∘
0.683 ∘
0.186
0.50
0.086
16
0.41
Late start only
18.97
0.678
0.156
0.96
0.105
12
0.32
Ours, wmax=0.5
19.08
0.688 ∘
0.156
0.65
0.091
12
0.35
Ours
19.18
0.688
0.178
0.51
0.087
12
0.35
Table 7: Single components at 256×256 , the six design clips of Tab. 1 . Fusion only: wmax=0.8 on E8; late start only: no fusion. Warp is in units of 10−2 ; NFE is the number of network evaluations per frame. Bold: best in column. ∘ : the 95% paired-bootstrap interval of the difference from our pipeline includes zero.
Set
Clips
σ^i
Mean wi
Reliable
SDSD, 1280×720 , 30 frames
25
0.0055
0.755
0.992
SDSD, 1280×720 , 300 frames ∗
6
0.0057
0.760
0.992
SDSD, 256×256
6
0.0059
0.770
0.989
SDE, 346×260
15
0.0022
0.708
0.990
Table 8: Fusion statistics of our pipeline , averaged over non-initial frames and then clips: the scale σ^i of Eq. 3 , the applied weight, and the fraction of pixels with reliable flow. The SDSD rows include design clips. ∗ The four long scenes and two design scenes.
1280×720
256×256
State
Raw
Fused
Reduction
Raw
Fused
Reduction
Input luminance
0.011
0.002
78%
0.005
0.001
74%
VAE latent z0
0.477
0.171
65%
0.168
0.066
59%
Inversion latent, t=67
0.607
0.234
62%
0.222
0.084
62%
Inversion latent, ts=265
0.852
0.364
58%
0.355
0.134
64%
Inversion endpoint, t=925
1.092
0.506
55%
0.491
0.192
63%
Table 9: Relative temporal variation (RTV) of internal states for raw and fused input, six design clips of 30 frames. The attention row gives the range over three latent resolutions and three times, t∈{1,265,925} . Reduction is 1−RTVfused/RTVraw per clip, averaged; it is positive on every clip in every row.
Key age
Mismatch / offset
Mismatch / content
LPIPS gap
1
0.69
0.59
0.023
2
0.85
0.72
0.037
3
0.94
0.80
0.043
4
1.00
0.84
0.049
5
1.03
0.87
0.050
6
1.06
0.89
0.051
Table 10: Content mismatch of state transport by key age (frames since the key frame), six design clips, 18 frames per age. Columns: RMS of ci relative to the RMS of the transported offset; RMS of ci relative to the spatial RMS of the current inversion latent; mean LPIPS of the key-state variant minus that of the offset variant.
Resolution
Input
V and N (median / max)
Band power (median / max)
ρ1 (median / max)
1280×720
Raw
4.2% / 14.0%
3.5% / 11.5%
0.001 / 0.027
1280×720
Fused
3.5% / 14.6%
3.3% / 11.5%
0.002 / 0.027
256×256
Raw
1.6% / 8.3%
1.7% / 4.1%
0.001 / 0.008
256×256
Fused
2.1% / 5.4%
2.1% / 4.8%
0.001 / 0.006
Table 11: Errors of the affine law ( Eq. 24 ) at the 14 interior times of T15 , predicted from the two ends of inversion, six design clips. V : spatial variance; N : temporal-difference energy; band power: eight radial frequency bands; ρ1 : lag-one spatial autocorrelation. Errors are relative for V , N , and band power and absolute for ρ1 . Over all cells the median error is 2.4% for V and N and 2.3% for band power.
State
Energy ratio
Relative response
VAE skip, level 0 (first convolution)
4.00
0.005
VAE skip, levels 1–3
2.20–2.49
0.14–0.18
VAE latent z0
2.21
0.10
Inversion latent, t=67
2.11
0.13
Inversion latent, ts=265
1.88
0.21
Inversion endpoint, t=925
1.75
0.42
Table 12: Response to injected noise , 60 fused frames of the six design clips at 1280×720 . White noise ξ with standard deviation 0.01 is added to the input. The attention row gives the range over three latent resolutions and three times. Energy ratio: median over frames of the response energy for 2ξ over that for ξ ; a linear map gives 4. Relative response: median response energy for ξ divided by the state’s variance.