Capturing high-quality images from only a few detected photons is a fundamental challenge in computational imaging. Single-photon avalanche diode (SPAD) sensors promise high-quality imaging in regimes where conventional cameras fail, but raw quanta frames contain only sparse, noisy, binary photon detections. Recovering a coherent image from a burst of such frames requires handling alignment, denoising, and demosaicing (for color) under noise statistics far outside those assumed by standard restoration pipelines or modern generative models. We present an approach that adapts large text-to-image latent diffusion models to the photon-limited domain of quanta burst imaging. Our method leverages the structural and semantic priors of internet-scale diffusion models while introducing mechanisms to handle Bernoulli photon statistics. By integrating latent-space restoration with burst-level spatio-temporal reasoning, our approach produces reconstructions that are both photometrically faithful and perceptually pleasing, even under high-speed motion. We evaluate the method on synthetic benchmarks and new real-world datasets, including the first color SPAD burst dataset and a challenging Deforming (XD) video benchmark. Across all settings, the approach substantially improves perceptual quality over classical and modern learning-based baselines, demonstrating the promise of adapting large generative priors to extreme photon-limited sensing. Code at https://github.com/Aryan-Garg/gQIR.
Figures & tables
Figure 1: gQIR: Photorealistic single image and burst reconstruction from ultra–high-speed color SPADs. Our pipeline reconstructs high-quality RGB images from 3-bit color-SPAD CFA nano-bursts (left) and merges SPAD photon cubes into temporally consistent bursts (right). From photon-starved inputs captured at 10k–50k fps in extreme, out-of-domain scenes, gQIR recovers sharp textures, accurate color, and coherent structure by leveraging a generative prior. For burst sequences up to 100k fps, FusionViT aligns and dynamically merges quanta latents, outperforming traditional and learning-based methods in both fidelity and perceptualness under motion.
Figure 2: Overview of gQIR. Three-stage framework for quanta burst reconstruction: (S1) a quanta-aligned VAE for joint denoising and demosaicing of SPAD nano-bursts, (S2) an adversarially finetuned LoRA [ 18 ] latent U-Net initialized with stable diffusion [ 47 ] weights for perceptual enhancement, and (S3) a latent burst FusionViT for motion-aware spatio-temporal fusion of burst of nano-burst inputs.
Figure 3: Qualitative comparison – single 3-bit frame reconstructions. Conventional finetuned baselines over-smooth high-frequency structures, especially in distant depth planes and textured regions, whereas gQIR preserves sharper details and more faithful facial features, benefitting from the inclusion of FFHQ faces [ 24 ] in the training set.
Figure 4: Encoder collapse under predegradation removal loss [ 70 , 34 ] . The encoder Eϕ∗ learns a perceptually meaningless shortcut thus producing constant outputs. Since the trainable encoder controls both, the supervision and prediction terms, the training curve quickly converges to a degenerate optimum without our proposed modifications.
Figure 5: Dynamic spatio-temporal latent burst merging. Naive averaging of flow-aligned burst latents yields blur under scene motion. FusionViT instead adaptively weights latents by motion and proximity to the reference, producing a sharper output.
Figure 6: Qualitative comparison – burst reconstruction. We simulate 1:1 GT–SPAD bursts by averaging 77 binary frames per input, preserving the original scene frame rate. QBP yields blurred reconstructions under fast motion due to small burst input while QUIVER breaks down due to motion-blurred nano-bursts, from realistic sampling. Our burst pipeline consistently recovers sharper structure and higher fidelity across extreme motion regimes, from 1000 to 100k fps.
Monochrome
Color
Method
Full-Reference
Non-Reference
Full-Reference
Non-Reference
PSNR ↑
SSIM ↑
LPIPS ↓
ManIQA ↑
ClipIQA ↑
MUSIQ ↑
PSNR ↑
SSIM ↑
LPIPS ↓
ManIQA ↑
ClipIQA ↑
MUSIQ ↑
InstantIR [ 19 ]
10.787
0.178
0.651
0.187
0.346
36.651
7.928
0.101
0.736
0.197
0.358
32.211
ft-Restormer [ 73 ]
28.728
0.816
0.294
0.262
0.435
40.439
26.433
0.739
0.388
0.235
0.395
36.026
ft-NAFNet [ 7 ]
28.276
0.830
0.261
0.300
0.473
39.129
26.881
0.757
0.338
0.251
0.431
36.732
(Ours) qVAE
28.180
0.863
0.327
0.299
0.487
44.895
26.280
0.791
0.435
0.272
0.432
38.613
Table 1: Fidelity and perceptual quality of 3-bit nano-burst input single RGB frame reconstruction. Fine-tuned Restormer and NAFNet attain higher PSNR due to optimization for lower distortion [ 2 ] , leading to oversmoothing, while gQIR achieves higher perceptual quality, consistent with visual results in Fig. 3 .
QBP [ 78 ]
QUIVER [ 8 ]
Burst-gQIR
Test-set (GT fps)
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
XVFI [ 51 ] (1000)
12.014
0.370
0.647
23.001
0.751
0.575
25.822
0.712
0.434
I2-2000fps [ 8 ] (2000)
16.043
0.549
0.468
25.060
0.874
0.366
31.214
0.878
0.296
XD (2k - 100k)
12.780
0.409
0.458
20.096
0.790
0.421
30.331
0.895
0.316
Cumulative:
13.380
0.448
0.496
22.429
0.814
0.429
29.832
0.856
0.330
Table 2: Burst reconstruction fidelity under extreme motion. Our method achieves superior scores due to cleaner flow procsesing and dynamic burst merging while keeping the traditional align-and-merge philosophy aided with a generative prior.
Method
PSNR ↑
SSIM ↑
EMVD [ 38 ]
20.019
0.587
FloRNN [ 30 ]
21.034
0.679
QBP [ 78 ]
21.548
0.703
Transform Denoise [ 3 ]
21.317
0.718
QUIVER [ 8 ]
26.214
0.790
QuDi [ 9 ]
28.641
0.811
Table 3: Burst Fidelity on I2-2k benchmark. Despite the PPP mismatch, our method reaches superior fidelity on I2-2k [ 8 ] .
Figure 7: SD3.5’s VAE solves the text problem [ 4 , 5 ] . SD3.5 [ 14 ] uses a 4× larger latent space than SD2.1. This yields sharper high-frequency details and legible text drawing capabilities.
Variants
PSNR ↑
SSIM ↑
ManIQA ↑
w/o det. encoding (A)
20.56
0.435
0.167
w/o LSA loss (B)
10.39
0.222
0.139
w/o (A) and (B)
10.30
0.218
0.136
Ours
24.78
0.665
0.194
Table 4: Ablation - Stage 1 Design Choices and Losses. Our latent space alignment loss and deterministic sampling gives the highest fidelity in 1 epoch for joint denoising and demosaicing. Both components are critical for meaningful convergence and avoiding catastrophic forgetting shown in Fig. 4 .
Stage
PSNR ↑
SSIM ↑
E∗↓
Alignment (S1)
20.038
0.759
9.088
Perceptual (S2)
24.114
0.846
8.508
Fidelity (S3)
27.630
0.869
8.005
Table 5: Ablation: All stages – fidelity versus temporal stability. Stage 2 improves fidelity over Stage 1 but slightly increases content drift, while Stage 3 provides the best overall trade-off between reconstruction quality and temporal stability.
Figure 8: Real color SPAD reconstructions. Qualitative results on binary bursts captured with a 1Mpx passive color SPAD prototype at 6k fps. Insets show demosaicing via sum-and-average.