Capturing high-quality images from only a few detected photons is a fundamental challenge in computational imaging. Single-photon avalanche diode (SPAD) sensors promise high-quality imaging in regimes where conventional cameras fail, but raw quanta frames contain only sparse, noisy, binary photon detections. Recovering a coherent image from a burst of such frames requires handling alignment, denoising, and demosaicing (for color) under noise statistics far outside those assumed by standard restoration pipelines or modern generative models. We present an approach that adapts large text-to-image latent diffusion models to the photon-limited domain of quanta burst imaging. Our method leverages the structural and semantic priors of internet-scale diffusion models while introducing mechanisms to handle Bernoulli photon statistics. By integrating latent-space restoration with burst-level spatio-temporal reasoning, our approach produces reconstructions that are both photometrically faithful and perceptually pleasing, even under high-speed motion. We evaluate the method on synthetic benchmarks and new real-world datasets, including the first color SPAD burst dataset and a challenging Deforming (XD) video benchmark. Across all settings, the approach substantially improves perceptual quality over classical and modern learning-based baselines, demonstrating the promise of adapting large generative priors to extreme photon-limited sensing. Code at https://github.com/Aryan-Garg/gQIR.
Figures & tables
Figure 1: gQIR: Photorealistic single image and burst reconstruction from ultra–high-speed color SPADs. Our pipeline reconstructs high-quality RGB images from 3-bit color-SPAD CFA nano-bursts (left) and merges SPAD photon cubes into temporally consistent bursts (right). From photon-starved inputs captured at 10k–50k fps in extreme, out-of-domain scenes, gQIR recovers sharp textures, accurate color, and coherent structure by leveraging a generative prior. For burst sequences up to 100k fps, FusionViT aligns and dynamically merges quanta latents, outperforming traditional and learning-based methods in both fidelity and perceptualness under motion.
Figure 2: Overview of gQIR. Three-stage framework for quanta burst reconstruction: (S1) a quanta-aligned VAE for joint denoising and demosaicing of SPAD nano-bursts, (S2) an adversarially finetuned LoRA [ 18 ] latent U-Net initialized with stable diffusion [ 47 ] weights for perceptual enhancement, and (S3) a latent burst FusionViT for motion-aware spatio-temporal fusion of burst of nano-burst inputs.
Figure 3: Qualitative comparison – single 3-bit frame reconstructions. Conventional finetuned baselines over-smooth high-frequency structures, especially in distant depth planes and textured regions, whereas gQIR preserves sharper details and more faithful facial features, benefitting from the inclusion of FFHQ faces [ 24 ] in the training set.
Figure 4: Encoder collapse under predegradation removal loss [ 70 , 34 ] . The encoder Eϕ∗ learns a perceptually meaningless shortcut thus producing constant outputs. Since the trainable encoder controls both, the supervision and prediction terms, the training curve quickly converges to a degenerate optimum without our proposed modifications.
Figure 5: Dynamic spatio-temporal latent burst merging. Naive averaging of flow-aligned burst latents yields blur under scene motion. FusionViT instead adaptively weights latents by motion and proximity to the reference, producing a sharper output.
Figure 6: Qualitative comparison – burst reconstruction. We simulate 1:1 GT–SPAD bursts by averaging 77 binary frames per input, preserving the original scene frame rate. QBP yields blurred reconstructions under fast motion due to small burst input while QUIVER breaks down due to motion-blurred nano-bursts, from realistic sampling. Our burst pipeline consistently recovers sharper structure and higher fidelity across extreme motion regimes, from 1000 to 100k fps.
Monochrome
Color
Method
Full-Reference
Non-Reference
Full-Reference
Non-Reference
PSNR ↑
SSIM ↑
LPIPS ↓
ManIQA ↑
ClipIQA ↑
MUSIQ ↑
PSNR ↑
SSIM ↑
LPIPS ↓
ManIQA ↑
ClipIQA ↑
MUSIQ ↑
InstantIR [ 19 ]
10.787
0.178
0.651
0.187
0.346
36.651
7.928
0.101
0.736
0.197
0.358
32.211
ft-Restormer [ 73 ]
28.728
0.816
0.294
0.262
0.435
40.439
26.433
0.739
0.388
0.235
0.395
36.026
ft-NAFNet [ 7 ]
28.276
0.830
0.261
0.300
0.473
39.129
26.881
0.757
0.338
0.251
0.431
36.732
(Ours) qVAE
28.180
0.863
0.327
0.299
0.487
44.895
26.280
0.791
0.435
0.272
0.432
38.613
Table 1: Fidelity and perceptual quality of 3-bit nano-burst input single RGB frame reconstruction. Fine-tuned Restormer and NAFNet attain higher PSNR due to optimization for lower distortion [ 2 ] , leading to oversmoothing, while gQIR achieves higher perceptual quality, consistent with visual results in Fig. 3 .
QBP [ 78 ]
QUIVER [ 8 ]
Burst-gQIR
Test-set (GT fps)
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
XVFI [ 51 ] (1000)
12.014
0.370
0.647
23.001
0.751
0.575
25.822
0.712
0.434
I2-2000fps [ 8 ] (2000)
16.043
0.549
0.468
25.060
0.874
0.366
31.214
0.878
0.296
XD (2k - 100k)
12.780
0.409
0.458
20.096
0.790
0.421
30.331
0.895
0.316
Cumulative:
13.380
0.448
0.496
22.429
0.814
0.429
29.832
0.856
0.330
Table 2: Burst reconstruction fidelity under extreme motion. Our method achieves superior scores due to cleaner flow procsesing and dynamic burst merging while keeping the traditional align-and-merge philosophy aided with a generative prior.
Method
PSNR ↑
SSIM ↑
EMVD [ 38 ]
20.019
0.587
FloRNN [ 30 ]
21.034
0.679
QBP [ 78 ]
21.548
0.703
Transform Denoise [ 3 ]
21.317
0.718
QUIVER [ 8 ]
26.214
0.790
QuDi [ 9 ]
28.641
0.811
Table 3: Burst Fidelity on I2-2k benchmark. Despite the PPP mismatch, our method reaches superior fidelity on I2-2k [ 8 ] .
Figure 7: SD3.5’s VAE solves the text problem [ 4 , 5 ] . SD3.5 [ 14 ] uses a 4× larger latent space than SD2.1. This yields sharper high-frequency details and legible text drawing capabilities.
Variants
PSNR ↑
SSIM ↑
ManIQA ↑
w/o det. encoding (A)
20.56
0.435
0.167
w/o LSA loss (B)
10.39
0.222
0.139
w/o (A) and (B)
10.30
0.218
0.136
Ours
24.78
0.665
0.194
Table 4: Ablation - Stage 1 Design Choices and Losses. Our latent space alignment loss and deterministic sampling gives the highest fidelity in 1 epoch for joint denoising and demosaicing. Both components are critical for meaningful convergence and avoiding catastrophic forgetting shown in Fig. 4 .
Stage
PSNR ↑
SSIM ↑
E∗↓
Alignment (S1)
20.038
0.759
9.088
Perceptual (S2)
24.114
0.846
8.508
Fidelity (S3)
27.630
0.869
8.005
Table 5: Ablation: All stages – fidelity versus temporal stability. Stage 2 improves fidelity over Stage 1 but slightly increases content drift, while Stage 3 provides the best overall trade-off between reconstruction quality and temporal stability.
Figure 8: Real color SPAD reconstructions. Qualitative results on binary bursts captured with a 1Mpx passive color SPAD prototype at 6k fps. Insets show demosaicing via sum-and-average.
Single-photon avalanche diode (SPAD) cameras operate fundamentally differently from conventional cameras due to their photon-counting nature. Each frame produces a binary image: pixels report zero if no photons arrived during exposure, and one if one or more photons arrived. Reconstructing a scene or inferring its properties from a single binary frame is difficult because many different images could produce the same measurement; thus, the inverse problem is fundamentally one-to-many. As we gather more binary measurements, the inherent uncertainty associated with the inverse problem and any associated inference diminishes. With sufficient photon counts, photon noise becomes negligible relative to the signal mean, enabling near-deterministic scene recovery and inference. This work characterizes the transition from stochastic to near-deterministic scene understanding as photon budget increases, analyzing how the stochasticity in photon arrival affects downstream inference tasks. Technically, we develop a conditional generative framework based on a Hypergeometric frame-thinning process for accumulated binary SPAD measurements. Generative models capture the one-to-many nature of photon-starved inverse problems, enabling empirical characterization of how this ambiguity diminishes with increasing measurements and its impact on downstream tasks like character recognition, QR code decoding, and facial analysis.
Optical flow remains challenging in high-speed and low-light scenes, where the limited frame rate and sensitivity of conventional cameras lead to motion blur and underexposure. Single-photon avalanche diode (SPAD) cameras offer single-photon sensitivity and extremely fine temporal sampling. However, individual slices in these high FPS binary photon streams are too sparse for dense correspondence. Temporal aggregation can provide the spatial cues required by optical flow, but accumulating photons at fixed coordinates blurs moving structures. Motion-aware aggregation can reduce this blur, yet it depends on the flow being estimated. To address this dependency, we propose QuantaFlow, the first method for dense optical flow directly from SPAD streams. Instead of constructing a fixed input representation, QuantaFlow embeds SPAD representation construction into iterative flow refinement. At each iteration, the current flow coarsely aligns the slices within the source and target sub-streams. A photon-flux transformation then constructs multi-scale representations containing intensity and structural cues, while adaptive multi-scale fusion balances photon noise and residual motion blur at each pixel. The fused representations drive a feature-warping flow update, and the refined flow guides representation construction in the next iteration. We further construct a synthetic dataset for SPAD optical-flow training and evaluation. Experiments on the synthetic dataset and real-world SPAD data demonstrate the effectiveness and generalization of QuantaFlow.
Single-photon cameras based on single-photon avalanche diode (SPAD) technology are gaining popularity for 3D sensing, thanks to their extreme sensitivity and time resolution. There are two key challenges with single-photon cameras that limit their widespread use: (i) they suffer from non-linear distortions called ''pile-up'' when operated in high-photon-flux conditions, and (ii) they generate a large volume of raw photon data, creating a severe data bottleneck at each sensor pixel. In this work, we show that while compressive capture techniques successfully mitigate data transfer challenges, they exacerbate the effects of dead-time distortion because they fail to retain sufficient information about the photon detection history to allow post-processing pile-up correction via existing methods. We propose a new computational-imaging method that combines free-running capture with an analysis-by-synthesis software pipeline to mitigate pile-up distortions. Our results with hardware emulations and full-scene and single-pixel simulations show that our method can reliably capture scene distance and reflectance over a wide range of illumination conditions. Our work will enable high-resolution SPAD cameras that are severely bandwidth-constrained to operate in real-world high-flux scenarios.