One Photon, Many Worlds: Posteriors and Predictions with Single-Photon Cameras

Haejoon Lee; Mohit Gupta; Vijayakumar Bhagavatula; Aswin C. Sankaranarayanan

eess.IVOct 2, 2026

One Photon, Many Worlds: Posteriors and Predictions with Single-Photon Cameras

Authors: Haejoon Lee, Mohit Gupta, Vijayakumar Bhagavatula, Aswin C. Sankaranarayanan

Organizations: Carnegie Mellon University · Carnegie Mellon University, Pittsburgh, PA, USA · University of Wisconsin-Madison · University of Wisconsin-Madison, Madison, WI, USA

Abstract

Single-photon avalanche diode (SPAD) cameras operate fundamentally differently from conventional cameras due to their photon-counting nature. Each frame produces a binary image: pixels report zero if no photons arrived during exposure, and one if one or more photons arrived. Reconstructing a scene or inferring its properties from a single binary frame is difficult because many different images could produce the same measurement; thus, the inverse problem is fundamentally one-to-many. As we gather more binary measurements, the inherent uncertainty associated with the inverse problem and any associated inference diminishes. With sufficient photon counts, photon noise becomes negligible relative to the signal mean, enabling near-deterministic scene recovery and inference. This work characterizes the transition from stochastic to near-deterministic scene understanding as photon budget increases, analyzing how the stochasticity in photon arrival affects downstream inference tasks. Technically, we develop a conditional generative framework based on a Hypergeometric frame-thinning process for accumulated binary SPAD measurements. Generative models capture the one-to-many nature of photon-starved inverse problems, enabling empirical characterization of how this ambiguity diminishes with increasing measurements and its impact on downstream tasks like character recognition, QR code decoding, and facial analysis.

Figures & tables

Figure 1. SPAD cameras operated in binary-frame mode record at most one detection per pixel per exposure. The top four rows shows samples from the conditional posterior given a single (simulated) binary SPAD frame, shown in the first column where each pixel has measured, at most, one photon . In such photon starved regimes, the inverse problem of image reconstruction is one-to-many, leading to many worlds of possibilities in the output. This uncertainty reduces as we increase the number of binary SPAD frames, as seen in subsequent rows. The conditional samples were produced by training diffusion models for each of the four applications (digits, edges, QR codes, and faces), and empirically characterizes how uncertainty in photon arrivals map to uncertainty in reconstruction and inference. In the case of the faces data, notice how different features like sunglasses, gender, facial expression, and identity take different number of frames to resolve correctly.

Figure 2. Example MNIST images simulated with the SPAD sensor model. The maximum expected photon level per binary frame is set to $\tau=0.01$ .

Figure 3. Deterministic restoration under photon starvation. For very small $N$ , the same low-count SPAD observation is compatible with many high-count targets. A deterministic denoiser trained with a L2 reconstruction loss produces a single averaged output, which can be blurry and cannot represent the remaining conditional uncertainty.

Figure 4. The overall process of conditional generator for SPAD image restoration.

Figure 5. Overview of the proposed latent binomial model for a single step. The prior encoder takes in the source SPAD image ${\bm{x}}_{t+1}$ and the frame counts $N_{t}$ and $N_{t+1}$ to predict the parameters of the Gaussian prior distribution of the latent variable ${\bm{z}}$ . The posterior encoder takes in the source SPAD image ${\bm{x}}_{t+1}$ , the increment image ${\bm{d}}_{t}$ , and the frame counts $N_{t}$ and $N_{t+1}$ to predict the parameters of the Gaussian posterior distribution. The decoder with a U-Net architecture takes in ${\bm{x}}_{t+1}$ , the frame counts $N_{t}$ and $N_{t+1}$ , and ${\bm{z}}$ to predict the photon-detection probability map ${\bm{\rho}}_{\theta}$ . During sampling, ${\bm{z}}$ is drawn from the prior, $\widehat{y}_{t,i}\sim\operatorname{Binomial}(M_{t},\rho_{\theta,i})$ , and ${\bm{x}}_{t}={\bm{x}}_{t+1}+\widehat{\bm{y}}_{t}$ . Here ${\bm{z}}$ denotes the set of latent groups described in Section 4.3 .

	$\displaystyle{\bm{x}}_{t}$	$\displaystyle\sim\prod_{i}\operatorname{Binomial}(N_{t},\rho_{i}),$
	$\displaystyle{\bm{x}}_{t+1}$	$\displaystyle\sim\prod_{i}\operatorname{Hypergeometric}(N_{t},x_{t,i},N_{t+1})$

ALGORITHM 1 Training the shared pairwise SPAD reverse model

{\bm{z}}\sim p_{\psi}\left(\cdot\mid{\bm{x}}_{t+1},N_{t},N_{t+1}\right)

ALGORITHM 2 Sampling by anchor-point Markov reversion

Figure 6. Conditional generation results for synthetic edge images. Each column fixes the same conditioning measurement ${\bm{x}}_{k}$ at a given frame count $N_{k}$ , followed by several high-count samples ${\bm{x}}_{1}$ generated conditioned on ${\bm{x}}_{k}$ , their sample mean, and their sample normalized standard deviation.

Figure 7. Conditional generation results on a real SPAD capture of a printed edge target. For each column, the figure shows the generated samples, their sample mean, and their sample standard deviation. At the rightmost corner, we show the photo of the printed target.

Figure 8. Downstream edge-prediction results for generated samples as a function of the number of conditioning SPAD frames

N_{k}

. The red dotted line denotes performance on high-count target SPAD images with

N_{1}=10{,}000

frames.

Figure 9. Conditional generation results for MNIST. For each fixed conditioning measurement ${\bm{x}}_{k}$ , the figure shows generated high-count samples, their sample mean, and their sample standard deviation.

Figure 10. Classification accuracy of generated MNIST samples as a function of the number of conditioning SPAD frames

N_{k}

Figure 11. Conditional generation results with real captures in OCR task. We exhibit the restored grid of digits with selected number of measurements.

Figure 12. Conditional QR-code generation results from the multistep model. Each column fixes the conditioning measurement ${\bm{x}}_{k}$ at a given frame count $N_{k}$ , followed by generated samples, their sample mean, and their sample standard deviation. Each standard deviation map is normalized to have maximum value of 1.

Figure 13. QR-code decoding accuracy of multistep generated samples compared with raw SPAD measurements. The generative model reaches stable decoding accuracy with far fewer conditioning frames.

Figure 14. Facial-geometry errors for generated FFHQ samples. We compare MediaPipe landmarks and head pose estimated from generated samples against those estimated from high-count target images with

N_{1}=100{,}000

frames.

Figure 15. MediaPipe landmark visualizations on generated FFHQ samples and raw SPAD measurements. The generated samples support landmark detection at much lower frame counts than the raw SPAD inputs.

Figure 16. Conditional generation results for real captured face images. For each fixed conditioning measurement ${\bm{x}}_{k}$ , the figure shows generated high-count samples, their sample mean, and their sample standard deviation.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 17. QR-code decoding accuracy of generated samples as a function of the number of conditioning SPAD frames

N_{k}

Figure 18. Comparison of single-step and multistep generated QR-code samples from the same low-count conditioning measurements.

Explore similar work

gQIR: Generative Quanta Image Reconstruction
Feb 23, 2026Aryan Garg, Sizhuo Ma, Mohit GuptaImage Reconstruction Computational Imaging
University of Wisconsin-Madison · Snap Inc.
Fundamental Recovery Bounds for SPAD Signals under Stationary Flux
Jan 12, 2026Lior Dvir, Nadav Torem, Mohit Gupta +1Bayesian Inverse Problems Diffusion Posterior Sampling
Viterbi Faculty of Electrical and Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel · Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI 53706, USA
High-Flux Count-Free Single-Photon 3D Cameras
Aug 18, 2026Kaustubh Sadekar, Vivek K Goyal, David Maier +1Computational Imaging Depth Estimation
Boston University

Feb 23, 2026J/K move · Enter open · S save

gQIR: Generative Quanta Image Reconstruction

Aryan Garg, Sizhuo Ma, Mohit Gupta

University of Wisconsin-Madison · Snap Inc.

Image Reconstruction Computational Imaging Diffusion Models for Image Restoration

Open paper AskPDF arXiv

Capturing high-quality images from only a few detected photons is a fundamental challenge in computational imaging. Single-photon avalanche diode (SPAD) sensors promise high-quality imaging in regimes where conventional cameras fail, but raw quanta frames contain only sparse, noisy, binary photon detections. Recovering a coherent image from a burst of such frames requires handling alignment, denoising, and demosaicing (for color) under noise statistics far outside those assumed by standard restoration pipelines or modern generative models. We present an approach that adapts large text-to-image latent diffusion models to the photon-limited domain of quanta burst imaging. Our method leverages the structural and semantic priors of internet-scale diffusion models while introducing mechanisms to handle Bernoulli photon statistics. By integrating latent-space restoration with burst-level spatio-temporal reasoning, our approach produces reconstructions that are both photometrically faithful and perceptually pleasing, even under high-speed motion. We evaluate the method on synthetic benchmarks and new real-world datasets, including the first color SPAD burst dataset and a challenging Deforming (XD) video benchmark. Across all settings, the approach substantially improves perceptual quality over classical and modern learning-based baselines, demonstrating the promise of adapting large generative priors to extreme photon-limited sensing. Code at https://github.com/Aryan-Garg/gQIR.

Abstract

Explore similar work

gQIR: Generative Quanta Image Reconstruction

Fundamental Recovery Bounds for SPAD Signals under Stationary Flux

High-Flux Count-Free Single-Photon 3D Cameras