cs.CVAug 18, 2026
SaveHigh-Flux Count-Free Single-Photon 3D Cameras
Organizations: Boston University
Abstract
Single-photon cameras based on single-photon avalanche diode (SPAD) technology are gaining popularity for 3D sensing, thanks to their extreme sensitivity and time resolution. There are two key challenges with single-photon cameras that limit their widespread use: (i) they suffer from non-linear distortions called ''pile-up'' when operated in high-photon-flux conditions, and (ii) they generate a large volume of raw photon data, creating a severe data bottleneck at each sensor pixel. In this work, we show that while compressive capture techniques successfully mitigate data transfer challenges, they exacerbate the effects of dead-time distortion because they fail to retain sufficient information about the photon detection history to allow post-processing pile-up correction via existing methods. We propose a new computational-imaging method that combines free-running capture with an analysis-by-synthesis software pipeline to mitigate pile-up distortions. Our results with hardware emulations and full-scene and single-pixel simulations show that our method can reliably capture scene distance and reflectance over a wide range of illumination conditions. Our work will enable high-resolution SPAD cameras that are severely bandwidth-constrained to operate in real-world high-flux scenarios.
Figures & tables
Fig. 1: Our method enables dense 3D imaging with dead-time-affected SPCs in memory-constrained settings and high-photon-flux conditions. Without compensation, uncorrected distance estimates (bottom left) suffer from dead-time distortion and lossy compression via a count-free histogrammer. In contrast, our approach extracts depth directly from these compressed, pile-up-distorted measurements, yielding dense 3D maps with orders-of-magnitude higher accuracy. (Bottom numbers report mean error [ME], standard deviation [SD], and 5% inliers).
| Method | Pixel Resources | Robust to Pile-up | High-res. Hardware Compatible |
| High-flux SPL [ 11 ] | High | Yes | No |
| CSPH [ 4 , 5 ] | Low | No | Yes |
| Hist.-less [ 8 ] | Low | No | Yes |
| SplineSketch [ 7 ] | Low | No a | No |
| EDH [ 9 , 10 ] | Low | No | No |
| This Work | Low | Yes | Yes |
TABLE I: Comparison of state-of-the-art. Although many existing techniques deal with the problems of SPC data compression and pile-up separately, there is no single technique that solves these problems jointly and is amenable to in-sensor implementation for dense pixel arrays. ( a The SplineSketch method handles signal peak pile-up, but only over a narrow range of signal flux levels using a look-up table).
Fig. 2: SPC image formation and memory-efficient histogramming techniques. (a) Each SPC pixel operates in tandem with a pulsed laser whose round-trip travel time gives an estimate of the scene distance. (b) The pixel captures the returning photon events as a stream of photon timestamps and attempts to locate the true peak location in the true underlying photon distribution . (c) High-resolution timestamp data consumes too much in-pixel memory. One way to compress it is to use an equi-width histogram with very few bins (8 bins in this case). These wide bins have limited distance resolution and introduce quantization artifacts. (d) An equi-depth histogram with 8 bins is a more efficient representation to track the laser-peak location as seen from a small cluster of bin boundaries around the true peak location. (Figure adapted from [ 10 ] .)
Fig. 3: Effect of dead-time distortion on EDH SPCs. In the absence of dead-time distortions, a compressive capture from a 32-bin EDH provides reliable estimates of the true scene distances. For a realistic dead-time of ns, the scene distance estimates are severely corrupted by pile-up. Due to the non-linearity of pile-up distortions, statistical averaging by increasing the capture time does not help.
Fig. 4: Effect of dead-time on SPC measurements. (a) In case of no dead-time, the binner tracks the median of the photon arrivals, but in case of dead-time, the binner tracks just the detections and suffers from a negative bias due to dead-time distortion. (b) When dead-time = 0, the narrowest bin tracks the peak, but in the presence of dead-time, the narrowest bin no longer tracks the peak of the photon arrivals.
Fig. 5: Markov chain model for an equi-depth histogrammer. (a) The photon detection stream lives on a discrete state space with possible locations; the binner control value lives on a state space of possible locations. The discrete photon detection times can be described using a Markov chain [ 11 ] . (b) A binner’s control value is incremented or decremented with some probability that is tied to the photon detection probabilities relative to the current control-value position. The binner’s control value, therefore, follows another Markov chain, which is driven by the Markov chain of photon detections. (c) An EDH consists of a bank of binners, each tracking a different quantile (shown by vertical dashed lines) of the perceived transient distribution, which may be different from the true transient distribution due to dead-time distortion. The binners converge to their respective stationary distributions that are centered around the actual quantiles.
Fig. 6: Three stages of the analysis-by-synthesis (AbS) pipeline. (1) Initialization stage uses SPC measurement and background flux estimated from passive capture. We also apply a spatial median filter to reduce noise in the initial distance estimate. (2) The optimization stage minimizes a loss function to obtain model parameters that best explain the observed EDH data. (3) A postprocessing stage applies a 3x3 median filter on the distance estimate and effective signal estimate to get the final outputs.
Fig. 7: Single-pixel simulation results. The single-pixel Monte Carlo simulation results, over a varying combinations of and , demonstrate that our 32-bin EDH with dead-time compensation (AbS) method achieves significantly lower distance RMSE (left) and higher 1% inliers as compared to other baselines. The MCHC [ 11 ] , Coates’s correction [ 29 ] , and 5% rule methods use 32-bin EWH measurements and are thus limited by the quantization effects. The histogramless method (HL) performs worse as the background flux increases due to nonlinearity in the measurements.
Fig. 8: Robustness of our AbS pipeline to calibration errors and model mismatch. The AbS pipeline estimates the scene distance and signal flux by assuming known calibrated values of , laser pulse FWHM, and . These simulations demonstrate a gradual increase in distance RMSE (top row) and a slight decrease in 1% inliers (bottom row) with increasing calibration errors in the background estimate (left column), pulse FWHM (middle column), and (right column). The scene distance was fixed at the half the maximum distance range.f The smooth trends across five different illumination conditions demonstrate that our AbS estimator continues to work even in the presence of slight calibration mismatches.
Fig. 9: AbS robustness to non-ideal transients in real hardware data. Our AbS pipeline can handle non-ideal transients such as single-pixel transients from real hardware data. Observe that given the emulated EDH boundaries (vertical lines), there is good agreement between the actual perceived transient and the estimate in both (a) high flux (HF) conditions and (b) low flux (LF). In high-flux conditions, our method reliably predicts the within-peak pile-up and also the “peak shadow” (the dip in the transient’s shape) that immediately follows the peak location, and spans a variable number of time locations depending on the dead-time.
| Dead Time | Distance Error (mm) | KL Div. | ||
| (ns) | 32-bin EWH | 32-bin EDH [Ours] | HF | LF |
| 21 | 285 | 4 | 0.21 | 0.23 |
| 48 | 285 | 12 | 0.15 | 0.18 |
| 81 | 285 | 13 | 0.19 | 0.21 |
| 89 | 285 | 8 | 0.14 | 0.15 |
| 98 | 285 | 9 | 0.17 | 0.17 |
TABLE II: Quantitative results on real hardware. Here we show quantitative comparison of distance error vs. ground truth distance and KL divergence of the recovered transient from the true transient across different sensor dead-times ( ) using real hardware data from Rapp, et al. [ 11 ] . (HF=high flux, LF=low flux)
Fig. 10: Results on 3D scenes with multi-path interference from iToF2dToF dataset. We compare our EDH-based AbS method with standard baselines that use Equi Width Histograms (EWH) and a histogramless approach. Considering resource-constrained scenarios, we keep the number of EWH bins and EDH bins equal for comparison. Quantitative results: mean error (ME, in meters), standard deviation (SD, in meters), and 1% inlier rate (1%in)—are reported beneath each result. As shown, our proposed method significantly reduces distance artifacts and outperforms the baselines. Additionally, we show recovered transients for some selected pixels from each scene, demonstrating how, despite the presence of multi-path interference, our method is able to converge closer to the largest peak in most scenarios.
Supplementary Figure 1: Plots showing empirical histograms of final CV boundaries over multiple runs vs analytical CV stationary distributions. The x-axis in this plot denotes the discretized window locations (length 1000) and the y-axis denotes the probability that the CV is found at a given window location. There is good agreement between the analytical model (dotted lines) and the empirical Monte Carlo simulations of the binner CV (solid jagged lines). The vertical dashed lines denote the true quantile positions, which line up with the CV stationary distributions’ modes.
Supplementary Figure 2: Error plot showing the difference between the mode of the stationary distribution and the quantile of the empirically simulated perceived transient, aggregated over different combinations of distance, signal strength, background strength, and dead-time. We show the spread of errors for each of the 31 quantiles that form a 32-bin EDH. Observe that the errors are quite small (hovering around the zero line), indicating good agreement between theory and empirical simulations.
Supplementary Figure 3: Example loss-function contour maps . We show three different loss functions (negative-log-likelihood, boundary MSE and quantile MSE) in a simulated example. NLL gives the best match between the final optimizer estimate and the true value but has a higher computational overhead. Boundary MSE provides reasonable estimates at slightly higher error. Quantile MSE provides reliable distance estimate but poor signal strength estimates. denotes the starting guess in the AbS pipeline, denotes the final converged position, and denotes the true signal and distance combination used in this specific example. The trajectory of estimates over each iteration of the optimizer is overlaid on the cost-function contours.
Supplementary Figure 4: Optimizer convergence of the AbS/EDH estimator: Each panel sweeps background flux ( SBR – ) for two target distances at one-fourth and one-half the maximum distance range, with a spatial median filter applied to the distance initialization. (Left) Nelder-Mead iteration count distributions; (right) distance RMSE. The combined loss optimization (50-iteration RMSE followed by 500-iteration NLL) maintains sub- RMSE and a tight iteration spread across all conditions, whereas NLL-only degrades catastrophically at for the shallower target (at one-fourth the maximum distance range), where dead-time distortion is most severe. All results use , dead-time , , independent pixels per condition.
Supplementary Figure 5: Depth RMSE vs. laser time period for three scene distances (7.5, 15, 22.5 cm) with no dead-time distortion ( ). The laser pulse width scales with ( ) to maintain constant pulse resolution across all conditions. All methods use 16 bins; photons/cycle, 100 pixels per condition. EDH+AbS (ours) maintains near-constant RMSE as grows, while EWH-based methods degrade proportionally as their bin width widens linearly with the laser time period. The result holds across all three scene distances.
Supplementary Figure 6: Results on 3D scenes with multi-path interference from iToF2dToF dataset We compare our AbS method with standard baselines using 16-bin Equal Width Histograms (EWH) and a histogramless approach. Quantitative results—Mean Error (ME, in meters), Standard Deviation (SD, in meters), and 1% inlier rate (1%in)—are reported beneath each result. As shown, our proposed method significantly reduces distance artifacts and outperforms the baselines. We also show results for signal flux estimation and distance estimated from 5% rule method and MCHC method when operated with high-resolution EWH. The CSPH and DeePEDH show consistent negative bias and lower 1% inliers. Additionally, we show recovered transients for some selected pixels from each scene, demonstrating how, despite the presence of multi-path interference, our method is able to converge closer to the largest peak in most scenarios.
Supplementary Figure 7: Full scene results for scenes from NYUv2 and Middlebury datasets with 0 dead-time. Observe that all methods provide reliable distance estimates, with the only source of noise being Poisson noise.
Supplementary Figure 8: Full scene results for scenes from NYUv2 and Middlebury datasets for dead-time = 75 ns. The longer dead-time and high flux conditions cause strong pile-up artifacts that cannot be addressed using the histogram-less method. Even an “oracle” EDH that has access to ground truth (GT) perceived transient suffers from within-peak pile-up distortions. Our AbS compensation provides reliable distance and signal strength estimates even in this high-flux pile-up regime.
Supplementary Figure 9: Robustness of our AbS pipeline to calibration errors and model mismatch. Our AbS pipeline estimates the scene distance and signal flux by assuming known calibrated values of , laser pulse FWHM, and . The results demonstrate a gradual increase in Mean Error (first column), standard deviation (second column) in distance estimates and in signal estimates (column 3 and 4), with increasing calibration errors in the background estimate (top row), pulse FWHM (middle row), and (bottom row). The smooth trends across five different illumination conditions demonstrate that our AbS estimator does not break under mismatch in the fixed parameters and thus works in other than ideal calibration conditions.
Supplementary Figure 10: Hardware emulation results for real-world SPAD pixel dataset. We show results of transient reconstruction using signal and distance values estimated with our AbS pipeline on real-world photon data streams. Observe that there is good agreement between the actual perceived transient and the estimate over a wide range of dead-times, in both low flux (LF) and high flux (HF) conditions. In high-flux conditions, our method reliably predicts the within-peak pile-up and also the “peak shadow” i.e. the slight dip in the transient’s shape that immediately follows the peak location, and spans a variable number of time locations depending on the dead-time.