Non-line-of-sight (NLOS) imaging reconstructs scenes hidden around a corner from indirect light recorded by a single-photon avalanche diode (SPAD). A single reconstruction is a large inverse problem: billions of photon timestamps must be binned, moved through memory, transformed and inverted. As SPAD arrays raise acquisition throughput, reconstruction becomes the limiting stage. We rebuild the GPU execution of two established wave-based algorithms, f-k migration and phasor-fields, for both streaming and offline processing. On the phasor-fields side we assemble the ring-and-radius kernels of previous work once and offline, using the analytic Fourier transform of a ring, so the propagation kernel never exists in dense form at runtime, reducing the memory and bandwidth. We reorganize the pipeline of both algorithms with fused kernels, warp-level photon binning, batched transforms, CUDA graph replay, and FP16 storage applied only where it reduces the actual bottleneck. Our implementations are up to 42x faster than the reference streaming pipeline and up to 14x faster than the fastest published GPU baseline, all while using a fraction of the memory (down to 2.5%), enabling vastly larger and finer reconstructions on the same hardware, or comparable ones within a much lower memory budget. We report an ablation of each implementation choice and propose three denoising strategies enabled by the resulting frame budget for next-generation NLOS video processing.
Figures & tables
Figure 1: Performance overview on the office scene, reconstructed at two volume sizes: a dense 720×520×512 , and a coarser 360×260×512 which therefore shows fewer details in the reconstruction. For each, the left panel reports reconstruction time per frame and the right panel peak video memory, with the methods grouped into three families: backprojection ( Sun et al., 2026 ) , f–k migration, and phasor-fields ( Nam et al., 2021 ; Mu et al., 2025 ) . Hatched bars are previous work and solid bars are ours. Backprojection is three to four orders of magnitude slower than the rest, so its bar runs off the top of the time panel. Against PttR, the fastest published GPU baseline, our f–k is 2.7× faster at the coarser volume and our phasor-fields about 14× , in a fraction of the memory; at the dense volume neither phasor-fields baseline fits in the 16GB of our GPU at all ( OOM ).
Compared work
Model
Comparison use
Main limitation for comparison
Fast backprojection ( Arellano et al., 2017 )
Backprojection
Rerun (partially modified)
Different reconstruction model; costly for many ellipsoids
CUDA irregular-relay NLOS ( Sun et al., 2026 )
Backprojection
Rerun
Different reconstruction model
NLOS at 5 FPS ( Nam et al., 2021 )
Phasor-fields
Rerun
Real-time but low-resolution setting
FPGA-accelerated NLOS ( Liao et al., 2022 )
Phasor-fields
Reported only
Specialized hardware; code unavailable
Physics to the Rescue ( Mu et al., 2025 )
Phasor-fields / learned
Rerun
Highest FPS reported for single-depth reconstruction
f–k migration ( Lindell et al., 2019 )
f–k migration
Offline reference
Original offline implementation
Table 1: Previous NLOS imaging methods considered in our evaluation. We distinguish between baselines rerun in our hardware and methods included as reported literature values. Partially modified denotes code changes required to run the method on our data, mainly for loading HDF5 capture datasets. Every method is run with its own published reconstruction operator.
Figure 2: Illustration of the capture setup and the collected data. An ultrafast laser emits a pulse onto the visible relay wall, from which light scatters toward the hidden scene. The scattered light then returns to the relay wall and is captured by the SPAD array as third-bounce photons. The recorded measurements consist of individual photon arrival events, which can be aggregated into transient histograms or directly accumulated into temporal-frequency phasors for wave-based reconstruction methods such as phasor-fields.
Figure 3: Overview of our producer-consumer pipeline. Labels P and C indicate whether a module is a producer, a consumer, or both.
Figure 4: Histogram construction time for DMA and GPU-resident buffers using different photon counts. By default, histograms are built with 1.2M photons, matching the binary datasets of Nam et al. (2021) ; the 20M and 100M cases are generated by uniformly oversampling this baseline.
Figure 5: Two scenes reconstructed with and without padding using f–k . The reconstruction on the left side does not show artifacts without padding, whereas the right one does; however, the hidden scene remains recognizable and the versions without padding were computed at higher speed.
Figure 6: Performance and video memory footprint of f–k with different padding and precision configurations, measured on a confocal dataset shaped as 256×256×256 .
Figure 8: Mean radial magnitude of the dense phasor-fields Fourier-domain kernel and of our radial approximation, averaged over temporal frequencies and depths. The comparison is one of shape rather than of exact values.
Figure 10: Denoising results from DDA on a dynamic capture, for temporal windows of Nw frames. Each row is one frame of the sequence, with the corresponding pose rendered at the left; the leftmost column reconstructs that frame without denoising. All panels of a row show the same frame, and the inset magnifies the same region in every panel.
Figure 11: Merging reconstructed frames in the complex domain on a simulated scene. Weighting the coherent mean by Ci(xv) attenuates the incoherent background speckle while preserving the hidden surface, whose phase is consistent across the window.
Figure 12: Merging phasor-fields and f–k migration with MoE. The inset magnifies the same region through a display tone curve ( γ=2.2 ).
Figure 13: Ablation of implementation decisions: (a) the photon-binning path, (b) the shared-memory Stolt-remapping variant of f–k migration, and (c) replaying a captured CUDA graph instead of launching each operation separately.
Approach
Mvox/s ↑
Time (s) ↓
VRAM (GB) ↓
RTX 4080 SUPER - Z : 2562×512 , 255 frequencies, 30 depths
Our f–k
4,846.80
0.0069±0.0000
0.750
Our phasor-fields
9,577.03
0.0035±0.0000
0.293
Phasor fields ( Nam et al., 2021 )
521.67
0.0643±0.0042
4.389
Phasor fields ( Mu et al., 2025 )
1,242.38
0.0270±0.0023
8.851
CUDA backprojection
1.21
26.593±0.377
0.366
Table 3: Performance of fast reconstruction algorithms. Throughput is the processed data volume divided by reconstruction time: for the wave-based methods this is the input transient cube of each block header, of which the phasor-fields family propagates the listed frequency and depth counts, while backprojection uses its reconstructed volume, 256×256×491 voxels on Z and 360×260×512 on office. The frequency counts are chosen from the virtual wavelength and capped at the Nyquist limit of the temporal sampling. Green and red cells mark the best and worst value per metric within each dataset.
Peak GPU memory usage ( GB )
Volume
ΩPF
dPF
f–k
Phasor fields
MoE
90×65×128
29
13
0.031
0.005
0.035
180×130×256
58
26
0.188
0.026
0.234
360×260×512
115
51
1.496
0.208
1.872
720×520×1024
229
103
11.960
1.658
14.976
Table 4: Peak GPU memory usage for our reconstruction methods without padding . ΩPF and dPF are the number of temporal frequencies and propagated depths used by the phasor-fields runs. Missing entries are reported as OOM . Precision: f–k in FP32 ; phasor fields and MoE in FP16 .
Peak GPU memory usage ( GB )
Volume
ΩPF
dPF
f–k
Phasor fields
MoE
90×65×128
29
13
0.133
0.009
0.143
180×130×256
58
26
1.064
0.072
1.147
360×260×512
115
51
8.477
0.567
9.170
720×520×1024
229
103
oom
4.530
oom
Table 5: Peak GPU memory usage for our reconstruction methods with padding . ΩPF and dPF are the number of temporal frequencies and propagated depths used by the phasor-fields runs. Missing entries are reported as OOM . Precision: f–k in FP32 ; phasor fields and MoE in FP16 .
Figure 14: Stage-time decomposition for the main reconstruction pipelines.
Figure 15: Frame time and peak GPU memory for FP16 and FP32 configurations.
Figure 16: Equivalence with reference implementation on the dynamic capture of Nam et al. (2021) . The sequence plot reports the mean absolute difference over time, while the frame panel shows the reference image, our reconstruction, FLIP metric and absolute difference. The reference for phasor-fields-based methods is Nam et al. (2021) ’s reconstruction; for f–k , the reference is f–k using FP32 . The two phasor-fields curves overlap.
Figure 17: The regions used in Table 6 . (a) The brightest 5% of pixels form the object (red contour), and the outer border forms the background (blue band). Their ratio is the contrast . (b) Sharpness is the mean gradient magnitude inside the same object contour.
Method
Padding
↑ Object
↓ Background
↑ Contrast
↑ Sharpness
Our f–k
no
0.615
0.0379
16.2
0.043
yes
0.656
0.0475
13.8
0.037
Our phasor-fields
no
0.405
0.0115
35.1
0.056
yes
0.410
0.0079
51.6
0.056
Our MoE ( β=0.3 )
no
0.449
7.4⋅10−6
60,981
0.053
yes
0.479
0.0009
512
0.052
Table 6: Background suppression on nlosbox1 , in FP32 . Each method is reported with and without padding. Green and bold mark the best value in each column. The sweep over the pooling weight β is reported in the supplementary material.
Figure 18: Offline throughput, in millions of voxels processed per second, for representative confocal and non-confocal datasets. Hatched bars are our implementations and solid bars the original MATLAB ones. Full timing and memory results are reported in the supplementary material.
Table 7: Offline comparison against the original MATLAB implementations, showing the largest configuration measured for each dataset. This is a representative extract: every dataset was also evaluated at lower spatial and temporal resolutions, and those results, together with the per-resolution trends, are reported in full in the supplementary material. Throughput is the reconstructed volume divided by reconstruction time. For exhaustive captures, the dimensions column lists the original measurement grid before normal moveout correction; normal moveout correction produces a 31×31×T confocal transient, which is the volume used for the throughput calculation. Max. tracked memory reports the peak footprint of each implementation’s main buffers: host RAM for the original MATLAB code, and the peak of all tracked CUDA allocations for ours, which counts device buffers and pinned host staging together and is therefore not a pure video-memory figure.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 19: Photon binning time for a dynamic NLOS capture of shape 190×190×3002 ( x×y×t ), whose photons are accumulated directly into 208 temporal frequencies. The photon count varies slightly from frame to frame and typically stays near 1.2⋅106 .
Figure 20: Teaser dataset from ( Lindell et al., 2019 ) . The first and second columns correspond to measurements of 180 and 10 minutes, respectively. We downsample the former to 106 and 5⋅105 photons to show that the hidden scene remains recognizable with fewer captured photons. The first and second rows show our f–k and phasor-fields reconstructions, respectively. The numbers within each reconstruction indicate the Peak Signal-to-Noise Ratio with respect to the most informed reconstruction, shown in the first column.
Algorithm 1 f–k migration ( Lindell et al., 2019 ) . Reconstruction pipeline following prior work. Pseudocode shown in a Python-like style. Nx , Ny , and Nz denote the volume dimensions. dmax refers to the maximum distance, whereas aptwidth is the physical size in meters of the scanned relay wall.
Algorithm 3 Phasor-fields as implemented by Nam et al. (2021) . Reconstruction pipeline following prior work. P holds the sampled phasor field P^ω and K the dense propagation kernel, both as defined in Section 4.3 of the main paper; P denotes the stack of transformed slices Pc produced by the first loop. Nx , Ny , and Nf denote the relay-wall spatial dimensions and the number of temporal-frequency slices; dmin , dmax , and Δd define the propagated-depth range, and w weights each frequency slice.
Algorithm 4 Our symmetry-aware phasor-fields dataflow. The dense kernel K from Algorithm 3 is replaced by a radial kernel Kr and a pixel-to-radius map R .
Figure 22: Per-configuration real-time comparison on the nlosbox1 dynamic capture. The top row shows a representative reconstruction, while the bottom row summarizes the runtime-memory trade-off across methods. Frame rates are derived from mean frame times, as in Table 2 of the main paper.
Figure 23: Image-space error on the office scene. Each row compares our reconstruction with its reference and reports FLIP and absolute error maps. The reference for the phasor-fields rows is Nam et al. (2021) ’s phasor-fields implementation, so the comparison isolates the radial kernel representation. For f–k , the reference is f–k using FP32 .
β
↑ Object
↓ Background
↑ Contrast
↑ Sharpness
0.15
0.520
8.7⋅10−5
6,008
0.050
0.20
0.495
3.8⋅10−5
12,993
0.051
0.25
0.471
1.7⋅10−5
28,186
0.052
0.30
0.449
7.4⋅10−6
60,981
0.053
0.35
0.428
3.2⋅10−6
132,666
0.054
0.40
0.408
1.4⋅10−6
291,566
0.055
Appendix
Table 8: Impact of the pooling weight β on the 100-frame nlosbox1 sequence (FP32, unpadded). Because every metric changes monotonically with β , there is no single optimal setting: β=0.5 provides the strongest background rejection but also the weakest object signal. Bold text highlights our chosen operating point ( β=0.3 ), which matches the MoE row in Table 6 of the main paper.
Table 9: Offline performance and memory comparison between our accelerated f–k and phasor-fields implementations and their original MATLAB counterparts, across every measured resolution. Throughput is the reconstructed volume divided by reconstruction time. For exhaustive captures, the dimensions column lists the original measurement grid before normal moveout correction, which produces a 31×31×T confocal transient volume. Max. tracked memory reports the peak footprint of each implementation’s main buffers: host RAM for the original MATLAB code, and the peak of all tracked CUDA allocations for ours, counting device buffers and pinned host staging together (see the discussion preceding these tables). Bold marks our value when it beats the baseline.
Table 10: Continuation of Table 9 .
↓ Time ( s )
↑ Throughput (Mvox/ s )
↓ Max. tracked memory ( GB )
Dataset
Dimensions
Algorithm
Ours
Original
Ours
Original
Ours
Original
Tal-generated dataset ( Royo and Luesia-Lahoz, 2024 ) , inspired by a Phasor Fields scene ( Liu et al., 2020 )
Image reconstruction recovers images, volumes, or scenes from incomplete, noisy, indirect, or compressed measurements using physical models and learned priors.
Imaging objects hidden outside the direct line of sight expands the effective field of view and is critical for applications such as autonomous driving and robotic perception. Despite impressive progress in time-of-flight (ToF)-based non-line-of-sight (NLOS) imaging, real-world deployment remains challenging because practical measurements are often collected over spatially limited, arbitrarily shaped relay regions-conditions that violate the planar-wall and dense-sampling assumptions made by most existing methods. To address these limitations, we propose a LOS-guided NLOS imaging pipeline that imposes no geometric assumptions on the relay surface and naturally supports both confocal and non-confocal configurations. Our method represents the hidden scene using 3D Gaussian primitives and couples them with an efficient, differentiable transient rendering model, enabling end-to-end optimization directly from measured transients. We validate our approach on real-world measurements from both a public dataset and a custom-built capture system. Across settings, our method achieves state-of-the-art reconstruction fidelity under spatially limited, sparsely sampled conditions, and significantly outperforms existing methods on complex, arbitrary relay surface geometries.
Yi Wang, Ziyu Zhan, Yuran Wang +5
Department of Precise Instrument, Tsinghua University, China · Department of Electrical Engineering, City University of Hong Kong, China · Yau Mathematical Science Center, Tsinghua University, China
Time-of-Flight non-line-of-sight (ToF NLOS) imaging techniques provide state-of-the-art reconstructions of scenes hidden around corners by inverting the optical path of indirect photons scattered by visible surfaces and measured by picosecond resolution sensors. The emergence of a wide range of ToF NLOS imaging methods with heterogeneous formulae and hardware implementations obscures the assessment of both their theoretical and experimental aspects. We present a comprehensive study of a representative set of ToF NLOS imaging methods by discussing their similarities and differences under common formulation and hardware. We first outline the problem statement under a common general forward model for ToF NLOS measurements, and the typical assumptions that yield tractable inverse models. We discuss the relationship of the resulting simplified forward and inverse models to a family of Radon transforms, and how migrating these to the frequency domain relates to recent phasor-based virtual line-of-sight imaging models for NLOS imaging that obey the constraints of conventional lens-based imaging systems. We then evaluate performance of the selected methods on hidden scenes captured under the same hardware setup and similar photon counts. Our experiments show that existing methods share similar limitations on spatial resolution, visibility, and sensitivity to noise when operating under equal hardware constraints, with particular differences that stem from method-specific parameters. We expect our methodology to become a reference in future research on ToF NLOS imaging to obtain objective comparisons of existing and new methods.
Julio Marco, Adrian Jarabo, Ji Hyun Nam +3
Universidad de Zaragoza–I3A, Spain · University of Wisconsin–Madison, USA · Department of Data Science, Inha University, South Korea +1
Practical non-line-of-sight (NLOS) measurements are affected by noise and deviations from idealized light transport, which can degrade reconstruction with conventional light-cone transform (LCT) methods. We propose the Measurement-Adaptive Light-Cone Transform (MALCT), a physics-guided framework that retains the efficient Fourier-domain inversion structure of LCT. {Within a unified variational formulation, MALCT combines the analytical response with a shared learned response through measurement-conditioned weighting and predicts a measurement-conditioned, frequency-dependent regularization map.} An attention-enhanced spatio-temporal feature extraction module captures long-range temporal dependencies and local spatial correlations to support reconstruction under low-signal conditions. Experiments on synthetic data and real measurements from different NLOS systems demonstrate improved reconstruction quality. Controlled evaluations further assess robustness to temporal-response broadening and reduced acquisition time. MALCT also preserves scene structures with spatial sampling grids down to 32×32, supporting its potential for NLOS imaging under limited acquisition budgets.
Lianfang Wang, Xueying Liu, Junnan Dong +4
School of Mathematical Sciences, Beijing Normal University, Beijing 100875, China · School of Physics, Nankai University, Tianjin 300071, China