A comprehensive study of time-of-flight non-line-of-sight imaging

Julio Marco; Adrian Jarabo; Ji Hyun Nam; Alberto Tosi; Diego Gutierrez; Andreas Velten

cs.CVMar 10, 2026

A comprehensive study of time-of-flight non-line-of-sight imaging

Authors: Julio Marco, Adrian Jarabo, Ji Hyun Nam, Alberto Tosi, Diego Gutierrez, Andreas Velten

Organizations: Universidad de Zaragoza–I3A, Spain · University of Wisconsin–Madison, USA · Department of Data Science, Inha University, South Korea · Politecnico di Milano, Italy

Abstract

Time-of-Flight non-line-of-sight (ToF NLOS) imaging techniques provide state-of-the-art reconstructions of scenes hidden around corners by inverting the optical path of indirect photons scattered by visible surfaces and measured by picosecond resolution sensors. The emergence of a wide range of ToF NLOS imaging methods with heterogeneous formulae and hardware implementations obscures the assessment of both their theoretical and experimental aspects. We present a comprehensive study of a representative set of ToF NLOS imaging methods by discussing their similarities and differences under common formulation and hardware. We first outline the problem statement under a common general forward model for ToF NLOS measurements, and the typical assumptions that yield tractable inverse models. We discuss the relationship of the resulting simplified forward and inverse models to a family of Radon transforms, and how migrating these to the frequency domain relates to recent phasor-based virtual line-of-sight imaging models for NLOS imaging that obey the constraints of conventional lens-based imaging systems. We then evaluate performance of the selected methods on hidden scenes captured under the same hardware setup and similar photon counts. Our experiments show that existing methods share similar limitations on spatial resolution, visibility, and sensitivity to noise when operating under equal hardware constraints, with particular differences that stem from method-specific parameters. We expect our methodology to become a reference in future research on ToF NLOS imaging to obtain objective comparisons of existing and new methods.

Figures & tables

$t$	Time.
$\Omega$	Frequency.
$c$	Speed of light.
$V$	Hidden scene volume.
$L,S$	Illumination and sensor relay surfaces.
$\mathbf{x}_{a},\mathbf{x}_{b},\mathbf{x}_{v}\!\in\!V$	Points in the hidden scene.
$\mathbf{x}_{l}\!\in\!L,\mathbf{x}_{s}\!\in\!S$	Points at visible planes $L$ and $S$ .

Table 1. Notation used throughout the paper. Note in several cases continuous functions are defined along their respective matricial equivalent such as

G(\cdot)\equiv{\mathbf{G}}

Figure 1. (a) ToF NLOS imaging setups measure multiply scattered photons following paths between a laser device at

\mathbf{x}_{0}

, a laser target

\mathbf{x}_{1}

, sensor target

\mathbf{x}_{n-1}

, and sensor device at

\mathbf{x}_{n}

. Classic approaches assume perfect calibration of laser and sensor devices and targets (red and blue), third-bounce-only occlusion-free transport (green), and diffuse surface reflectance (purple). Under these assumptions, time-resolved NLOS measurements can be modeled by different types of Radon transforms. (b) NLOS light transport of photons with time of flight

t_{lv}+t_{sv}

under unconstrained

\mathbf{x}_{l}

and

\mathbf{x}_{s}

is modeled by the elliptical Radon transform (ERT), whose inverse is typically approximated via filtered backprojection ( Section 4.1 ). (c) By assuming planar locality at surface points

\mathbf{x}_{v}

, NLOS light transport can be approximated by the planar Radon transform (PRT), whose inverse can be solved analytically by means of Laplacian-filtered backprojection ( Section 4.2 ). (d) Confocal setups co-locate

\mathbf{x}_{l}\equiv\mathbf{x}_{s}

, under which the ERT becomes a spherical Radon transform (SRT); existing methods leveraged this constraint to implement efficient solvers for the inverse SRT ( Section 4.2 ).

	Forward model	Inverse model
General		$\hskip-5.0ptf(\mathbf{x}_{v})\approx[K_{f}(\mathbf{x}_{v})\ast\iint[K_{H}(\mathbf{x}_{l},\mathbf{x}_{s},t)$ $\ast H(\mathbf{x}_{s},\mathbf{x}_{l},t-(t_{lv}+t_{sv}))]\mathrm{d}\mathbf{x}_{l}\mathrm{d}\mathbf{x}_{s}]_{t=0}$ ( Eq. 22c )
ERT	$H(\mathbf{x}_{l},\mathbf{x}_{s},t)=\bar{G}\int\delta(d_{lv}+d_{sv}-ct)f(\mathbf{x}_{v})\mathrm{d}\mathbf{x}_{v}$ ( Eq. 7c )	$f(\mathbf{x}_{v})\!\approx\nabla_{\mathbf{x}_{v}}^{2}\iint H(\mathbf{x}_{l},\mathbf{x}_{s},-(t_{lv}+t_{sv}))\mathrm{d}\mathbf{x}_{l}\mathrm{d}\mathbf{x}_{s}$ ( Eq. 8c ) ( Velten et al., 2012 )
PRT	$H(\mathbf{x}_{l}^{\prime},\mathbf{x}_{s}^{\prime},t-(t_{lv}+t_{sv}))\approx$ $\iiint{f(\mathbf{x}_{v}^{\prime})}\delta(k_{x}x^{\prime}+k_{y}y^{\prime}+k_{z}z^{\prime})dx^{\prime}dy^{\prime}dz^{\prime}$ ( Eq. 9c )	$f(\mathbf{x}_{v})\approx\iint\nabla_{t}^{2}H(\mathbf{x}_{l},\mathbf{x}_{s},-(t_{cv}+t_{sv}))\mathrm{d}\mathbf{x}_{l}\mathrm{d}\mathbf{x}_{s}$ ( Eq. 10c ) ( Liu et al., 2019a )
SRT	$H(\mathbf{x}_{s},t)=\bar{G}\int\delta(2d_{sv}-ct)f(\mathbf{x}_{v})\mathrm{d}\mathbf{x}_{v}$ ( Eq. 11 )	$\mathrm{R}_{t}\{H\}(\bar{\mathbf{x}}_{s},\bar{t})=\int a(\bar{\mathbf{x}}_{v}-\bar{\mathbf{x}}_{s})\mathrm{R}_{z}\{f\}(\bar{\mathbf{x}}_{v})\mathrm{d}\bar{\mathbf{x}}_{v}$ ( Eq. 12c ) ( O’Toole et al., 2018 )

Table 2. Summary of forward and inverse models in NLOS imaging ( Section 4 ) under assumptions Asm.1 to Asm.5 .

	Continuous form	Matricial form
General ( Eq. 22 )	$\hskip-5.0ptf(\mathbf{x}_{v})\approx[K_{f}(\mathbf{x}_{v})\ast\iint[K_{H}(\mathbf{x}_{l},\mathbf{x}_{s},t)\ast$ $H(\mathbf{x}_{s},\mathbf{x}_{l},t-(t_{lv}+t_{sv}))]\mathrm{d}\mathbf{x}_{l}\mathrm{d}\mathbf{x}_{s}\|_{t=0}$	${\mathbf{f}}\approx{\mathbf{K}}_{{\mathbf{f}},\square}{\mathbf{A}}_{\square}^{\intercal}{\mathbf{K}}_{{\mathbf{h}},\square}{\mathbf{h}}$
ERT ( Velten et al., 2012 ) ( Eq. 8 )	$f(\mathbf{x}_{v})\!\approx\nabla_{\mathbf{x}_{v}}^{2}\iint H(\mathbf{x}_{l},\mathbf{x}_{s},-(t_{lv}+t_{sv}))\mathrm{d}\mathbf{x}_{l}\mathrm{d}\mathbf{x}_{s}$	${\mathbf{f}}\approx{\mathbf{K}}_{\nabla,\mathbf{x}_{v}}{\mathbf{A}}_{\mathrm{ert}}^{\intercal}{\mathbf{h}}$
PRT ( Liu et al., 2019a ) ( Eq. 10 )	$f(\mathbf{x}_{v})\approx\iint\nabla_{t}^{2}H(\mathbf{x}_{l},\mathbf{x}_{s},-(t_{cv}+t_{sv}))\mathrm{d}\mathbf{x}_{l}\mathrm{d}\mathbf{x}_{s}$	${\mathbf{f}}\approx{\mathbf{A}}_{\mathrm{prt}}^{\intercal}{\mathbf{K}}_{\nabla,t}{\mathbf{h}}$
SRT ( O’Toole et al., 2018 ) ( Eq. 13 )	$\widehat{f}(\Omega_{\mathbf{x}_{v}})\approx\mathrm{R}_{z}^{-1}\Bigl\{\widehat{a}^{-1}(\Omega_{\bar{\mathbf{x}}_{v}-\bar{\mathbf{x}}_{s}})\mathrm{R}_{t}\{\widehat{H}\}(\Omega_{\bar{\mathbf{x}}_{s}})\Bigr\}\left(\Omega_{\mathbf{x}_{v}}\right)$	${\mathbf{f}}\approx{{\mathbf{R}}}^{-1}_{z}{{\mathbf{F}}}^{-1}\widehat{{\mathbf{A}}}_{\mathrm{ilct}}{\mathbf{F}}{\mathbf{R}}_{t}{\mathbf{h}}$

Table 3. Summary of continuous and matricial form of NLOS imaging inverse models ( Section 4 ) under assumptions Asm.1 to Asm.5 .

Figure 2. (a) Top view of the simulated scene made of a $$1\text{\,}\mathrm{m}$\times$1\text{\,}\mathrm{m}$$ patch co-planar and at 1 meter from the relay wall, under confocal acquisition. (b) Reconstructions obtained at increasing photon counts (columns) for every method (rows). (c,d) PSNR and MS-SSIM metrics of all the methods for the simulated photon counts ( $x$ axis). Point markers correspond to the reconstructions shown in (b). — Figure 2. (a) Top view of the simulated scene made of a $1\text{\,}\mathrm{m}$\times$1\text{\,}\mathrm{m}$ patch co-planar and at 1 meter from the relay wall, under confocal acquisition. (b) Reconstructions obtained at increasing photon counts (columns) for every method (rows). (c,d) PSNR and MS-SSIM metrics of all the methods for the simulated photon counts ( $x$ axis). Point markers correspond to the reconstructions shown in (b).

Figure 3. (a) Simulated scene with a $$1\text{\,}\mathrm{m}$\times$1\text{\,}\mathrm{m}$$ patch at 1 meter from the relay wall, confocal capture. (b) XZ frequency spectrum of the unfiltered backprojection output. (c) Front view (XY) of the reconstructions analyzed methods. (d) XZ frequency spectrum of the reconstructions yielded by each method. (e) Frequency spectrum of our estimated filtering operator for each method ( Eq. 29 ). — Figure 3. (a) Simulated scene with a $1\text{\,}\mathrm{m}$\times$1\text{\,}\mathrm{m}$ patch at 1 meter from the relay wall, confocal capture. (b) XZ frequency spectrum of the unfiltered backprojection output. (c) Front view (XY) of the reconstructions analyzed methods. (d) XZ frequency spectrum of the reconstructions yielded by each method. (e) Frequency spectrum of our estimated filtering operator for each method ( Eq. 29 ).

Figure 4. Variations on the width of the LoG filter, from $4$ to $19\text{\,}\mathrm{c}\mathrm{m}$ . Wider filters skew the reconstructions towards lower frequencies, which mitigates noise but fails to correctly reproduce surface edges.

Figure 5. Variations on the LCT parameter $\alpha$ which controls response to SNR on the Wiener filtering. Lower alpha values mitigate noise at the expense of blurring the edges of the reconstructed surface.

Figure 6. Variations on the central wavelength $\lambda_{c}=1/\Omega_{c}$ for the PF-CC illumination. Increasing $\lambda_{c}$ mitigates noise at the expense of losing geometric detail. Wavelengths shorter than the temporal uncertainty of the system ( $$80\text{\,}\mathrm{p}\mathrm{s}$\equiv$2.39\text{\,}\mathrm{c}\mathrm{m}$/c$ ) are unable to resolve the scene geometry. — Figure 6. Variations on the central wavelength $\lambda_{c}=1/\Omega_{c}$ for the PF-CC illumination. Increasing $\lambda_{c}$ mitigates noise at the expense of losing geometric detail. Wavelengths shorter than the temporal uncertainty of the system ( $$80\text{,}\mathrm{p}\mathrm{s} $\equiv$ 2.39\text{,}\mathrm{c}\mathrm{m} $/c$ ) are unable to resolve the scene geometry.

Panel of Figure 7 — Figure 7. Analysis of lateral resolution between two coplanar patches of size $0.5\text{\,}\mathrm{m}$\times$1\text{\,}\mathrm{m}$ under simulated confocal capture. (a) Reconstructions of patches with varying gap sizes $\Delta x\in\{$ 4\text{,}\mathrm{c}\mathrm{m} $,$ 16\text{,}\mathrm{c}\mathrm{m} $\}$ at a fixed depth $z=$ 1\text{,}\mathrm{m} $. Plots show the cross sections for each method and gap size; the ground truth geometry is represented with gray dotted lines. All methods consistently fail to sharply represent small gap sizes such as $4\text{\,}\mathrm{c}\mathrm{m}$ due to limitations on parameters inherent to the capture setup, such as the sampling rate of the impulse response function, aperture size, and temporal resolution. (b) Reconstructions of patches at varying depths $z\in\{$1\text{\,}\mathrm{m}$,$2\text{\,}\mathrm{m}$,$4\text{\,}\mathrm{m}$\}$ with a fixed gap size of $\Delta x=$12\text{\,}\mathrm{c}\mathrm{m}$ . Plots show the cross sections for each method, each curve represents a different depth; the ground truth geometry is represented with gray dotted lines. Consistently, all methods show increasing degradation of the lateral resolution as the distance $z$ between the relay wall and the patches increases.

Figure 8. Analysis of surface visibility under non-confocal acquisition. First row shows a top-view schematic of scenes with a single patch under position and rotation changes. Red arrows represent light paths starting at the laser point and following a virtual specular reflection. Green and gray regions denote the area of the patch (denoted with dotted lines) that hits or misses, respectively, the SPAD domain (blue) after virtual specular reflection. a, b) Visibility of co-planar patches depends on the patch lateral position. c-f) Lateral offset, rotation, and depth jointly decrease visibility.

Figure 9. Analysis of surface visibility under confocal acquisition in the same scene setups as Fig. 8 . A surface point will be visible when its normal intersects with the confocal SPAD-laser domain. This yields higher visibility of non-coplanar and surfaces in peripheral regions.

Figure 10. Comparison of reconstructions of a single centered patch at 1 meter captured under equal-photon counts (rows) in confocal (left table) and non-confocal setups (right table). Note no data is available for 25M and 50M photon counts for confocal data (left column, two last rows).

Figure 11. Comparison of reconstructions of a single patch captured under variations on depth and horizontal offset, under confocal (a) and non-confocal setups (b) with approximately 6M gathered photons.

Figure 12. Comparison of a single centered patch at 1m depth under variations in vertical rotation of 0, 30, and 60 degrees (rows) in confocal (a) and non-confocal setups (b) gathering approximately 5 million number of photons in total. LCT and PF tables show reconstructions for increasing parameter values: LCT varies the SNR parameter $\alpha$ ; PF varies the carrier wavelength $\lambda_{c}$ of the pulsed virtual illumination function. Default method parameter values are highlighted in bold.

Figure 13. Comparison of two patches at 1m depth with varying gap sizes between them (rows) in confocal (a) and non-confocal setups (b) gathering approximately 5 million number of photons in total. LCT and PF tables show reconstructions for increasing parameter values: LCT varies the SNR parameter $\alpha$ ; PF varies the carrier wavelength $\lambda_{c}$ of the pulsed virtual illumination function. Default method parameter values are highlighted in bold.

Explore similar work

Sep 3, 2026

Cascaded Non-Line-of-Sight Imaging

Time-of-flight non-line-of-sight (NLOS) imaging recovers information from hidden objects by analyzing the time of flight of indirect photons scattered on a visible (relay) wall. Most methods make the simplifying assumption that photons travel exclusively three-bounce paths, thus ignoring other useful information encoded in higher-order photons (with, e.g., four- or five-bounce paths). We present a novel cascaded NLOS imaging approach that leverages higher-order information and allows imaging a broader range of single- and multi-corner scenarios. We combine ultra-fast laser scanning with recent time-gated 2D sensor arrays to capture the scene's impulse response on a visible relay wall. From the captured impulse response, our method computes an analogous virtual impulse response at any other hidden wall. This effectively allows us to concatenate a second, virtual NLOS imaging system that leverages higher-order illumination. We validate our cascaded imaging method both in simulation and with a real prototype, demonstrating NLOS imaging with fourth- and fifth-bounce illumination of objects in challenging orientations and hidden around two corners. We also analyze how wave-based NLOS imaging interacts with rough hidden walls, which explains and helps overcome existing visibility limitations. We further illustrate how to image hidden objects from different perspectives, thus observing previously unseen features, by relying on multiple hidden walls.

Diego Royo, María Peña, Forrest B. Peterson +3

Universidad de Zaragoza–I3A, Spain · University of Wisconsin–Madison, United States of America

Computational Imaging Image Reconstruction

Ask PDF

Jun 19, 2026

Non-line-of-sight imaging with arbitrary relay surface geometries via 3D Gaussian Transient Rendering

Imaging objects hidden outside the direct line of sight expands the effective field of view and is critical for applications such as autonomous driving and robotic perception. Despite impressive progress in time-of-flight (ToF)-based non-line-of-sight (NLOS) imaging, real-world deployment remains challenging because practical measurements are often collected over spatially limited, arbitrarily shaped relay regions-conditions that violate the planar-wall and dense-sampling assumptions made by most existing methods. To address these limitations, we propose a LOS-guided NLOS imaging pipeline that imposes no geometric assumptions on the relay surface and naturally supports both confocal and non-confocal configurations. Our method represents the hidden scene using 3D Gaussian primitives and couples them with an efficient, differentiable transient rendering model, enabling end-to-end optimization directly from measured transients. We validate our approach on real-world measurements from both a public dataset and a custom-built capture system. Across settings, our method achieves state-of-the-art reconstruction fidelity under spatially limited, sparsely sampled conditions, and significantly outperforms existing methods on complex, arbitrary relay surface geometries.

Yi Wang, Ziyu Zhan, Yuran Wang +5

Department of Precise Instrument, Tsinghua University, China · Department of Electrical Engineering, City University of Hong Kong, China · Yau Mathematical Science Center, Tsinghua University, China

3D Reconstruction Image Reconstruction

Ask PDF

May 18, 2026

Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling

LiDARs are being increasingly deployed for consumer imaging in handheld, wearable, and robotic applications. These sensors can capture the time-of-flight of light at picosecond resolution, which in principle, enables them to capture information about objects hidden from their field of view. While such non-line-of-sight (NLOS) imaging capabilities have been shown on research-grade LiDARs, they are challenging to achieve on consumer devices due to poor signal quality resulting from low laser power, low spatial resolution, and object and camera motion. Inspired by burst photography and synthetic aperture radar, we propose a multi-frame fusion strategy to overcome these challenges and demonstrate NLOS imaging on consumer LiDAR. We first introduce the motion-induced aperture sampling model to unify the effects of object shape, object motion, and camera motion under a single measurement model. Using this model, we demonstrate several NLOS capabilities on a smartphone-grade LiDAR: (1) 3D reconstruction, (2) single and multi-object tracking, and (3) camera localization using hidden objects. Previously, NLOS imaging capabilities were largely restricted to bulky and expensive research-grade hardware that requires extensive setup and calibration. Our results represent a shift towards plug-and-play NLOS imaging, where anyone can image hidden objects with off-the-shelf hardware ($<100) and no additional setup. We believe that democratization of such capabilities will advance consumer applications of NLOS imaging.

Siddharth Somasundaram, Aaron Young, Akshat Dave +2

Massachusetts Institute of Technology, Cambridge, MA, United States · Dartmouth College, Hanover, NH, United States

3D Reconstruction Image Reconstruction

Ask PDF