SpiderCam: Low-Power Snapshot Depth from Differential Defocus

Marcos A. Ferreira; Tianao Li; John Mamish; Josiah Hester; Yaman Sangar; Qi Guo; Emma Alexander

Name	Type	FPGA	Resolution	Core Power (W)		Real-World Quantitative Results
				Min	Max	On-Device Working Range	Full System Power
Jin, 2014 [ 31 ]	Stereo	Kintex-6	640 $\times$ 480	1.93	2.67	none	none
Raj, 2014 [ 32 ]	DfD	Virtex-4 †	400 $\times$ 400	0.46	0.58	0.77–0.80 m ( 0.5 $\%$ err )	2 W + camera
Mattoccia, 2015 [ 44 ]	Stereo	Spartan-7	640 $\times$ 480	0.44	0.68	qualitative	2.5 W
Ttofis, 2015 [ 65 ]	Stereo	Kintex-7	1280 $\times$ 720	0.92	1.53	qualitative	2.8 W
Puglia, 2017 [ 55 ]	Stereo	Zynq Artix	1024 $\times$ 768	0.43	0.68	none	2 W

Table 1: Power consumption for passive FPGA-based depth imaging systems. “Core Power” numbers reflect only power consumed by logic for depth computation on aligned images; I/O and sensor power are ignored. Power estimates are derived using vendor tools and data from the corresponding works, normalized to a processing rate of 30 Mpix/sec across methods, see Sec. S8 for details. In addition to SOTA power efficiency in core power estimates, we are one of the very few systems to report full system power consumption and real-world depth performance (from noisy sensor captures rather than on existing datasets collected with large-baseline, high-power, pre-aligned image pairs). Note that we report two estimates for our method: a “normalized” estimate that handles more pixels per second on a less efficient FPGA for fairer comparisons with the literature, and a lower estimate reflecting the true parameters of our system. Additionally, we include Focal Split (Luo, 2025) [ 41 ] , which reports a similar method on a Raspberry Pi 5 with real-world depth and power performance. † This work uses an Xilinx Virtex-2 FPGA, but due to vendor tool unavailability, we estimate power for the more efficient Virtex-4.

Figure 1: Method overview. Our camera uses a beam splitter and a pair of differentially-defocused low power sensors to observe the same scene with an offset depth of field. Our algorithm processes these images to produce depth maps thresholded by confidence.

Figure 2: Captured images and depth maps computed on-device. Note the handling of continuous depths (vs. disparity levels), reflective textures, and transparent objects that active sensing would struggle with, as well as objects in motion due to snapshot capture.

Figure 3: Quantitative analysis on images from device. (a) Mean absolute error (MAE) of depth as a function of true depth, with working range bounds. Our method robustly outperforms the Focal Split algorithm [ 41 ] in the low power hardware setting, and accounting for spatial variation in our compact optics is vital. Purple dots show errors from depth maps computed on-device and demonstrate good agreement with off-line depth computations (rest of figure), which were performed in 32-bit floating point. (b) The MAE vs. depth curve of our method with dynamic sparsity values set by per-image confidence percentiles (grays) and the MAE (solid purple) and output pixel density (100-sparsity, dashed purple) resulting from the fixed confidence thresholds used in our method. (c) Working range increases as data is sparsified by per-image confidence percentiles.

Figure 4: Power/accuracy trade-offs for alternate algorithms. We estimate max core power on the ECP5 with manufacturer’s tools and evaluate working range and MAE from 0.4m to 1.0m offline on images collected by our hardware prototype. Adding computational components improves accuracy while increasing power cost. Our algorithm is shown with the purple star.

Figure 5: Resource Usage Across Methods. We compare the resource utilization of several configurations of our system as well as systems in the literature. “DS” and “SS” denote evaluation at 2 and 1 scales, respectively.

Figure S1 : General overview of the algorithm.

Figure S2 : Preprocessing module.

Figure S3 : Computation for the

N^{th}

scale.

Filter / Operation	Adders	True Mult.	Easy Mult.	Total FLOPs
Gaussian $5\times 5$ Filter	8	2	8	18
Downsampler Filter	2	0	4	6
Upsampler Filter	6	4	4	14
$V_{N}$ & $W_{N}$	2	2	0	4
Cross Mult. w/ Weights (DX_DY_ENABLE = 1)	8	12	10	30
Cross Mult. w/ Weights (DX_DY_ENABLE = 0)	0	4	0	4

Table S1 : FLOP counts and arithmetic breakdown per operation.

N

refers to the number of scales. Total FLOPs is the sum of all arithmetic operations per filter or stage.

N (Scales)	DX_DY_ENABLE	Adders	True Mult.	Easy Mult.	Dividers	Total FLOPs
1	0	28	14	28	1	71
1	1	36	22	38	1	97
2	0	70	36	64	1	171
2	1	86	52	84	1	223
3	0	124	66	108	1	299
3	1	148	90	138	1	377

Table S2 : Aggregate per-pixel operation counts across

N

scales using the per-operation table and the updated per-scale rule (2× Gaussian, 2× Downsampler,

(1+2\times N^{th})

Upsamplers per scale, and 1× Cross Mult. with Weights per scale),

V_{N}\text{ \& }W_{N}

plus one final Add and Divide.

N (Scales)	DX_DY_ENABLE	Scale Buffers	Latency Buffers	Total Buffers
1	0	29	0	29
1	1	33	0	33
2	0	93	12	105
2	1	105	14	119
3	0	227	36	263
3	1	255	42	297

Table S3 : Aggregate buffered line requirements for

N=1

–

3

scales, including buffers used in scales, buffers consumed by Latency Buffer, and Total buffers used. Each buffer is the size of one line of the image in FP16. For both DX_DY_ENABLE modes.

Name	Type	FPGA	Resolution	Stereo Disparity Levels	Mpix/sec	Estimated Core Power (W)			Measured Full-System Power	Resource Utilization
						Reported	Min	Max		LUTs	REGs	BRAMs	DSPs
Jin, 2014 [ 31 ]	Stereo	Kintex-6	640 $\times$ 480	60	15.6	10.6	1.93	2.67	none	61000	61000	165	0
Raj, 2014 [ 32 ]	DfD	Virtex-4	400 $\times$ 400	n/a	48.0	none	0.46	0.58	2W + camera	10028	7609	29	50
Mattoccia, 2015 [ 44 ]	Stereo	Spartan-7	640 $\times$ 480	32	9.2	none	0.44	0.68	2.5W	23749	9030	67	0
Ttofis, 2015 [ 65 ]	Stereo	Kintex-7	1280 $\times$ 720	64	55.3	2.8	0.92	1.53	none	57492	71192	302	458
Puglia, 2017 [ 55 ]	Stereo	Zynq Artix	1024 $\times$ 768	64	23.6	0.17	0.43	0.68	2W	29057	15208	80	0

Table S4 : Extended version of Tab. 1 . This table includes the reported power from other works. We re-did calculations from other works to account for differing pixel processing rates - our analysis favors methods that process pixels above 30 Mpix/s and penalize works below 30 Mpix/s. We under-estimate power for all works except [ 55 ] . † The internal design of the Lattice ECP5 means that its resource utilization values can’t be directly compared to those of other FPGAs. Each LUT, BRAM, and DSP module on the ECP5 is smaller than on the 7-Series, so a design on the ECP5 will typically take more of these smaller resources.

Figure S4 : Heatmap comparison of our method with and without field curvature correction. We decompose an image in 16 radial regions (row 1) and show the predicted depth vs. true depth heatmaps (filtered with a 90% sparsity) for every radial regions and their combined region. We show that compared to using constant optical parameters (row 2), applying spatial variation (row 3) yields better depth predictions that are much more aligned with the diagonal, indicating better accuracy, in all radial zones. We also observe with the confidence filtering, radial zones in the center tend to perform better at longer depth ranges than the peripheral zones, which is well aligned with the observation of Petzval field curvature, where the focal planes are curved towards the lens.

Figure S5 : Ablation studies. (a) Heatmaps of Focal Split [ 41 ] and different variations of our method. (b) Predicted depth maps on the calibration dataset at three different depths from our front-parallel calibration dataset. We show that incorporating spatial variation, multiscale, and multi-derivative in our algorithm helps improve depth estimation in our lower power setting.

Figure S6 : Comparison of different confidence. We show the mean depth error vs. true depth curves at 90% sparsity on the calibration data using different confidence metrics. We observed an increase in the working range with

VW

(numerator in Eq. 3 ) as confidence compared with

W^{2}

or

|W|

, which were used in previous work [ 41 , 40 ] .

Component Description	Commercial off-the-shelf	Part number / path to design files on GitHub	Qty
FPGA development board	yes	LFE5UM5G-85F-EVN	1
Image sensor cables	yes	900-0150180325-ND	2
FT232H breakout board	yes	2264 (Adafruit)	1
16mm beamsplitter	yes	CCM5-BS016	1
M09 mounted lens	yes	AC050-008-A-ML	2
Lens adapter	yes	S05TM09	2

Table S5 : A list of all of the significant parts making up our system. Commercial off-the-shelf components can be purchased at a store, while non-commercial off-the-shelf parts are those which we designed for this project and can be found in our GitHub repository ( https://github.com/NUBIVlab/SpiderCam ) using the path names. To replicate our work, our design files must be used to order these from manufacturers.

Figure S7 : Optical assembly. This photograph shows the 3D printed mount that affixes the image sensor to the beamsplitter and aligns it to the lens. The image sensor, lens, and flat-flex cable connecting the optical assembly to the FPGA development board are visible.

Figure S8 : Depth maps computed on-device for front-parallel textured plane. This data was used to generate purple dots in Fig. 3 . Note that density naturally falls off away from the center of the working range. Our confidence estimate captures edge textures with sufficient focus, and field curvature shifts the effective working range across the image. Edge effects dominate error in sparse images.

Figure S9 : Depth maps computed on-device with their references for real scenes. Note MAE from top to bottom of 0.057 m, 0.058 m, and 0.063 m, consistent with data in Fig. S8

Figure S10 : Screenshot of representative power consumption trace. We demonstrate how we used the Otii Arc Pro to measure power consumption for Tab. S6 by measuring power over time. This screenshot shows power for our system measured in a dual-scale configuration with dx and dy filters at 32.5 FPS. Notably, the frame rate is evident from the frequency of fluctuations in power consumption as seen in this graph - during active processing of a frame, power consumption is 650 mW; in-between frames, it drops to 420 mW. Our average power is 611 mW, which is slightly lower than our demonstrated 624 mW due to having captured this trace when the FPGA was cold compared to an extended period of usage during the recording of our demonstration that allowed the chip to warm up to its stable temperature. At the higher warm temperature, resistive losses in the device rise as well, resulting in slightly higher overall power draw.

Work	Scales	Compute $I_{x},I_{y}$	FPS	Overall power (mW)
[ 41 ]	-	-	2.1	4900
[ 55 ]	-	-	30	2000
[ 44 ]	-	-	$>30$	$<2500$
ours	2	yes	32.5	624
ours	2	no	32.5	562
ours	1	yes	32.5	489

Table S6 : Full system power consumption including image sensors, I/O, and FPGA, as reported in the literature and measured for several configurations of our method.

Scales	$I_{x},I_{y}$	SV	WR (m)	MAE (m)	Core power (mW)
2	yes	yes	0.551	0.044	312
2	yes	no	0.311	0.058	280
2	no	yes	0.282	0.075	276
2	no	no	0.259	0.071	257
1	yes	yes	0.423	0.069	214
1	yes	no	0.210	0.076	195

Table S7 : Power-accuracy tradeoff for alternate algorithms. We report the working range, MAE, and core power in Tab. S7 .

SpiderCam: Low-Power Snapshot Depth from Differential Defocus

Abstract

Figures & tables

Depth Estimation

High-Flux Count-Free Single-Photon 3D Cameras

Zero-Shot Depth from Defocus

On-Device Super Resolution Imaging Using Low-Cost SPAD Array and Embedded Lightweight Deep Learning

Abstract

Depth Estimation

Explore similar work

High-Flux Count-Free Single-Photon 3D Cameras

Zero-Shot Depth from Defocus

On-Device Super Resolution Imaging Using Low-Cost SPAD Array and Embedded Lightweight Deep Learning