Real-Time Branch-to-Tool Distance Estimation for Autonomous UAV Pruning: Benchmarking Five DEFOM-Stereo Variants from Simulation to Jetson Deployment

Yida Lin; Bing Xue; Mengjie Zhang; Sam Schofield; Richard Green

cs.CVMar 27, 2026

Real-Time Branch-to-Tool Distance Estimation for Autonomous UAV Pruning: Benchmarking Five DEFOM-Stereo Variants from Simulation to Jetson Deployment

Authors: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Organizations: Centre for Data Science and Artificial Intelligence, Victoria University of Wellington, Wellington, New Zealand · Department of Computer Science and Software Engineering, University of Canterbury, Canterbury, New Zealand

Abstract

Autonomous tree pruning with unmanned aerial vehicles (UAVs) is a safety-critical real-world task: the onboard perception system must estimate the metric distance from a cutting tool to thin tree branches in real time so that the UAV can approach, align, and actuate the pruner without collision. We address this problem by training five variants of DEFOM-Stereo - a recent foundation-model-based stereo matcher - on a task-specific synthetic dataset and deploying the checkpoints on an NVIDIA Jetson Orin Super 16 GB. The training corpus is built in Unreal Engine 5 with a simulated ZED Mini stereo camera capturing 5,520 stereo pairs across 115 tree instances from three viewpoints at 2m distance; dense EXR depth maps provide exact, spatially complete supervision for thin branches. On the synthetic test set, DEFOM-Stereo ViT-S achieves the best depth-domain accuracy (EPE 1.74 px, D1-all 5.81%, delta-1 95.90%, depth MAE 23.40 cm) but its Jetson inference speed of ~2.2 FPS (~450 ms per frame) remains too slow for responsive closed-loop tool control. A newly introduced balanced variant, DEFOM-PrunePlus (~21M backbone, ~3.3 FPS on Jetson), offers the best deployable accuracy-speed trade-off (EPE 5.87 px, depth MAE 64.26 cm, delta-1 87.59%): its frame rate is sufficient for real-time guidance and its depth accuracy supports safe branch approach planning at the 2m operating range. The lightweight DEFOM-PruneStereo (~6.9 FPS) and DEFOM-PruneNano (~8.5 FPS) run fast but sacrifice substantial accuracy (depth MAE > 57 cm), making estimates too unreliable for safe actuation. Zero-shot inference on real photographs confirms that full-capacity models preserve branch geometry, validating the sim-to-real transfer. We conclude that DEFOM-PrunePlus provides the most practical accuracy-latency balance for onboard distance estimation, while ViT-S serves as the reference for future hardware.

Figures & tables

Component	Count	Description
Tree instances	115	Individual simulated trees
View categories	3	Upward, downward, parallel
Stereo pairs per view	16	Captured at 2 m distance
Stereo pairs per tree	48	$3\times 16$
Total stereo pairs	5520	Full UE5 dataset
Per-sample labels	Dense	RGB left/right + EXR depth

TABLE I : UE5 Virtual Tree Branches Dataset Composition

Model	Backbone	Decoder	Conv Layers	GRU	fnet dim	DINOv2 Scale
DEFOM-Stereo ViT-S	DINOv2 ViT-S	Dual DPT (RefineNet)	${\sim}$ 66	3-level [128]	256-d	1.0 $\times$
DEFOM-Stereo ViT-L	DINOv2 ViT-L	Dual DPT (RefineNet)	${\sim}$ 66	3-level [128]	256-d	1.0 $\times$
DEFOM-PrunePlus	DINOv2 ViT-S	EnhancedDPT (2 RCU)	${\sim}$ 14	2-level [128]	192-d	1.0 $\times$
DEFOM-PruneStereo	DINOv2 ViT-S	FastDPT	${\sim}$ 6	2-level [96]	128-d	0.75 $\times$
DEFOM-PruneNano	DINOv2 ViT-S	TurboDecoder + Ghost	${\sim}$ 3	DS-GRU [64]	96-d	0.5 $\times$

TABLE II : Architectural Differences Among the Five Evaluated Models

Setting	ViT-S	ViT-L	D-Prune+	D-PruneS.	D-PruneN.
Batch size	10	6	10	10	10
Train iters	18	18	14	10	7
Scale iters	8	8	4	3	2
Valid iters	32	32	20	12	9
Mixed prec.	On	Off	On	On	On
Stabilization	None	3-ep. warm-up	None	None	None

TABLE III : Model-Specific Training Hyperparameters (shared: LR

4{\times}10^{-5}

, crop

384{\times}512

, 300 epochs, patience 30)

Model	EPE (px) $\downarrow$	Disp. RMSE (px) $\downarrow$	D1-all (%) $\downarrow$	Bad 1.0 (%) $\downarrow$	$\delta_{1}$ (%) $\uparrow$	$\delta_{2}$ (%) $\uparrow$	$\delta_{3}$ (%) $\uparrow$	Depth MAE (cm) $\downarrow$	Depth RMSE (cm) $\downarrow$	Depth AbsRel $\downarrow$
DEFOM-Stereo ViT-S	1.744	5.824	5.81	13.29	95.90	98.15	99.00	23.40	157.83	0.041
DEFOM-Stereo ViT-L	1.630	5.371	6.36	16.15	95.75	98.11	98.98	26.94	170.70	0.046
DEFOM-PrunePlus	5.874	12.796	20.12	41.15	87.59	93.76	96.49	64.26	283.78	0.146
DEFOM-PruneStereo	12.322	23.753	23.76	37.93	82.71	90.03	93.93	57.63	279.98	0.207
DEFOM-PruneNano	13.056	24.226	40.12	59.49	68.96	83.53	90.45	112.16	414.36	0.379

TABLE IV : Benchmark Results for the Five Models on the UE5 Synthetic Test Set

Panel of Figure 5 — Fig. 1 : Qualitative disparity comparison on four UE5 synthetic test scenes. Each row shows one scene; columns show the left RGB input, right RGB input, ground-truth disparity, and predicted disparity from the five models. ViT-S and ViT-L closely reproduce the GT structure, DEFOM-PrunePlus preserves scene layout with softer edges, while DEFOM-PruneStereo and DEFOM-PruneNano show increasing fragmentation and boundary blurring.

Model	PyTorch (ms)	TRT FP16 (ms)	FPS (TRT)	Depth MAE (cm)	Usable?
DEFOM-Stereo ViT-S	1210	450	2.2	23.40	$\triangle$
DEFOM-Stereo ViT-L	3350	1180	0.8	26.94	$\times$
DEFOM-PrunePlus	800	300	3.3	64.26	✓
DEFOM-PruneStereo	380	145	6.9	57.63	$\times$
DEFOM-PruneNano	240	118	8.5	112.16	$\times$

TABLE V : Jetson Orin Super Inference Latency and Accuracy Summary

Fig. 2 : Onboard pruning pipeline on Jetson Orin Super. The stereo model provides the dense geometric signal that is converted into branch-to-tool distance for the cutting controller. At $\sim$ 3.3 FPS with DEFOM-PrunePlus, the system updates depth faster than the UAV can close the remaining distance.

Panel of Figure 8 — Fig. 3 : Zero-shot transfer from UE5 training to real tree-branch photographs. Each row shows one real scene; columns show the left RGB input and predicted disparity from the five models. ViT-S and ViT-L preserve thin-branch continuity and boundary sharpness, DEFOM-PrunePlus retains reasonable structure but with softer boundaries, while DEFOM-PruneStereo and DEFOM-PruneNano show fragmented depth and blurred boundaries.

Metric Depth Estimation

Metric depth estimation predicts scene distances in physical units from images, supporting robotics, mapping, augmented reality, and 3D reconstruction.

See all 62 papers

Training Deep Stereo Matching Networks on Tree Branch Imagery: A Benchmark Study for Real-Time UAV Forestry Applications
Feb 23, 2026Yida Lin, Bing Xue, Mengjie Zhang +2Stereo Matching Stereo Depth Estimation
Centre for Data Science and Artificial Intelligence, Victoria University of Wellington, Wellington, New Zealand · Department of Computer Science and Software Engineering, University of Canterbury, Canterbury, New Zealand
Progressive Per-Branch Depth Optimization for DEFOM-Stereo and SAM3 Joint Analysis in UAV Forestry Applications
Feb 24, 2026Yida Lin, Bing Xue, Mengjie Zhang +23D Reconstruction Depth Estimation
Centre for Data Science and Artificial Intelligence Victoria University of Wellington, Wellington, New Zealand · Department of Computer Science and Software Engineering University of Canterbury, Canterbury, New Zealand
Low-Cost Stereo Vision for Robust 3D Positioning of Thin Radiata Pine Branches in Autonomous Drone Pruning
May 6, 2026Yida Lin, Bing Xue, Mengjie Zhang +2Aerial Robotics Stereo Matching
Centre of Data Science and Artificial Intelligence & School of Engineering and Computer Science, Victoria University of Wellington, PO Box 600, Wellington, New Zealand · Computer Vision Research Lab & Department of Computer Science and Software Engineering, Canterbury University, PO Box 8041, Christchurch, New Zealand

Feb 23, 2026J/K move · Enter open · S save

Training Deep Stereo Matching Networks on Tree Branch Imagery: A Benchmark Study for Real-Time UAV Forestry Applications

Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield+1

Centre for Data Science and Artificial Intelligence, Victoria University of Wellington, Wellington, New Zealand · Department of Computer Science and Software Engineering, University of Canterbury, Canterbury, New Zealand

Stereo Matching Stereo Depth Estimation UAV Remote Sensing

Open paper AskPDF arXiv

Autonomous drone-based tree pruning needs accurate, real-time depth estimation from stereo cameras. Depth is computed from disparity maps using

Z = f B/d

, so even small disparity errors cause noticeable depth mistakes at working distances. Building on our earlier work that identified DEFOM-Stereo as the best reference disparity generator for vegetation scenes, we present the first study to train and test ten deep stereo matching networks on real tree branch images. We use the Canterbury Tree Branches dataset -- 5,313 stereo pairs from a ZED Mini camera at 1080P and 720P -- with DEFOM-generated disparity maps as training targets. The ten methods cover step-by-step refinement, 3D convolution, edge-aware attention, and lightweight designs. Using perceptual metrics (SSIM, LPIPS, ViTScore) and structural metrics (SIFT/ORB feature matching), we find that BANet-3D produces the best overall quality (SSIM = 0.883, LPIPS = 0.157), while RAFT-Stereo scores highest on scene-level understanding (ViTScore = 0.799). Testing on an NVIDIA Jetson Orin Super (16 GB, independently powered) mounted on our drone shows that AnyNet reaches 6.99 FPS at 1080P -- the only near-real-time option -- while BANet-2D gives the best quality-speed balance at 1.21 FPS. We also compare 720P and 1080P processing times to guide resolution choices for forestry drone systems.

Abstract

Metric Depth Estimation

Explore similar work

Training Deep Stereo Matching Networks on Tree Branch Imagery: A Benchmark Study for Real-Time UAV Forestry Applications

Progressive Per-Branch Depth Optimization for DEFOM-Stereo and SAM3 Joint Analysis in UAV Forestry Applications

Low-Cost Stereo Vision for Robust 3D Positioning of Thin Radiata Pine Branches in Autonomous Drone Pruning