WildDepth-v2: A Synchronized RGB--LiDAR--Radar Dataset and Benchmark for Metric Depth, Motion, and Tracking of Wildlife

Muhammad Aamir; Sangyun Shin; Naoya Muramatsu; Matthew Wijers; Amir Patel; Andrew Loveridge; Andrew Markham

cs.CVOct 22, 2026

WildDepth-v2: A Synchronized RGB--LiDAR--Radar Dataset and Benchmark for Metric Depth, Motion, and Tracking of Wildlife

Authors: Muhammad Aamir, Sangyun Shin, Naoya Muramatsu, Matthew Wijers, Amir Patel, Andrew Loveridge, Andrew Markham

Organizations: Department of Computer Science, University of Oxford, Oxford, UK · University of Cape Town, Cape Town, South Africa · Wildlife Conservation Research Unit, Department of Biology, University of Oxford, UK · Department of Computer Science, University College London, UK

Abstract

Measuring animal distance and motion in metric units underpins abundance estimation, morphometrics, and movement ecology, yet monocular video lacks metric scale and wildlife datasets lack metric ground truth. We present WildDepth 2, extending the synchronized RGB-LiDAR WildDepth dataset in three directions. (i) Imaging radar: 37 RGB-LiDAR-4D-radar sequences (19,716 frames) with per-point Doppler velocity and a measured error budget, to our knowledge the first imaging-radar wildlife recordings. (ii) Benchmarking: a frozen, checksummed hidden test set for RGB-only distance estimation and RGB+sparse-LiDAR depth completion, evaluated separately on animal and background pixels. (iii) Motion and tracking: metric 3D trajectories obtained by lifting 2D tracks using LiDAR, radar, or monocular depth. Across 48,723 animal frames, object-level radar calibration improves monocular depth over scene-level LiDAR calibration without requiring LiDAR at inference (AbsRel 0.120 vs. 0.198), although animal-level LiDAR remains more accurate (0.052). Across 431 tracks, once-per-track radar scaling reduces 3D position error four-fold (2.20 to 0.58 m) and distance-sampling detection-function bias from -13% to -2%, matching once-per-track LiDAR calibration. Transferring radar Doppler onto LiDAR produces motion masks 3.7 times denser than radar alone at 72% precision, reduces moving-animal LiDAR accumulation error by 37%, and estimates radial speed (r = 0.60), albeit with approximately 50% underestimation.

Figures & tables

Dataset	Source	# Species	# Frames	RGB	LiDAR	Radar
AnimalKingdom [ 42 ]	YouTube	850	30k	✓	✗	✗
MammalNet [ 11 ]	YouTube	173	20k	✓	✗	✗
KABR [ 26 ]	Drone	3	13k	✓	✗	✗
Meerkats [ 48 ]	Zoo	1	848k	✓	✗	✗
ChimpACT [ 37 ]	Zoo	1	160k	✓	✗	✗
MammAlps [ 19 ]	Camera traps	5	6k	✓	✗	✗

Table 1: Publicly available wildlife datasets. WildDepth-v2 is the only one with metric depth, and the only one with imaging radar. Frame counts are curated frames.

Figure 1: WildDepth-v2 overview. RGB–LiDAR recordings from three sites (left), a synchronized RGB–LiDAR–radar subset at Longleat in which each radar return carries range, azimuth, elevation, Doppler velocity and RCS (centre), and the benchmark tasks (right).

Subset	Species	Snippets	Frames	Range (m)	Radar
Kgalagadi	26	131	75,741	12–140	✗
Bubye	6	37	84,700	5–50	✗
Longleat	10	70	42,500	5–50	partial
Longleat subset (new)	10	37	19,716 ∗	5–55 †	✓
Hidden test (Sec. 4 )	–	13 scenes	902	5–50	✗

Table 2: WildDepth-v2 statistics. A snippet is a sequence of consecutive frames of the same species (or group) in a consistent setting.

Figure 2: Sensor density on animals, by ground-truth distance: single-sweep LiDAR points inside the SAM2 mask and radar returns associated with the detection box (median over 48,723 animals).

Figure 3: RGB–LiDAR–radar examples. Radar returns (coloured by Doppler velocity) sit on the animals in range and azimuth but are displaced vertically when projected into the image, reflecting the 1–2 m elevation uncertainty of the radar extrinsic.

Task	Input	Output	GT / data
T1 Animal distance	RGB	metric depth map	LiDAR in mask; hidden A
T2 Depth completion	RGB + 1 LiDAR sweep	metric depth map	11-sweep LiDAR; hidden B
T3 Radar metric scale	RGB + radar	metric distance	LiDAR in mask; radar subset
T4 Motion segmentation	LiDAR + radar	moving points	RGB boxes / flow; radar subset
T5 Metric 3D tracking	RGB (+LiDAR/radar)	3D tracks, speed	LiDAR tracks; radar subset

Table 3: WildDepth-v2 tasks. GT = ground truth.

Figure 4: Ground-truth camera-to-animal distance of released-snippet detections versus the hidden test set.

	RMSE (m) $\downarrow$		AbsRel $\downarrow$		SILog $\downarrow$		$\delta{<}1.25\uparrow$
Model	UC	LC	UC	LC	UC	LC	UC	LC
DepthAnything	8.69	4.43	0.38	0.155	0.72	0.245	0.45	0.767
DA-v2	11.91	6.42	0.41	0.210	0.99	0.274	0.39	0.590
DA-v3	10.33	4.14	0.47	0.153	0.82	0.225	0.41	0.787
DepthPro	12.03	5.66	0.56	0.243	0.82	0.323	0.42	0.625
DPT	13.62	5.79	0.72	0.278	0.70	0.293	0.34	0.588

Table 4: Camera-to-animal distance on the released snippets, all bins (11,498 detections, 5–50 m). UC = uncalibrated; LC = per-frame LiDAR-calibrated (scene anchors).

Panel of Figure 5 — Figure 5: Per-distance camera-to-animal error on the released snippets. Solid: LiDAR-calibrated; dashed: uncalibrated.

Model	LC AbsRel	LC RMSE (m)	LC SILog	LC $\delta{<}1.25$	UC AbsRel
MoGe-2	0.094	3.25	0.206	0.97	0.335
Depth Anything V2 (metric outdoor)	0.169	4.16	0.168	0.89	0.458
Marigold v1.1 ‡	0.252	6.51	0.247	0.40	0.359
DPT †	0.274	6.57	0.337	0.65	0.871
Depth Pro	0.337	7.14	0.375	0.53	3.064
Depth Anything V2 (relative) †	0.398	9.21	0.470	0.22	0.317

Table 5: Hidden test, Track A (RGB only; 510 detections in 352 frames, 13 scenes), scored by the competition’s scoring program. † Relative disparity inverted to depth. ‡ Affine-invariant depth (UC not meaningful).

Method	RMSE animal	RMSE bg.	AbsRel animal	AbsRel all	$\delta{<}1.25$
Nearest-neighbour fill	6.10	4.51	0.191	0.122	0.92
MoGe-2 + scale-shift to sweep	5.33	4.52	0.193	0.203	0.80

Table 6: Hidden test, Track B (RGB + one LiDAR sweep; 902 frames). Per-pixel error on animal-mask and background pixels.

Anchor	Sensor	AbsRel [95% CI]	medAE (m)	$\delta{<}1.25$	cov.
MoGe-2 metric, no anchor	–	0.146 [0.145, 0.148]	1.93	0.80	1.00
UC: 1 LiDAR point (scale)	L	0.442 [0.437, 0.448]	6.53	0.21	1.00
LC-scene: all LiDAR $<$ 55 m	L	0.198 [0.196, 0.201]	2.49	0.55	1.00
LC-box: held-out LiDAR in masks	L	0.052 [0.050, 0.054]	0.17	0.95	1.00
RC-pixel: radar returns per pixel	R	0.483 [0.471, 0.495]	4.54	0.41	1.00
RC-box: object-level radar (affine)	R	0.120 [0.113, 0.131]	0.60	0.85	1.00

Table 7: Metric calibration of MoGe-2 by anchor source: camera-to-animal distance on the test sequences, frames that contain at least one radar-associated animal (20,869 animals). Sensor used at inference: L = LiDAR, R = radar. 95% CIs from a frame-level bootstrap. ∗ The anchor used in an earlier version of this study, which snaps radar returns to their nearest LiDAR point and uses the LiDAR depth.

Radar range estimator	coverage	median rel. err.	AbsRel
median of box returns	0.90	0.064	0.728
nearest range cluster	0.90	0.042	0.349
densest range cluster	0.90	0.052	0.763
MoGe-2-gated ( $\pm$ 35%)	0.76	0.026	0.056

Table 8: LiDAR-free radar range estimators, development sequences (errors relative to held-out LiDAR).

Figure 6: Camera-to-animal AbsRel by distance and anchor source (test sequences). Radar range is the best LiDAR-free cue below 30 m and the worst beyond it, where few returns land on the animal.

Radii (m)	$\tau$ (m/s)	event frames	densif.	prec. radar	prec. LiDAR
0.6/1.0/1.5 (ours)	0.3	3,815	4.0 $\times$	0.64	0.75
0.6/1.0/1.5 (ours)	0.5	2,783	3.7 $\times$	0.65	0.72
0.6/1.0/1.5 (ours)	0.7	1,995	3.2 $\times$	0.63	0.65
0.6/1.0/1.5 (ours)	1.0	1,375	2.9 $\times$	0.62	0.57
0.6/0.6/0.6	0.5	2,783	1.8 $\times$	0.65	0.72
0.6/0.6/0.6	1.0	1,375	1.3 $\times$	0.62	0.55

Table 9: Doppler transfer from radar to single-sweep LiDAR (static-platform event frames). Densification = moving LiDAR points / moving radar returns; precision = fraction inside animal boxes.

Figure 7: Doppler transfer. Sparse moving radar returns (left, projected with their elevation error) become a dense motion mask on the co-registered LiDAR (right) for walking camels.

	Doppler-moving animals			all animals			gate (Doppler)
Sweeps	single	accum.	gated	accum.	Doppler	flow	prec.	rate
5 (0.5 s)	2.02	1.96	2.02	2.05	2.05	2.06	0.54	0.09
15 (1.5 s)	1.81	2.04	1.81	2.13	2.11	2.09	0.72	0.08
30 (3.0 s)	1.64	2.60	1.64	2.25	2.17	2.08	0.85	0.08

Table 10: Doppler-gated LiDAR accumulation: dense-depth MAE (m) against held-out single-sweep LiDAR, by accumulation window. Gate precision = fraction of gated animals whose error decreases.

Depth source	LiDAR	Pos. RMSE (m)	Dist. AbsRel	Speed err. (m/s)	Jitter (m)	$\sigma$ bias
MoGe-2 metric	–	2.20	0.149	0.15	0.31	-13.2%
UC, per frame	L	6.27	0.399	0.28	1.72	-8.1%
LC-scene, per frame	L	1.52	0.089	0.11	0.28	-3.6%
LiDAR scale, first frame only	L (1 frame)	0.52	0.031	0.10	0.35	+0.2%
Radar range (object level)	R	0.53	0.036	0.13	0.72	-2.1%
MoGe-2 $\times$ radar scale (per track)	R	0.58	0.033	0.10	0.35	-2.1%

Table 11: Metric 3D tracking on the radar subset (431 tracks). Errors against LiDAR-lifted trajectories (median over tracks); speed error on 169 tracks with reference speed

>

0.3 m/s; jitter = std of the raw frame-to-frame distance change;

\sigma

bias = bias of the half-normal detection-function scale fitted to per-track distances (distance sampling).

Figure 8: Per-track detection distances from LiDAR (grey), MoGe-2 alone, MoGe-2 scaled by radar per track, and single-point LiDAR calibration (UC).

Figure 9: In-box Doppler against LiDAR-derived range rate (static platform; median and inter-quartile band per bin).

Figure 10: Radar error budget. Left: residual of static radar returns to the nearest LiDAR point, in the ground plane (range/azimuth; matches gated at 1 m) and vertically. Right: radar–LiDAR timestamp offset.

Method	MAE (m) $\downarrow$	$R^{2}\uparrow$
Planar depth	$0.13\pm 1.14$	$-0.10\pm 0.13$
MoGe-2 + SSA	$0.16\pm 1.12$	$0.25\pm 1.36$

Table 12: RGB–LiDAR densification, red hartebeest at

{\approx}

97 m.

Figure 11: Target-point densification: RGB (left) and dense metric point cloud (right) for an elephant and a lion (Longleat) and a red hartebeest (Kgalagadi).

Figure 12: 4D reconstructions of a walking camel (top), giraffe (middle) and Ankole cow (bottom).

Metric Depth Estimation

Metric depth estimation predicts scene distances in physical units from images, supporting robotics, mapping, augmented reality, and 3D reconstruction.

See all 64 papers

WildDepth: A Multimodal Dataset for 3D Wildlife Perception and Depth Estimation
Mar 17, 2026Muhammad Aamir, Naoya Muramatsu, Sangyun Shin +6Depth Estimation 3D Reconstruction
University of Oxford · University of Cape Town · University College London
JustDepth: Real-Time Radar-Camera Depth Estimation with Single-Scan LiDAR Supervision
Jul 24, 2026Wooyung Yun, Dongwook Kim, Soomok LeeCamera-Radar Fusion Metric Depth Estimation
Department of Artificial Intelligence, Ajou University, Suwon 16499, Republic of Korea · Faculty of Department of Computer Science, Kennesaw State University, Marietta GA, 30060, United States
WildBox: A Dataset and Benchmark for Aerial Monocular 3D Detection of African Savanna Wildlife
Jun 19, 2026Vandita Shukla, Kilian Meier, Lucie Laporte-Devylder +6Monocular Depth Estimation 3D Object Detection
13D Optical Metrology Unit, Fondazione Bruno Kessler, Trento, Italy · 2Computer Vision and Machine Learning Systems group, Institute for Geoinformatics, University of Muenster, Muenster, Germany · School of Civil, Aerospace and Design Engineering, University of Bristol, Bristol, UK +6

Mar 17, 2026J/K move · Enter open · S save

WildDepth: A Multimodal Dataset for 3D Wildlife Perception and Depth Estimation

Muhammad Aamir, Naoya Muramatsu, Sangyun Shin, Matthew Wijers+5

University of Oxford · University of Cape Town · University College London

Depth Estimation 3D Reconstruction Camera-LiDAR Fusion

Open paper AskPDF arXiv

Depth estimation and 3D reconstruction have been extensively studied as core topics in computer vision. Starting from rigid objects with relatively simple geometric shapes, such as vehicles, the research has expanded to address general objects, including challenging deformable objects, such as humans and animals. However, for the animal, in particular, the majority of existing models are trained based on datasets without metric scale, which can help validate image-only models. To address this limitation, we present WildDepth, a multimodal dataset and benchmark suite for depth estimation, behavior detection, and 3D reconstruction from diverse categories of animals ranging from domestic to wild environments with synchronized RGB and LiDAR. Experimental results show that the use of multi-modal data improves depth reliability by up to 10% RMSE, while RGB-LiDAR fusion enhances 3D reconstruction fidelity by 12% in Chamfer distance. By releasing WildDepth and its benchmarks, we aim to foster robust multimodal perception systems that generalize across domains.

Abstract

Metric Depth Estimation

Explore similar work

WildDepth: A Multimodal Dataset for 3D Wildlife Perception and Depth Estimation

JustDepth: Real-Time Radar-Camera Depth Estimation with Single-Scan LiDAR Supervision

WildBox: A Dataset and Benchmark for Aerial Monocular 3D Detection of African Savanna Wildlife