WildDepth-v2: A Synchronized RGB--LiDAR--Radar Dataset and Benchmark for Metric Depth, Motion, and Tracking of Wildlife
Authors: Muhammad Aamir, Sangyun Shin, Naoya Muramatsu, Matthew Wijers, Amir Patel, Andrew Loveridge, Andrew Markham
Organizations: Department of Computer Science, University of Oxford, Oxford, UK · University of Cape Town, Cape Town, South Africa · Wildlife Conservation Research Unit, Department of Biology, University of Oxford, UK · Department of Computer Science, University College London, UK
Measuring animal distance and motion in metric units underpins abundance estimation, morphometrics, and movement ecology, yet monocular video lacks metric scale and wildlife datasets lack metric ground truth. We present WildDepth 2, extending the synchronized RGB-LiDAR WildDepth dataset in three directions. (i) Imaging radar: 37 RGB-LiDAR-4D-radar sequences (19,716 frames) with per-point Doppler velocity and a measured error budget, to our knowledge the first imaging-radar wildlife recordings. (ii) Benchmarking: a frozen, checksummed hidden test set for RGB-only distance estimation and RGB+sparse-LiDAR depth completion, evaluated separately on animal and background pixels. (iii) Motion and tracking: metric 3D trajectories obtained by lifting 2D tracks using LiDAR, radar, or monocular depth. Across 48,723 animal frames, object-level radar calibration improves monocular depth over scene-level LiDAR calibration without requiring LiDAR at inference (AbsRel 0.120 vs. 0.198), although animal-level LiDAR remains more accurate (0.052). Across 431 tracks, once-per-track radar scaling reduces 3D position error four-fold (2.20 to 0.58 m) and distance-sampling detection-function bias from -13% to -2%, matching once-per-track LiDAR calibration. Transferring radar Doppler onto LiDAR produces motion masks 3.7 times denser than radar alone at 72% precision, reduces moving-animal LiDAR accumulation error by 37%, and estimates radial speed (r = 0.60), albeit with approximately 50% underestimation.
Figures & tables
Dataset
Source
# Species
# Frames
RGB
LiDAR
Radar
AnimalKingdom [ 42 ]
YouTube
850
30k
✓
✗
✗
MammalNet [ 11 ]
YouTube
173
20k
✓
✗
✗
KABR [ 26 ]
Drone
3
13k
✓
✗
✗
Meerkats [ 48 ]
Zoo
1
848k
✓
✗
✗
ChimpACT [ 37 ]
Zoo
1
160k
✓
✗
✗
MammAlps [ 19 ]
Camera traps
5
6k
✓
✗
✗
Table 1: Publicly available wildlife datasets. WildDepth-v2 is the only one with metric depth, and the only one with imaging radar. Frame counts are curated frames.
Figure 1: WildDepth-v2 overview. RGB–LiDAR recordings from three sites (left), a synchronized RGB–LiDAR–radar subset at Longleat in which each radar return carries range, azimuth, elevation, Doppler velocity and RCS (centre), and the benchmark tasks (right).
Subset
Species
Snippets
Frames
Range (m)
Radar
Kgalagadi
26
131
75,741
12–140
✗
Bubye
6
37
84,700
5–50
✗
Longleat
10
70
42,500
5–50
partial
Longleat subset (new)
10
37
19,716 ∗
5–55 †
✓
Hidden test (Sec. 4 )
–
13 scenes
902
5–50
✗
Table 2: WildDepth-v2 statistics. A snippet is a sequence of consecutive frames of the same species (or group) in a consistent setting.
Figure 2: Sensor density on animals, by ground-truth distance: single-sweep LiDAR points inside the SAM2 mask and radar returns associated with the detection box (median over 48,723 animals).
Figure 3: RGB–LiDAR–radar examples. Radar returns (coloured by Doppler velocity) sit on the animals in range and azimuth but are displaced vertically when projected into the image, reflecting the 1–2 m elevation uncertainty of the radar extrinsic.
Task
Input
Output
GT / data
T1 Animal distance
RGB
metric depth map
LiDAR in mask; hidden A
T2 Depth completion
RGB + 1 LiDAR sweep
metric depth map
11-sweep LiDAR; hidden B
T3 Radar metric scale
RGB + radar
metric distance
LiDAR in mask; radar subset
T4 Motion segmentation
LiDAR + radar
moving points
RGB boxes / flow; radar subset
T5 Metric 3D tracking
RGB (+LiDAR/radar)
3D tracks, speed
LiDAR tracks; radar subset
Table 3: WildDepth-v2 tasks. GT = ground truth.
Figure 4: Ground-truth camera-to-animal distance of released-snippet detections versus the hidden test set.
RMSE (m) ↓
AbsRel ↓
SILog ↓
δ<1.25↑
Model
UC
LC
UC
LC
UC
LC
UC
LC
DepthAnything
8.69
4.43
0.38
0.155
0.72
0.245
0.45
0.767
DA-v2
11.91
6.42
0.41
0.210
0.99
0.274
0.39
0.590
DA-v3
10.33
4.14
0.47
0.153
0.82
0.225
0.41
0.787
DepthPro
12.03
5.66
0.56
0.243
0.82
0.323
0.42
0.625
DPT
13.62
5.79
0.72
0.278
0.70
0.293
0.34
0.588
Table 4: Camera-to-animal distance on the released snippets, all bins (11,498 detections, 5–50 m). UC = uncalibrated; LC = per-frame LiDAR-calibrated (scene anchors).
Figure 5: Per-distance camera-to-animal error on the released snippets. Solid: LiDAR-calibrated; dashed: uncalibrated.
Model
LC AbsRel
LC RMSE (m)
LC SILog
LC δ<1.25
UC AbsRel
MoGe-2
0.094
3.25
0.206
0.97
0.335
Depth Anything V2 (metric outdoor)
0.169
4.16
0.168
0.89
0.458
Marigold v1.1 ‡
0.252
6.51
0.247
0.40
0.359
DPT †
0.274
6.57
0.337
0.65
0.871
Depth Pro
0.337
7.14
0.375
0.53
3.064
Depth Anything V2 (relative) †
0.398
9.21
0.470
0.22
0.317
Table 5: Hidden test, Track A (RGB only; 510 detections in 352 frames, 13 scenes), scored by the competition’s scoring program. † Relative disparity inverted to depth. ‡ Affine-invariant depth (UC not meaningful).
Method
RMSE animal
RMSE bg.
AbsRel animal
AbsRel all
δ<1.25
Nearest-neighbour fill
6.10
4.51
0.191
0.122
0.92
MoGe-2 + scale-shift to sweep
5.33
4.52
0.193
0.203
0.80
Table 6: Hidden test, Track B (RGB + one LiDAR sweep; 902 frames). Per-pixel error on animal-mask and background pixels.
Anchor
Sensor
AbsRel [95% CI]
medAE (m)
δ<1.25
cov.
MoGe-2 metric, no anchor
–
0.146 [0.145, 0.148]
1.93
0.80
1.00
UC: 1 LiDAR point (scale)
L
0.442 [0.437, 0.448]
6.53
0.21
1.00
LC-scene: all LiDAR < 55 m
L
0.198 [0.196, 0.201]
2.49
0.55
1.00
LC-box: held-out LiDAR in masks
L
0.052 [0.050, 0.054]
0.17
0.95
1.00
RC-pixel: radar returns per pixel
R
0.483 [0.471, 0.495]
4.54
0.41
1.00
RC-box: object-level radar (affine)
R
0.120 [0.113, 0.131]
0.60
0.85
1.00
Table 7: Metric calibration of MoGe-2 by anchor source: camera-to-animal distance on the test sequences, frames that contain at least one radar-associated animal (20,869 animals). Sensor used at inference: L = LiDAR, R = radar. 95% CIs from a frame-level bootstrap. ∗ The anchor used in an earlier version of this study, which snaps radar returns to their nearest LiDAR point and uses the LiDAR depth.
Radar range estimator
coverage
median rel. err.
AbsRel
median of box returns
0.90
0.064
0.728
nearest range cluster
0.90
0.042
0.349
densest range cluster
0.90
0.052
0.763
MoGe-2-gated ( ± 35%)
0.76
0.026
0.056
Table 8: LiDAR-free radar range estimators, development sequences (errors relative to held-out LiDAR).
Figure 6: Camera-to-animal AbsRel by distance and anchor source (test sequences). Radar range is the best LiDAR-free cue below 30 m and the worst beyond it, where few returns land on the animal.
Radii (m)
τ (m/s)
event frames
densif.
prec. radar
prec. LiDAR
0.6/1.0/1.5 (ours)
0.3
3,815
4.0 ×
0.64
0.75
0.6/1.0/1.5 (ours)
0.5
2,783
3.7 ×
0.65
0.72
0.6/1.0/1.5 (ours)
0.7
1,995
3.2 ×
0.63
0.65
0.6/1.0/1.5 (ours)
1.0
1,375
2.9 ×
0.62
0.57
0.6/0.6/0.6
0.5
2,783
1.8 ×
0.65
0.72
0.6/0.6/0.6
1.0
1,375
1.3 ×
0.62
0.55
Table 9: Doppler transfer from radar to single-sweep LiDAR (static-platform event frames). Densification = moving LiDAR points / moving radar returns; precision = fraction inside animal boxes.
Figure 7: Doppler transfer. Sparse moving radar returns (left, projected with their elevation error) become a dense motion mask on the co-registered LiDAR (right) for walking camels.
Doppler-moving animals
all animals
gate (Doppler)
Sweeps
single
accum.
gated
accum.
Doppler
flow
prec.
rate
5 (0.5 s)
2.02
1.96
2.02
2.05
2.05
2.06
0.54
0.09
15 (1.5 s)
1.81
2.04
1.81
2.13
2.11
2.09
0.72
0.08
30 (3.0 s)
1.64
2.60
1.64
2.25
2.17
2.08
0.85
0.08
Table 10: Doppler-gated LiDAR accumulation: dense-depth MAE (m) against held-out single-sweep LiDAR, by accumulation window. Gate precision = fraction of gated animals whose error decreases.
Depth source
LiDAR
Pos. RMSE (m)
Dist. AbsRel
Speed err. (m/s)
Jitter (m)
σ bias
MoGe-2 metric
–
2.20
0.149
0.15
0.31
-13.2%
UC, per frame
L
6.27
0.399
0.28
1.72
-8.1%
LC-scene, per frame
L
1.52
0.089
0.11
0.28
-3.6%
LiDAR scale, first frame only
L (1 frame)
0.52
0.031
0.10
0.35
+0.2%
Radar range (object level)
R
0.53
0.036
0.13
0.72
-2.1%
MoGe-2 × radar scale (per track)
R
0.58
0.033
0.10
0.35
-2.1%
Table 11: Metric 3D tracking on the radar subset (431 tracks). Errors against LiDAR-lifted trajectories (median over tracks); speed error on 169 tracks with reference speed > 0.3 m/s; jitter = std of the raw frame-to-frame distance change; σ bias = bias of the half-normal detection-function scale fitted to per-track distances (distance sampling).
Figure 8: Per-track detection distances from LiDAR (grey), MoGe-2 alone, MoGe-2 scaled by radar per track, and single-point LiDAR calibration (UC).
Figure 9: In-box Doppler against LiDAR-derived range rate (static platform; median and inter-quartile band per bin).
Figure 10: Radar error budget. Left: residual of static radar returns to the nearest LiDAR point, in the ground plane (range/azimuth; matches gated at 1 m) and vertically. Right: radar–LiDAR timestamp offset.
Method
MAE (m) ↓
R2↑
Planar depth
0.13±1.14
−0.10±0.13
MoGe-2 + SSA
0.16±1.12
0.25±1.36
Table 12: RGB–LiDAR densification, red hartebeest at ≈ 97 m.
Figure 11: Target-point densification: RGB (left) and dense metric point cloud (right) for an elephant and a lion (Longleat) and a red hartebeest (Kgalagadi).
Figure 12: 4D reconstructions of a walking camel (top), giraffe (middle) and Ankole cow (bottom).
Metric depth estimation predicts scene distances in physical units from images, supporting robotics, mapping, augmented reality, and 3D reconstruction.
Depth estimation and 3D reconstruction have been extensively studied as core topics in computer vision. Starting from rigid objects with relatively simple geometric shapes, such as vehicles, the research has expanded to address general objects, including challenging deformable objects, such as humans and animals. However, for the animal, in particular, the majority of existing models are trained based on datasets without metric scale, which can help validate image-only models. To address this limitation, we present WildDepth, a multimodal dataset and benchmark suite for depth estimation, behavior detection, and 3D reconstruction from diverse categories of animals ranging from domestic to wild environments with synchronized RGB and LiDAR. Experimental results show that the use of multi-modal data improves depth reliability by up to 10% RMSE, while RGB-LiDAR fusion enhances 3D reconstruction fidelity by 12% in Chamfer distance. By releasing WildDepth and its benchmarks, we aim to foster robust multimodal perception systems that generalize across domains.
Muhammad Aamir, Naoya Muramatsu, Sangyun Shin +6
University of Oxford · University of Cape Town · University College London
Accurate yet low-latency depth is essential for radar-camera perception in autonomous systems. Cameras provide rich appearance but lack metric scale, whereas automotive radar offers metric range but is sparse and noisy. Many pipelines are multi-stage or depend on auxiliary annotations, increasing latency and limiting portability. We introduce JustDepth, a single-stage radar-camera depth estimator trained only with radar, camera, and single-scan LiDAR. All radar returns are aggregated into a fixed-width 1D representation, decoupling runtime from point count. A Height Fusion Block fuses modalities, a lightweight GNN propagates depth globally, and a training-only confidence decoder stabilizes learning with zero test-time cost. We mitigate stripe artifacts via simple augmentations and quantify them using the Vertical-Horizontal Gradient Ratio (VHGR). On nuScenes, compared to recent state-of-the-art methods, JustDepth maintains accuracy while reducing inference time by 39.7x and stripe artifacts by 66% as measured by VHGR.
Wooyung Yun, Dongwook Kim, Soomok Lee
Department of Artificial Intelligence, Ajou University, Suwon 16499, Republic of Korea · Faculty of Department of Computer Science, Kennesaw State University, Marietta GA, 30060, United States
We introduce WildBox, a dataset and benchmark for monocular 3D detection of wildlife from drone video, comprising 237,505 3D bounding box annotations across seven African savanna species grouped into six benchmark classes. Annotations follow a KITTI/Omni3D-compatible format in a per-segment scale-normalised camera frame, with instance identities maintained across each segment. We evaluate two open-vocabulary monocular 3D architectures, OVMono3D-LIFT and DetAny3D, under zero-shot, ground-truth 2D box prompt, and supervised fine-tuning protocols. Open-vocabulary 2D foundation models provide usable zero-shot wildlife localisation (50.55 AP@50), but zero-shot 3D detection collapses to 0.00 AP across both architectures and every 2D-input condition tested, including ground-truth 2D box prompts, thus isolating the failure to the 3D stage. Fine-tuning on WildBox recovers performance to 8.68 +/- 0.47 [email protected] and 13.17 +/- 0.69 AP3D macro. Depth contributes 84% of normalised Hausdorff distance after fine-tuning and over 99% in zero-shot, identifying monocular aerial depth as the dominant open problem in this regime. A coarse-to-fine curriculum, i.e. pretraining on a merged zebra class before fine-tuning on the Grevy's/plains split, improves macro 3D performance with less total compute, with the largest gains on the two zebra subclasses. WildBox is released with video-level splits, evaluation code, and baseline checkpoints to enable progress in 3D wildlife perception from drone video.
13D Optical Metrology Unit, Fondazione Bruno Kessler, Trento, Italy · 2Computer Vision and Machine Learning Systems group, Institute for Geoinformatics, University of Muenster, Muenster, Germany · School of Civil, Aerospace and Design Engineering, University of Bristol, Bristol, UK +6