LynxScenes: A Large-Scale Real-World Benchmark for Outdoor Wheel-Legged Robot Perception, Navigation, Imagination and Reconstruction

Jun Tian; Hao Wu; Ruo Chen; Yili Liao; Hongyi Lin; Jincheng Yu; Chao Li; Yingjie Niu

cs.ROOct 13, 2026

LynxScenes: A Large-Scale Real-World Benchmark for Outdoor Wheel-Legged Robot Perception, Navigation, Imagination and Reconstruction

Authors: Jun Tian, Hao Wu, Ruo Chen, Yili Liao, Hongyi Lin, Jincheng Yu, Chao Li, Yingjie Niu

Organizations: Deep Robotics · University of Illinois at Urbana-Champaign · The Hong Kong University of Science and Technology · Zhejiang University · Tsinghua University · Massachusetts Institute of Technology

Abstract

Wheel-legged robots operating in outdoor spaces shared with humans require accurate perception, socially compliant navigation, future-scene reasoning, and scene reconstruction. However, existing datasets typically support these capabilities in isolation and rarely provide dense spatial and behavior-level supervision on real wheel-legged platforms. We introduce LynxScenes, a large-scale real-world dataset and benchmark collected with a LYNX M20 wheel-legged robot. It contains approximately 100K 30-s navigation clips spanning 833.6 h and 3,294.7 km, with 3.0M annotated frames. The synchronized recordings include four fisheye cameras, dual LiDARs, IMU, odometry, and robot states. Annotations comprise 20.82M oriented 3D boxes across 27 categories, 13.89M road-structure annotations, and 20 navigation instructions organized into three layers and aligned with continuous future trajectories. LynxScenes supports four complementary tasks: instruction-conditioned trajectory prediction, BEV-based perception of 3D objects and road structures, future scene prediction, and multi-view scene reconstruction. For trajectory prediction, we further introduce a log-replay-based pseudo-closed-loop evaluator that measures safety, progress, and instruction compliance beyond trajectory imitation. Extensive evaluations across representative navigation, perception, world-model, and reconstruction methods reveal substantial remaining challenges in close-range outdoor robot navigation. The dataset and evaluation toolkit are planned for public release.

Figures & tables

Dataset

Real

Outdoor

Legged

LiDAR

3D Obj.

Temp. ID

Road/ Trav.

Sem. Cmd.

Cmd.

\rightarrow

Traj.

Scale

Duration

Distance

Real-world multimodal and robot-navigation datasets

M2DGR ( Yin et al., 2021 )

\checkmark

\checkmark

–

\checkmark

–

36 sequences

3.8 h

10.7 km

RECON ( Shah et al., 2021 )

\checkmark

\checkmark

–

\checkmark

–

>

5K trajectories

>

40 h

N/R

SCAND ( Karnan et al., 2022 )

\checkmark

\checkmark

\checkmark

\checkmark

–

138 trajectories

8.7 h

40.2 km

GND ( Liang et al., 2025 )

\checkmark

\checkmark

–

\checkmark

–

\checkmark

–

10 campuses

11.1 h

53 km

M3ED ( Chaney et al., 2023 )

\checkmark

\checkmark

\checkmark

\checkmark

–

57 sequences

3.4 h

N/R

Table 1: Comparison with representative robot-navigation datasets and benchmarks. “Temp. ID” denotes persistent object identities across temporal observations; “Road/Trav. GT” denotes explicit road-structure, map, or traversability supervision; and “Cmd.

\rightarrow

Traj.” denotes direct conditioning of continuous trajectory or waypoint prediction on semantic navigation instructions. Scale is reported using the primary unit provided by each dataset. “N/R” indicates that a physical duration or traveled distance is not reported, while “–” denotes not applicable or unavailable.

Figure 2: Overview of the data-collection platform, representative navigation behaviors, and multimodal annotations in our benchmark. (a)–(e) show the DEEP Robotics LYNX M20 wheel-legged quadruped, its physical configuration, onboard sensing setup, and representative outdoor navigation scenarios. (f)–(g) illustrate robot behaviors together with temporally aligned semantic instructions covering primary navigation, special motion, and obstacle interaction. (h) visualizes synchronized four-view fisheye observations at different times and the corresponding bird’s-eye-view annotations, including surrounding 3D objects, road structures, and the ego trajectory.

Figure 3: Dataset statistics. (A) Composition of 3D object annotations. (B) Composition of road-structure annotations. (C) Distribution of navigation instructions. (D) Coverage of the three instruction layers. (E) Scene-type distribution. (F) Environment and time-of-day conditions. (G) Object distance and per-frame object counts. (H) Ego speed and trajectory curvature.

Figure 4: Qualitative visualizations of the four benchmark tasks in LynxScenes. (a) Instruction-conditioned trajectory prediction: examples of predicted future trajectories under three navigation commands. (b) BEV-based perception of 3D objects and road structures. (c) Future scene prediction: generated future egocentric observations at different rollout times. (d) Multi-view scene reconstruction: novel-view rendering results under different viewpoint changes.

	Trajectory imitation				Replay-based navigation quality						Overall
Model	ADE $\downarrow$	FDE $\downarrow$	AHE $\downarrow$	FHE $\downarrow$	NC $\uparrow$	WAC $\uparrow$	TTC $\uparrow$	CL $\uparrow$	EP $\uparrow$	IC $\uparrow$	QNS $\uparrow$
VLN-based policies
InternNav	0.079	0.098	0.038	0.062	0.945	0.955	0.871	0.815	0.893	0.818	0.764
NaVILA	0.030	0.059	0.028	0.048	0.968	0.975	0.912	0.838	0.941	0.893	0.843
OmniNav	0.090	0.152	0.033	0.054	0.939	0.949	0.863	0.811	0.866	0.803	0.743
StreamVLN	0.036	0.067	0.032	0.053	0.964	0.972	0.905	0.843	0.933	0.884	0.833

Table 2: Instruction-conditioned trajectory prediction.

Method	Input	Line mAP $\uparrow$	Box mAP $\uparrow$	HDS $\uparrow$
BEVFormer-Tiny †	Cam.+Pose	16.92	52.57	54.39
MapTRv2 †	Camera	13.63	45.90	47.57
BEVFusion	Cam.+LiDAR	19.29	67.11	64.48
SparseDrive	Cam.+Pose	21.08	58.70	61.12

Table 3: BEV-based perception of road structures and 3D objects.

Model	Q-Align I $\uparrow$	Q-Align V $\uparrow$	CLIP I $\uparrow$	CLIP IQA+ $\uparrow$	LAION Aes $\uparrow$	DOVER T $\uparrow$	Smooth. $\uparrow$	Dynamic $\uparrow$	FVD $\downarrow$
Matrix-Game-3	2.759	0.474	76.72	0.380	4.374	0.066	0.978	1.000	964.1
WorldStereo	3.310	0.633	85.73	0.474	4.786	0.050	0.896	0.995	386.2
Cosmos3-Nano	3.340	0.619	87.85	0.588	4.770	0.088	0.969	0.895	411.3
LingBot-World	3.629	0.717	83.11	0.575	4.853	0.078	0.942	1.000	764.7
Wan2.2-TI2V-5B	3.445	0.660	87.73	0.475	4.769	0.094	0.955	1.000	616.3
CogVideoX1.5-5B-I2V	2.734	0.479	85.79	0.386	4.629	0.051	0.921	0.985	692.7

Table 4: Future scene prediction.

Method	$K$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	MS-SSIM $\uparrow$	DISTS $\downarrow$	FSIMc $\uparrow$	GMSD $\downarrow$
ReCoSplat	8–16	15.937	0.2425	0.5113	0.3901	0.2133	0.6364	0.2436
YoNoSplat	8–16	15.390	0.2736	0.6721	0.3096	0.3986	0.5113	0.2928
AnySplat	8–16	14.544	0.2096	0.5119	0.3359	0.2128	0.6429	0.2393
FreeSplatter-S	2	14.537	0.2373	0.6461	0.3099	0.3277	0.5687	0.2656
MVSplat	2	11.777	0.1938	0.6307	0.3008	0.3312	0.5807	0.2774

Table 5: Multi-view scene reconstruction.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Benchmark task pipeline. The same real-world multimodal recordings and structured annotations support four tasks: (A) instruction-conditioned trajectory prediction, (B) BEV-based perception of 3D objects and road structures, (C) future scene prediction, and (D) multi-view scene reconstruction. Task-specific inputs are constructed from the shared recordings; LiDAR, ego-motion, instruction, and camera-pose conditions are used when required by the corresponding model. Outputs are evaluated using task-specific quantitative metrics and qualitative comparisons.

Modality	Number	Specification	Coverage	Native Rate
Fisheye Camera	4	$1920\times 1536$	Front/Rear/Left/Right	10 Hz
3D LiDAR	2	RoboSense Airy, 96 ch.	Front/Rear	10 Hz
IMU	1	Inertial measurements	Ego state	200 Hz
LIO/Odometry	1	6-DoF pose	Ego motion	$\sim$ 200 Hz
Motion State	1	$v_{x}$ , $v_{y}$ , $\omega_{z}$	Ego motion	$\sim$ 20 Hz
GPS	1	Global positioning	Reference only	$\sim$ 2 Hz

Appendix

Table 6: Sensing and state modalities of the data-collection platform.

Figure 6: Camera–LiDAR calibration validation. LiDAR point clouds are projected onto the calibrated fisheye camera views to qualitatively verify cross-modal alignment. Projected points are color-coded by depth along the camera axis, with red indicating nearer points and blue indicating farther points.

3D objects (27)
Group	Categories
Human	Pedestrian, Pedestrian Accessories
Mobility	Bicycle, Bicycle Rider, Scooter, Scooter Rider, Motorcycle, Motorcycle Rider, Tricycle, Tricycle Rider
Vehicle	Car, SUV, Van, Truck, Bus
Traffic Facility	Cone, Traffic Barrier, Road Barrel, Reflective Triangle, Fire Hydrant
Other	Robot Dog, Platform Cart, Construction Cart, Excavator, Forklift, Dog, Cat

Appendix

Table 7: Annotation taxonomy: 27 object categories, 4 road-structure classes, and 20 navigation instructions in three layers.

Figure 7: Qualitative trajectory predictions with NaVILA. Columns correspond to three navigation instructions: continue straight at junction, take left branch, and take right branch. Red curves denote predicted trajectories, while green curves denote ground-truth trajectories.

Figure 8: Qualitative line detection comparison on a simple straight-road test scene. The panels show BEVFormer-Tiny (top left), BEVFusion (top right), MapTRv2 (bottom left), and SparseDrive (bottom right). Ground-truth road geometry is shown in cyan, yellow, green, and blue; predictions are shown in magenta and orange. Predictions with confidence at least 0.25 are displayed. The candidate pool was constructed using ground-truth content only.

Model	Model Type	Visual Input	Motion / Text Condition	FPS
Matrix-Game-3	Action-conditioned world model	$I_{0}$	Trajectory-derived keyboard/mouse actions	17
WorldStereo	Camera-controlled world model	$I_{0}$	Camera intrinsics + W2C trajectory	16
Cosmos3-Nano	Action-conditioned world model	$I_{0}$	Resampled 9-D motion sequence	10
LingBot-World	Camera-controlled world model	$I_{0}$	Interpolated camera trajectory	16
Wan2.2-TI2V-5B	Generic image-to-video	$I_{0}$	Forward-motion text prompt	24
CogVideoX1.5-5B-I2V	Generic image-to-video	$I_{0}$	Forward-motion text prompt	16

Appendix

Table 8: World-model baselines and their evaluation interfaces.

I_{0}

denotes the first rectified front-view frame. Native frame rates and conditioning formats are retained during generation.

Figure 9: Qualitative 3DGS multi-view scene reconstruction under viewpoint changes. (a) Renderings from increasing forward camera offsets and (b) renderings at different camera heights within the same reconstructed scene. $d$ and $h$ denote the relative camera translation and height with respect to the reference viewpoint.

Explore similar work

GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation
Feb 20, 2026Turcan Tuna, Jonas Frey, Frank Fu +5Legged Robotics Simultaneous Localization and Mapping
Robotic Systems Lab, ETH Zurich, Zurich 8092, Switzerland · 1ETH Zurich, Switzerland · 2Stanford University, USA +1
STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation
Mar 10, 2026Konyul Park, Daehun Kim, Jiyong Oh +7Off-Road Robot Navigation 3D Occupancy Prediction
Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, 08826, Korea. · Department of Electrical and Computer Engineering, Seoul National University, Seoul, 08826, Korea.
DR-VLA: Learning Human-Like Quadruped Navigation from Egocentric Web Videos via Domain-Invariant Scene Representations
Oct 13, 2026Jun Tian, Yingjie Niu, Ruo Chen +2Robot Navigation Cross-Embodiment Robot Learning
Deep Robotics · University of Illinois at Urbana-Champaign · Zhejiang University +2

Feb 20, 2026J/K move · Enter open · S save

GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation

Turcan Tuna, Jonas Frey, Frank Fu, Katharine Patterson+4

Robotic Systems Lab, ETH Zurich, Zurich 8092, Switzerland · 1ETH Zurich, Switzerland · 2Stanford University, USA+1

Legged Robotics Simultaneous Localization and Mapping Robotic Data Collection Robot State Estimation Robotic Perception

Open paper AskPDF arXiv

Accurate state estimation and multi-modal perception are prerequisites for autonomous legged robots in complex, large-scale environments. To date, no large-scale public legged-robot dataset captures the real-world conditions needed to develop and benchmark algorithms for legged-robot state estimation, perception, and navigation. To address this, we introduce the GrandTour dataset, a multi-modal legged-robotics dataset collected across challenging outdoor and indoor environments, featuring an ANYbotics ANYmal-D quadruped equipped with the Boxi multi-modal sensor payload. GrandTour spans a broad range of environments and operational scenarios across distinct test sites, ranging from alpine scenery and forests to demolished buildings and urban areas, and covers a wide variation in scale, complexity, illumination, and weather conditions. The dataset provides time-synchronized sensor data from spinning LiDARs, multiple RGB cameras with complementary characteristics, proprioceptive sensors, and stereo depth cameras. Moreover, it includes high-precision ground-truth trajectories from satellite-based RTK-GNSS and a Leica Geosystems total station. This dataset supports research in SLAM, high-precision state estimation, and multi-modal learning, enabling rigorous evaluation and development of new approaches to sensor fusion in legged robotic systems. With its extensive scope, GrandTour represents the largest open-access legged-robotics dataset to date. The dataset is available at https://grand-tour.leggedrobotics.com on HuggingFace (ROS-independent), and in ROS formats, along with tools and demo resources.

Abstract

Explore similar work

GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation

STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation

DR-VLA: Learning Human-Like Quadruped Navigation from Egocentric Web Videos via Domain-Invariant Scene Representations