DR-VLA: Learning Human-Like Quadruped Navigation from Egocentric Web Videos via Domain-Invariant Scene Representations

Jun Tian; Yingjie Niu; Ruo Chen; Hongyi Lin; Chao Li

cs.ROOct 13, 2026

DR-VLA: Learning Human-Like Quadruped Navigation from Egocentric Web Videos via Domain-Invariant Scene Representations

Authors: Jun Tian, Yingjie Niu, Ruo Chen, Hongyi Lin, Chao Li

Organizations: Deep Robotics · University of Illinois at Urbana-Champaign · Zhejiang University · Tsinghua University · Massachusetts Institute of Technology

Abstract

Quadruped robots performing outdoor deliveries must interpret complex scenes, anticipate changes, and navigate in socially compliant ways. Robot demonstrations are costly to collect at the scale needed for generalization. Egocentric web videos offer abundant human navigation experience, but differences in viewpoint and motion make this experience difficult to transfer to robots. We introduce DR-VLA, a Mixture-of-Transformers framework that transfers human navigation experience through a domain-invariant scene representation (DISR). The framework combines vision-language understanding and latent world modeling with a flow-matching action expert. We introduce EgoLynx, comprising 10,000 hours of curated egocentric web video (EgoLynx-Web) and 2,000 clips of teleoperated robot navigation collected during real-world deliveries (EgoLynx-Robot). We pretrain the understanding and world experts on EgoLynx-Web without action labels, then adapt the model to robot observations and train the action expert on EgoLynx-Robot to predict executable trajectories. We also propose the Quadruped Navigation Score (QNS), a pseudo-closed-loop metric combining safety, instruction compliance, comfort, and progress. On the held-out EgoLynx-Robot test split, DR-VLA improves QNS by approximately 5% and reduces open-loop final heading error (FHE) by 14.6%, each relative to the best competing result for that metric. Ablations show that web-video pretraining improves generalization and that the model's key components contribute to performance. As a future-prediction target, DISR outperforms five generic visual representations and yields performance close to task-specific 3D perception without dedicated geometric labels or point clouds for target construction. Trained separately on NAVSIM v1 without web pretraining, DR-VLA achieves 92.1 PDMS, suggesting that the framework also applies to autonomous driving.

Figures & tables

Figure 1: Overview. DR-VLA learns to understand and anticipate scenes from egocentric web videos (left) and to act from teleoperated quadruped clips (center bottom). Three experts (center top) map four fisheye views and a navigation instruction to a trajectory and to future domain-invariant scene representations (DISR), which encode semantic, spatial, and relational structure (right).

Figure 2: DR-VLA and the DISR Pretraining. Top: Combined Spatial and Semantic information are encoded by $E_{\phi}$ into the DISR latent $\mathbf{z}$ and reconstructed by $D_{\phi}$ . Bottom left: the understanding, world, and action experts share attention and are trained with $\mathcal{L}_{\text{cap}}$ , $\mathcal{L}_{\text{world}}$ (targets from the frozen $E_{\phi}$ ), and $\mathcal{L}_{\text{act}}$ . Bottom right: block-wise attention mask.

VLN-based policies
	Open-loop				Pseudo-closed-loop
Model	ADE $\downarrow$	FDE $\downarrow$	AHE $\downarrow$	FHE $\downarrow$	NC $\uparrow$	WAC $\uparrow$	TTC $\uparrow$	CL $\uparrow$	EP $\uparrow$	IC $\uparrow$	QNS $\uparrow$
InternNav ( Wei et al., 2026a )	0.079	0.098	0.038	0.062	0.945	0.955	0.871	0.815	0.893	0.818	0.764
NaVILA ( Cheng et al., 2025 )	0.030	0.059	0.028	0.048	0.968	0.975	0.912	0.838	0.941	0.893	0.843
OmniNav ( Xue et al., 2026 )	0.090	0.152	0.033	0.054	0.939	0.949	0.863	0.811	0.866	0.803	0.743
StreamVLN ( Wei et al., 2026b )	0.036	0.067	0.032	0.053	0.964	0.972	0.905	0.843	0.933	0.884	0.833

Table 1: Quantitative navigation results on the EgoLynx-Robot test split. ADE/FDE and AHE/FHE denote average/final displacement and heading errors, respectively. The pseudo-closed-loop metrics comprise no collision (NC), walkable-area compliance (WAC), time-to-collision (TTC), clearance (CL), ego progress (EP), instruction compliance (IC), and the Quadruped Navigation Score (QNS).

Figure 3: Predicted trajectories in unseen scenes. Predictions are shown on replayed observations from four scenes (columns). The rows correspond to variants trained on EgoLynx-Web only (using DPVO pseudo-trajectories), EgoLynx-Robot only, or both.

Representation	Use	Sensors	Box/Line labels	Front end	QNS $\uparrow$	Latency (ms) $\downarrow$
DISR	Target	Camera	No	None	0.884	120
OCC+BEV	Target	Camera	Yes	None	0.885	127
DISR	Input	Camera	No	DISR extraction	0.897	220
OCC+BEV	Input	Camera + PC	Yes	OCC+BEV	0.906	347

Table 3: DISR and OCC+BEV as targets or inputs on EgoLynx-Robot. Sensors are inference-time inputs. PC denotes sensor point clouds, used only offline in OCC+BEV-target. Box/Line labels are additional geometric supervision. Front end denotes online representation extraction. Latency includes the full perception and policy pipeline. The DISR-target QNS is from Table 1 .

Method	Input	Backbone	NC	DAC	TTC	Comf.	EP	PDMS
TransFuser ( Chitta et al., 2023 )	C+L	ResNet-34	97.7	92.8	92.8	100	79.2	84.0
DiffusionDrive ( Liao et al., 2025 )	C+L	ResNet-34	98.2	96.2	94.7	100	82.2	88.1
WoTE ( Li et al., 2025b )	C+L	ResNet-34	98.5	96.8	94.9	99.9	81.9	88.3
DiffusionDriveV2 ( Zou et al., 2025 )	C+L	ResNet-34	98.3	97.9	94.8	99.9	87.5	91.2
PRIX ( Wozniak et al., 2026 )	C	ResNet-34	98.1	96.3	94.1	100	82.3	87.8
iPad ( Guo et al., 2026 )	C	ResNet-34	98.6	98.3	94.9	100	88.0	91.7

Table 4: Results on NAVSIM v1 navtest ( Dauner et al., 2024 ) . Sub-scores and PDMS of prior work are taken from the cited papers. “C” cameras, “L” LiDAR.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Table 7

Explore similar work

Co-training with Ego-centric Video and Demonstration for Robot Navigation Task
Jun 1, 2026Shoya Kuno, Yumo Ouchi, Kanata SuzukiMobile Robotics Robot Navigation
Spatial Robotics Research Center, Fujitsu Limited., Kanagawa 211-8588, Japan.
ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining
Jun 15, 2026Hao Li, Ganlong Zhao, Yufei Liu +8Cross-Embodiment Robot Learning Vision-Language-Action Models
ACE Robotics · CUHK MMLab · SJTU +2
Implicit Geometry Representations for Vision-and-Language Navigation from Web Videos
Mar 10, 2026Mingfei Han, Haihong Hao, Liang Ma +6Vision-Language Navigation 3D Scene Representation
Department of Computer Vision, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates. · School of Information Science and Technology, University of Science and Technology of China. · Shenzhen Campus of Sun Yat-Sen University.

Jun 1, 2026J/K move · Enter open · S save

Co-training with Ego-centric Video and Demonstration for Robot Navigation Task

Shoya Kuno, Yumo Ouchi, Kanata Suzuki

Spatial Robotics Research Center, Fujitsu Limited., Kanagawa 211-8588, Japan.

Mobile Robotics Robot Navigation Cross-Embodiment Robot Learning Learning from Demonstration Vision-Language-Action Models Robot Imitation Learning

Open paper AskPDF arXiv

Vision-language-action (VLA) models are promising for diverse robotic tasks, but their performance heavily depends on large-scale high-quality training data, whose collection on real robots is costly and time-consuming. While prior work has explored augmenting manipulation datasets with egocentric human videos, applying such approaches to mobile robot navigation remains challenging due to viewpoint changes during locomotion. In this paper, we propose a framework that converts egocentric walking videos into datasets for mobile robot imitation learning. The proposed method estimates camera motion from human videos and transforms it into action representations compatible with ground mobile robots. By jointly training a VLA model on human-derived and robot-collected datasets, the model achieves improved language understanding and more robust action generation than training with either data source alone. Experiments on a fruit-search navigation task demonstrate that human egocentric videos provide an effective and scalable data source for mobile robot learning.

Abstract

Explore similar work

Co-training with Ego-centric Video and Demonstration for Robot Navigation Task

ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining

Implicit Geometry Representations for Vision-and-Language Navigation from Web Videos