GeoLoco: Leveraging 3D Geometric Priors from Visual Foundation Model for Robust RGB-Only Humanoid Locomotion

Yufei Liu; Xieyuanli Chen; Hainan Pan; Chenghao Shi; Yanjie Chen; Kaihong Huang; Zhiwen Zeng; Huimin Lu

cs.ROMar 8, 2026

GeoLoco: Leveraging 3D Geometric Priors from Visual Foundation Model for Robust RGB-Only Humanoid Locomotion

Authors: Yufei Liu, Xieyuanli Chen, Hainan Pan, Chenghao Shi, Yanjie Chen, Kaihong Huang, Zhiwen Zeng, Huimin Lu

Organizations: College of Intelligence Science and Technology, National University of Defense Technology, China

Abstract

The prevailing paradigm of perceptive humanoid locomotion relies heavily on active depth sensors. However, this depth-centric approach fundamentally discards the rich semantic and dense appearance cues of the visual world, severing low-level control from the high-level reasoning essential for general embodied intelligence. While monocular RGB offers a ubiquitous, information-dense alternative, end-to-end reinforcement learning from raw 2D pixels suffers from extreme sample inefficiency and catastrophic sim-to-real collapse due to the inherent loss of geometric scale. To break this deadlock, we propose GeoLoco, a purely RGB-driven locomotion framework that conceptualizes monocular images as high-dimensional 3D latent representations by harnessing the powerful geometric priors of a frozen, scale-aware Visual Foundation Model (VFM). Rather than naive feature concatenation, we design a proprioceptive-query multi-head cross-attention mechanism that dynamically attends to task-critical topological features conditioned on the robot's real-time gait phase. Crucially, to prevent the policy from overfitting to superficial textures, we introduce a dual-head auxiliary learning scheme. This explicit regularization forces the high-dimensional latent space to strictly align with the physical terrain geometry, ensuring robust zero-shot sim-to-real transfer. Trained exclusively in simulation, GeoLoco achieves robust zero-shot transfer to the Unitree G1 humanoid and successfully negotiates challenging terrains.

Figures & tables

Fig. 1: Overview of GeoLoco. Our framework enables robust humanoid locomotion on diverse terrains—including stairs, ramps, and uneven blocks—using only a monocular RGB camera. By conceptualizing 2D pixels as high-dimensional 3D geometric representations, GeoLoco eliminates the dependency on active depth sensors (LiDAR/Depth) while maintaining expert-level traversal performance.

Fig. 2: The GeoLoco Architecture. A frozen, scale-aware Visual Geometry Encoder extracts multi-scale 3D priors from asynchronous RGB streams (10 Hz). These features are fused with high-frequency proprioceptive (50 Hz) via a Multi-Head Cross-Attention mechanism. To bridge the sim-to-real gap, a dual-head auxiliary decoder (middle right) regularizes the latent space by reconstructing local terrain topography and predicting system dynamics during training.

Fig. 3: Multi-Scale geometric activations from the frozen VFM. We visualize patch token activations from intermediate transformer layers (4/8/12).

Fig. 4: Visualization of attention heatmap produced by the proprioceptive-query cross-attention module for a representative stair scene.

Parameter	Randomization Range	Unit
Dome Light Intensity	$[300,2000]$	lux
Directional Light Intensity	$[1000,6000]$	lux
Light Color Variation	$\pm 30$	%
Lighting Angles	$[0,2\pi]$	rad
Terrain Material	Random Texture Set	–
Camera Exposure/Contrast	$[0.8,1.2]$	factor

TABLE I: Visual Domain Randomization Parameters

Fig. 5: Training environments featuring diverse geometries and randomized textures. This prevents overfitting to visual appearances, ensuring the policy learns robust 3D priors for sim-to-real transfer.

Medium Terrain Difficulty
Method	Modality	Stairs-Up $\uparrow$		Stairs Down		Slope		Gap
Method	Modality	$R_{\mathrm{succ}}(\%,\uparrow)$	$R_{\mathrm{trav}}(\%,\uparrow)$	$R_{\mathrm{succ}}(\%,\uparrow)$	$R_{\mathrm{trav}}(\%,\uparrow)$	$R_{\mathrm{succ}}(\%,\uparrow)$	$R_{\mathrm{trav}}(\%,\uparrow)$	$R_{\mathrm{succ}}(\%,\uparrow)$	$R_{\mathrm{trav}}(\%,\uparrow)$
DreamWaq [ 3 ]	Blind	41.28	44.10	61.73	64.95	79.62	82.88	3.84	4.92
PIE [ 17 ]	Depth	90.63	92.10	93.22	94.58	98.91	99.42	92.44	93.76
MoRE [ 18 ]	Depth	81.94	83.21	84.10	85.67	96.72	97.88	82.35	83.90
PLANC [ 19 ]	Height map	91.37	92.76	94.05	95.44	99.36	99.72	93.18	94.92

TABLE II: Benchmarked Comparison in Simulation. We compare GeoLoco against blind, depth-based, and other RGB-based baselines across various terrain difficulties. Best RGB-based result in bold.

Fig. 6: Real-world zero-shot deployment on Unitree G1. The policy successfully traverses challenging geometric discontinuities under varied conditions, including (a) dark background stairs, (b) a gap obstacle, (c) high-riser stairs, and (d) a slope.

Category	Variant	Modality	Fusion Strategy	Difficulty $\uparrow$	$R_{\mathrm{succ}}(\%,\uparrow)$
Full Method	GeoLoco (Ours)	RGB	Cross-Attention	0.87	86.4
(A) Representation	Scratch CNN (End-to-End)	RGB	Cross-Attention	0.58	60.4
	Semantic VFM (DINOv2)	RGB	Cross-Attention	0.61	66.1
	CLS Tokens	RGB	Cross-Attention	0.55	58.7
	Single-Layer Patch Token	RGB	Cross-Attention	0.75	74.8
(B) Modality Fusion	Naive MLP Concatenation	RGB	Concatenation	0.76	71.2

TABLE III: Quantitative ablation results in simulation. We report the maximum Mean Difficulty Level reached through curriculum learning and the corresponding Success Rate

R_{\mathrm{succ}}\%

on the final curriculum stage. All results are averaged over 5 random seeds.

Method	Stairs ( $0.23\,\mathrm{m}$ riser)		Gap ( $0.25\,\mathrm{m}$ width)
Method	$R_{\mathrm{succ}}(\%,\uparrow)$	Time ( $\downarrow$ )	$R_{\mathrm{succ}}(\%,\uparrow)$	Time ( $\downarrow$ )
Proprioceptive (Blind)	$30\%$	$10.1$	$0\%$	$6.5$
CNN-based	$40\%$	$8.2$	$40\%$	$5.8$
GeoLoco (Ours)	80%	5.4	70%	4.2

TABLE IV: Quantitative Real-World Performance on Unitree G1. Each method was evaluated

10

trials per terrain scenario. Best result in bold.

Humanoid Robot Locomotion

Humanoid robot locomotion studies how bipedal robots walk, run, recover from disturbances, and traverse varied terrain.

See all 145 papers

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies
Jul 6, 2026Yunchao Zhang, Yijia Weng, Ruizhe Liu +3Latent Action Learning Geometric Representation Learning
Department of Computer Science, The University of Hong Kong, Hong Kong, China · Department of Computer Science, Stanford University, Stanford, California, USA · Department of Data Science and AI, Monash University, Melbourne, Australia
PRIOR: Perceptive Learning for Humanoid Locomotion with Reference Gait Priors
Mar 19, 2026Chenxi Han, Shilu He, Yi Cheng +2Humanoid Robot Locomotion Robot Locomotion
Tsinghua University · ZERITH Robotics · Shanghai University
CReF: Cross-modal and Recurrent Fusion for Depth-conditioned Humanoid Locomotion
Mar 31, 2026Yuan Hao, Ruiqi Yu, Shixin Luo +3Humanoid Robot Locomotion Robot Locomotion
The authors are with Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China. · Guoteng Zhang is with the School of Control Science and Engineering, Shandong University, Jinan 250061, China.

Jul 6, 2026J/K move · Enter open · S save

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies

Yunchao Zhang, Yijia Weng, Ruizhe Liu, Ming Hu+2

Department of Computer Science, The University of Hong Kong, Hong Kong, China · Department of Computer Science, Stanford University, Stanford, California, USA · Department of Data Science and AI, Monash University, Melbourne, Australia

Latent Action Learning Geometric Representation Learning Robotic Manipulation Dynamic Robotic Manipulation Motion Representation Learning

Open paper AskPDF arXiv

Learning motion latents for robotic manipulation heavily relies on extracting motion patterns from visual sequences, yet effective action abstractions require understanding three-dimensional geometric transformations. Here, we introduce GeoMoLa (Geometry-Aware Motion Latents), which learns discrete motion latent codes by predicting how point clouds evolve during manipulation rather than reconstructing visual observations. This four-dimensional objective -- spatial geometry changing through time -- forces latent representations to encode actual physical motion rather than appearance patterns. GeoMoLa achieves state-of-the-art performance using only single-view RGB-D input, while existing methods require multi-view reconstruction, succeeding across diverse manipulation benchmarks. Our ablations reveal that geometric prediction is the key to driving performance, quantitatively validating that manipulation depends on spatial understanding. Furthermore, the learned codes exhibit effective motion abstraction: applying them to novel scenes produces physically consistent transformations regardless of visual context. Our real-world experiments also confirm this robustness capability, achieving robust manipulation with minimal demonstrations in cluttered environments where geometric reasoning determines success. Thus, we demonstrate that effective motion latents for robot control can better emerge from understanding motion through its three-dimensional effects rather than pixel-level patterns.

Abstract

Humanoid Robot Locomotion

Explore similar work

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies

PRIOR: Perceptive Learning for Humanoid Locomotion with Reference Gait Priors

CReF: Cross-modal and Recurrent Fusion for Depth-conditioned Humanoid Locomotion