PointWalker: Mapless Quadruped Locomotion Directly From Raw LiDAR Point Clouds

Sicen Li; Zhen Chu; Chao Li; Jun Zheng; Qiuguo Zhu; Jun Wu

cs.ROOct 11, 2026

PointWalker: Mapless Quadruped Locomotion Directly From Raw LiDAR Point Clouds

Authors: Sicen Li, Zhen Chu, Chao Li, Jun Zheng, Qiuguo Zhu, Jun Wu

Organizations: College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China · Yunshenchu Technology Co., Ltd., Hangzhou, China · Hangzhou Public Library, Hangzhou, China · Zhejiang Key Laboratory of Additive Manufacturing Technology and Equipment, Hangzhou, China

Abstract

Quadruped robots deployed for inspection and field missions must cross heterogeneous terrain, from stairs, slopes, and gaps to high platforms, stepping stones, and balance beams, where safe support is sparse or see-through. Existing methods interpose a terrain map or a depth image between sensing and control. Maps are costly to maintain, corrupted by odometry drift and outliers, and misled by see-through gratings, while depth cameras degrade with lighting and surface material. In this paper, we present PointWalker, a mapless framework whose single reinforcement learning policy directly consumes raw point clouds from four light detection and ranging (LiDAR) sensors. Voxelized points are encoded by linear self-attention and fused with proprioception through cross-attention, so terrain geometry reaches the policy without maps or foothold estimation. One proximal policy optimization (PPO) run, with terrain-aware rewards and dual-channel simulation of see-through gratings, yields one policy for all seven terrain classes with a 90.1% mean success rate in simulation. Deployed zero-shot on a Deep Robotics X30 at 50 Hz onboard, it crosses 0.8-m gaps, climbs 0.8-m platforms and 45° steel-grating stairs, carries a 50-kg payload, and completes a 2.5-km mountain route.

Figures & tables

Fig. 1: Real-world experiments with the X30 quadruped robot, all driven by one policy. (a) Stepping stones, (b) an 80-cm gap, (c) an 80-cm-high platform, and (d) a balance beam. (e) Outdoor mountain slope and (f) outdoor stairs on the 2.5-km route (25-kg payload; Fig. 15 ), (g) industrial steel-grating stairs, and (h) stair ascent with a 50-kg payload.

Fig. 2: PointWalker framework. (a) Environment: the policy receives proprioceptive history, commands, and four-LiDAR point clouds; privileged information (true states, dynamics) feeds the value network and the terrain-aware rewards in training only. (b) Actor network: linear self-attention point encoding, cross-attention fusion, a Gaussian action head with two auxiliary heads, and PPO training with an asymmetric critic. (c) Deployment: ONNX export and zero-shot 50-Hz inference on the onboard RK3588.

Reward term	Equation	Weight
Lin. vel. tracking	$\exp(-4\\|\boldsymbol{v}^{xy,\mathrm{cmd}}-\boldsymbol{v}^{xy}\\|^{2})$	$+1.1$
Ang. vel. tracking	$\exp(-4(\omega_{z}^{\mathrm{cmd}}-\omega_{z})^{2})$	$+0.6$
Lin. velocity (z)	$v_{z}^{2}$	$-2.0$
Ang. velocity (xy)	$\\|\boldsymbol{\omega}^{xy}\\|^{2}$	$-0.06$
Body height	$(h^{\mathrm{des}}-h^{\mathrm{b}})^{2}$	$-5.0$
Terrain alignment	$1-\exp(-\\|\boldsymbol{g}-\boldsymbol{n}_{t}\\|^{2})$	$-1.0$

TABLE I: Reward Terms and Weights

Terrain	Difficulty parameter	Curriculum range
Stairs	Step height (tread $0.28$ m)	$0.1\to 0.28$ m
Slopes	Slope ( $\tan\theta$ )	$0.2\to 0.65$
Discrete obstacles	Obstacle height	$0.1\to 0.23$ m
Gaps	Gap width	$0.15\to 1.1$ m
High platforms	Platform height	$0.25\to 1.2$ m
Stepping stones	Stone spacing	$0.1\to 0.55$ m

TABLE II: Difficulty Curriculum of the Seven Training Terrains

Parameter	Value
Training configuration
Parallel environments	8192
GPUs / training time	$4\times$ RTX 5090 / $\sim$ 50 h
Optimizer / learning rate	Adam / $3\times 10^{-4}$
Discount $\gamma$ / GAE $\lambda$	0.99 / 0.95
Clip $\epsilon$ / grad. clip / epochs	0.2 / 1.0 / 5

TABLE III: Training and Domain Randomization Configuration

Representation	Params (M)	FLOPs (M)	Latency (ms)
PointWalker, linear attn., 0.1-m voxels	1.47	7.35	2.01 $\,\pm\,$ 0.85
w/ Standard attention	1.47	22.67	8.23 $\,\pm\,$ 3.68
w/ 0.05-m voxels	1.47	19.52	7.33 $\,\pm\,$ 2.40
w/ 0.2-m voxels	1.47	4.36	1.03 $\,\pm\,$ 0.45
Depth-image encoding	2.16	11.82	4.50 $\,\pm\,$ 2.25
Height-map encoding	1.26	2.90	1.18 $\,\pm\,$ 0.43

TABLE IV: Compute Cost of the Perception Representations

Fig. 3: Snapshots of the seven training terrains in simulation (Table II ); points colored by cross-attention weight (Section III-B ): red = high, blue = low.

Method	Stairs	Slopes	Obst.	Gaps	Plat.	Stones	Beams	Mean
PointWalker (full)	93.2	96.3	93.2	86.1	85.3	83.8	92.7	90.1
Depth-image enc.	94.6	94.9	91.7	71.5	82.9	68.4	89.3	84.8
Height-map enc.	86.7	97.2	93.3	82.4	87.1	75.8	89.1	87.4
Blind	82.6	84.1	90.3	18.5	15.2	6.4	11.7	44.1
w/o Cross-attn.	82.4	92.1	89.6	74.8	75.2	68.9	77.5	80.1
w/ Standard attn.	94.3	95.7	94.0	87.2	84.8	84.7	92.1	90.4

TABLE V: Simulation Success Rates (%)

Fig. 4: Edge-landing ratio (invalid fraction of the privileged ring scan, Section IV-A ) on the three foothold-restricted terrains; error bars: standard deviation across five seeds.

Fig. 5: Gap crossing with the overspeed tolerance (simulation). Top: keyframes a–e in time order; middle and bottom: base-frame linear and angular velocities, dashed lines marking the keyframe instants.

Fig. 6: Climbing onto a high platform with the jump incentive (simulation). Top: keyframes a–f in time order; middle and bottom: base-frame velocities with the keyframe instants marked.

Fig. 7: Ascent of the steel-grating stairs (top) and the simultaneous point-cloud attention (bottom; rows time-aligned, red = high, blue = low): attention concentrates on the support under the stance feet and the tread about to be stepped on; the sparse points preserve the step structure with no map.

Fig. 8: Balance-beam crossing with the simultaneous point-cloud attention (rows time-aligned; red = high, blue = low): attention concentrates on the beam under the feet and extends to the next segment to be stepped on; the sparse points preserve the narrow strip with no map.

Fig. 9: Failure of the w/o Dual-channel terrain variant (Section IV-A ) on the same stairs. Left: a front leg reaches the first tread but the robot cannot mount; center: stalled at the stair base, torso pitched up; right: top view of the see-through treads (PointWalker: Fig. 7 ).

Fig. 10: Foot-edge penalty, same-site contrast on stepping stones. (a) PointWalker (w/ Foot-edge penalty): feet land inside the 35-cm stone tops. (b) w/o Foot-edge penalty: a front foot slips off a stone edge (red circle) and the robot falls.

Fig. 11: Overspeed tolerance, same-site contrast at the same 0.8-m gap (multi-exposure composites). (a) PointWalker (w/ Overspeed tolerance): accelerates before the near edge and lands on the far side. (b) w/o Overspeed tolerance: momentum falls short; the nose strikes the far edge and the robot hangs.

Fig. 12: Jump incentive, same-site contrast (multi-exposure composites; descending from a single-tier platform and climbing the 0.8-m platform). (a) PointWalker (w/ Jump incentive): hooks the top edge and pulls itself onto the top. (b) w/o Jump incentive: claws at the top edge and stalls (red circle).

Fig. 13: Payload tests with rigid backpack loads (multi-exposure composites): left, 30 kg (55% of body weight) onto the 0.8-m platform; right, 50 kg (91%) up the grating stairs.

Fig. 14: LiDAR occlusion test. Top-left and bottom-left: one front and one rear LiDAR blocked by adhesive covers (two in total). Right: multi-exposure traverse of stairs (20-cm steps), platform, and 20 ∘ slope under occlusion.

Fig. 15: Long-range outdoor and natural-terrain tests. Left: rocky section of the 2.5-km route (200-m gain, 25-kg payload); right: gravel, sand, and cobblestone, none of the seven training terrains.

Explore similar work

UniPoint: Unified Point-Level Sensor Fusion for Humanoid Locomotion Across Challenging Terrains
Sep 20, 2026Sicen Li, Zhen Chu, Chao Li +2Humanoid Robot Locomotion Point Cloud Learning
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China · Yunshenchu Technology Co., Ltd., Hangzhou, China · Zhejiang Key Laboratory of Additive Manufacturing Technology and Equipment, Hangzhou, China
JEPLO: Joint-Embedding Predictive Learning for LiDAR-Based Legged Locomotion
Sep 14, 2026Qihao Yuan, Yixuan Qiu, Ziyu Cao +2Joint-Embedding Predictive Architecture Terrain-Aware Robot Locomotion
Faculty of Science and Engineering, University of Groningen, The Netherlands · Department of Electrical Engineering, Linköping University, Sweden
LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors
Jun 14, 2026Loukas Kordos, Leonard T. Franz, Simon Rappenecker +4Legged Robotics RL for Robotics
Technical University of Munich. · University of Tübingen. · Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience. +1

Sep 20, 2026J/K move · Enter open · S save

UniPoint: Unified Point-Level Sensor Fusion for Humanoid Locomotion Across Challenging Terrains

Sicen Li, Zhen Chu, Chao Li, Qiuguo Zhu+1

College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China · Yunshenchu Technology Co., Ltd., Hangzhou, China · Zhejiang Key Laboratory of Additive Manufacturing Technology and Equipment, Hangzhou, China

Humanoid Robot Locomotion Point Cloud Learning Terrain-Aware Robot Locomotion Multimodal Sensor Fusion

Open paper AskPDF arXiv

Open-world deployment requires humanoid robots to cross highly heterogeneous terrain safely, with perception that simultaneously provides wide coverage, local accuracy, and redundancy against sensor failure. Existing approaches struggle to satisfy all three: one forward depth camera or nearby height sampling covers too little; odometry-corrected elevation maps drift under aggressive motion and miss thin vertical structures; image-level encoding costs grow with camera count. We present UniPoint, a humanoid whole-body locomotion framework built on multi-source point-level sensor fusion. Measurements from a 360° light detection and ranging (LiDAR) sensor and two depth cameras are early-fused into one base-frame point set. Voxelization resamples it to a fixed number of tokens encoded by linear self-attention and proprioception-queried cross-attention, decoupling forward cost from sensor count. The point set retains standing thin barriers; a single-modality failure removes only part of the tokens, so the policy degrades gracefully. A single training run with terrain-aware rewards, perception-degradation injection, and domain randomization produces one policy for all eight terrain types, deployed on an onboard RK3588 without fine-tuning. On a DR02 humanoid, 20 trials at each of nine real-world settings over seven terrain types validate the policy on 70-cm-high platforms, 100-cm gaps, thin barriers, and sparse or narrow footholds; it also generalizes zero-shot outdoors.

Abstract

Explore similar work

UniPoint: Unified Point-Level Sensor Fusion for Humanoid Locomotion Across Challenging Terrains

JEPLO: Joint-Embedding Predictive Learning for LiDAR-Based Legged Locomotion

LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors