Humanoid robots operating in human-centric environments must coordinate the entire body to traverse tight, cluttered spaces while maintaining balance. Existing methods often struggle to jointly consider dynamic stability and collision avoidance from high-dimensional perceptual inputs. Moreover, for humanoids, safe behavior is inherently multimodal: the same obstacle may admit multiple valid traversals, which is difficult to represent with a single deterministic policy. To address these challenges, we propose SWAP (Steerable Whole-body Avoidance Policy), a hierarchical framework that separates collision-aware decision making from stabilization. A high-level, perception-conditioned diffusion policy generates multimodal, collision-aware motion proposals, while a low-level whole-body RL controller tracks these proposals and maintains dynamic balance. To efficiently train the high-level policy, we obtain scalable supervision via hindsight labeling: we roll out the low-level policy under randomized commands, place obstacles only in the space that each executed motion did not sweep, so the motion is collision-free by construction, and re-render the perception the robot would have seen. At deployment, Monte Carlo sampling with simple cost signals steers the diffusion model toward safe, task-aligned behaviors without heavy reward tuning. Experiments in simulation and on a real Unitree G1 humanoid show reliable avoidance with static and dynamic obstacles. Test-time steering further enables customization, improving targeted rewards (e.g., speed, orientation, or posture) by 34% on average while maintaining a high collision avoidance rate. Together, these results support a practical solution to humanoid whole-body collision avoidance with test-time steerability. Project website: https://iscoyizj.github.io/SWAP-steerable-humanoid-oa-website/
Figures & tables
Figure 2: Overview of the SWAP framework. The high-level diffusion policy generates collision-aware whole-body trajectories from perception observations, which are tracked by a low-level RL locomotion controller.
Figure 3: Hindsight labeling pipeline with kinematic replay. Randomized rollouts generate diverse motions; the swept volume of the replayed trajectory is used to label a conservative obstacle representation, and perception is re-rendered along the replay for training the diffusion policy.
Figure 4: Perception encoder architecture . Point clouds are encoded via PointNet++ and a GRU into a perception embedding zt . To ensure spatial awareness, zt is supervised via reconstruction of point clouds and obstacle configuration.
Figure 5: Simulation results and real-world settings for five obstacle avoidance tasks.
Figure 6: Ablation study on hindsight labeling. (a) Trajectory count and data augmentation, without test-time steering: more base trajectories and higher augmentation multipliers yield higher success rates. (b) Positive/negative augmentation ratio, with test-time steering: a 10% ratio achieves optimal performance; higher ratios (50%) introduce too many negative samples, degrading performance at high augmentation levels. Error bars indicate standard deviation over 5 seeds.
Figure 7: Training obstacles (orange) and held-out test obstacles (blue) for representative wall configurations. The closest training configuration overlaps 84–90% with the test scene, but the exact wall positions, yaw angles, and gap sizes are never seen during training.
Figure 8: Steering improvement across six objectives. Radar charts compare baseline (no steering) versus different steering methods for (a) static and (b) dynamic obstacles.
Figure 9: Comparison of three steering configurations: (a) holding both arms, (b) nominal without steering, and (c) turning left. Each panel shows the values of both objectives over time.
Humanoid locomotion in highly confined environments requires navigating dense environmental obstacles and complex self-collision bounds while maintaining multi-contact dynamic feasibility. Traditional trajectory optimizers frequently struggle in these restricted spaces, as navigating the large collision space with splines on particle abstractions is insufficient and leads to poor local minima. To address this, we propose a three-stage whole-body planning framework that formulates kinematic path planning directly over kinematically reachable rigid-body volumes. By integrating differentiable collision avoidance into a reachability-constrained formulation, our framework synthesizes volume-informed guides that reliably guide a full-order trajectory optimizer over long horizons. We show that these optimized plans serve as high-quality references to train a residual reinforcement learning policy for robust online execution. We validate our approach on the Unitree G1 humanoid across three benchmark testbeds exceeding NIST emergency response standards, achieving restricted confinement ratios (Cr<1.5). Our framework generates feasible trajectories across 12-to-18-second tasks with complex foot and hand contacts where standard baselines fail, while the learned policy successfully tracks these plans under extensive domain randomization in physics simulation.
Carlos Gonzalez, Luis Sentis
Department of Aerospace Engineering and Engineering Mechanics, The University of Texas at Austin, TX 78712, USA.
Humanoid robots often execute motion commands through whole-body controllers (WBCs) that track targets while maintaining balance and stability. However, most WBCs are blind to scene geometry, which can lead to collisions from imperfect target motions that are geometrically unsafe due to perception, planning, or teleoperation errors. We propose RECAL, a Robot--Environment Cross-Attention Layer that wraps a blind WBC to trade off target tracking against collision avoidance using external scene geometry. RECAL supports collision-aware tracking of floating-base and end-effector commands, including collision avoidance for held objects. It represents the robot, held objects, and environment as point clouds, using cross-attention between robot/object points and the environment to produce geometry-aware control features. In simulation, RECAL improves collision avoidance while preserving target-tracking performance across frozen-arm and adaptive-arm locomotion, object-carrying, and standing-manipulation scenarios relative to alternative geometry-aware WBC architectures. We further demonstrate the controller on a real Digit V3 humanoid robot.
Mohitvishnu S. Gadde, Ashish Malik, Pranay Dugar +3
Collaborative Robotics and Intelligent Systems Institute (CoRIS), Oregon State University, Corvallis, OR 97331
Collision-free motion is often aided by tactile and proximity sensors distributed on the body of the robot due to their resistance to occlusion as opposed to external cameras. However, how to shape the sensor's properties, such as sensing coverage; type; and range, to enable avoidant behavior remains unclear. In this work, we present a reinforcement learning framework for whole-body collision avoidance on a humanoid H1-2 robot and use it to characterize how sensor properties shape learned avoidance behavior. Using dodgeball as a benchmark task, we ablate the properties of sensors distributed across the upper body of the robot and find that raw proximity measurements can substitute for explicit object localization provided the sensing range is sufficient and that sparse non-directional proximity signals outpace dense directional alternatives in sample efficiency.
Carson Kohlbrenner, Niraj Pudasaini, William Xie +3
University of Colorado Boulder, 1111 Engineering Drive, Boulder, CO USA.