Distance Field Driven Reinforcement Learning for Reactive and Robust Collision Avoidance

Taiga Ikeshima; Sheila Sutjipto; Lan Wu; Francisco J. Naranjo-Campos; Taiki Ishigaki; Taisuke Kobayashi; Teresa A. Vidal-Calleja; Eiichi Yoshida

cs.ROOct 21, 2026

Distance Field Driven Reinforcement Learning for Reactive and Robust Collision Avoidance

Authors: Taiga Ikeshima, Sheila Sutjipto, Lan Wu, Francisco J. Naranjo-Campos, Taiki Ishigaki, Taisuke Kobayashi, Teresa A. Vidal-Calleja, Eiichi Yoshida

Organizations: Tokyo University of Science, Japan. · the University of Technology Sydney, Australia. · Universidad Carlos III de Madrid, Spain. · the National Institute of Informatics, Japan.

Abstract

Robust and reactive collision avoidance remains a central challenge for deploying Reinforcement Learning (RL) policies on real robotic manipulators, particularly when transferring behaviours learned in simulation to the physical world. This paper presents an RL framework that integrates the Euclidean Distance Field (EDF) directly into the observation space, providing the agent with continuous geometric cues that support stable and responsive avoidance behaviours. By leveraging the rich spatial information encoded in the EDF, the method enables a streamlined reward formulation that achieves effective obstacle avoidance both with and without Artificial Potential Field guidance. The proposed approach is validated on a 7-DoF redundant manipulator, demonstrating zero-shot transfer from simulation to real hardware. Despite being trained exclusively with randomised static objects, the deployed policy generalizes to previously unseen real environments and exhibits robust online avoidance of dynamic obstacles. These results highlight the potential of EDF-based representations to mitigate the sim-to-real gap and support reliable and real-time collision-aware behaviour in complex and changing environments.

Figures & tables

Fig. 1 : Real-time collision avoidance using RL with a Euclidean Distance and Gradient Field, learnt in simulation and transferred to real-robot experiments.

Panel of Figure 2 — Fig. 2 : Cross-sectional $xy$ -plane slice of the Euclidean Distance Field (EDF). (a) The EDF in an unobstructed environment. (b) The same environment with a single obstacle; the resulting change in the distance field is highlighted by the red points within the yellow dashed outline.

Fig. 3 : Overview of the EDF-based RL pipeline.

Symbol	Meaning	Value
$\mu$	Direction-alignment term weight	$1.01$
$\lambda_{c}$	Goal-distance penalty (collision)	$2.0$
$\lambda_{nc}$	Goal-distance penalty (no collision)	$1.0$
$w$	Weight on repulsive term in APF	$0.2$
$\gamma$	Orthogonal escape term weight	$0.2$

TABLE I : Reward and APF guidance hyperparameters

Category	Parameter	Range [min, max]
	$x$ [m]	$[0.45,\ 1.00]$
Obstacles	$y$ [m]	$[-0.50,\ 0.50]$
	$z$ [m]	$[0.20,\ 0.80]$
	$x$ [m]	$[0.50,\ 0.80]$
Goal	$y$ [m]	$[-0.30,\ 0.30]$
	$z$ [m]	$[0.30,\ 0.50]$

TABLE II : Sampling ranges used for obstacle and goal generation in simulation

Panel of Figure 7 — Fig. 5 : EDF sample conditions. (a) EE-setting. (b) FB-setting.

Condition	Environment-related observation	APF
Baseline RL	Obstacle centre coordinates (w.r.t. world frame)	$\times$
RL-APF	Obstacle centre coordinates (w.r.t. EE)	$\circ$
EDF-RL-EE	EDF at EE	$\times$
EDF-RL-APF-EE	EDF at EE	$\circ$
EDF-RL-FB	EDF at EE + $N{=}4$ body points	$\times$
EDF-RL-APF-FB	EDF at EE + $N{=}4$ body points	$\circ$

TABLE III : Summary of the six simulation configurations

Condition	Reaching goal	Reaching goal (%)	Average number	Reaching time (s)	Time to learning
Condition	w/o collision (%)	Reaching goal (%)	of collisions	(per episode)	convergence (h)
Baseline RL	$35.0$	$51.0$	$27.9$	$14.5$	$2.23$
RL-APF	$\bm{64.0}$	$\bm{86.0}$	$\bm{4.10}$	$\bm{9.38}$	$\bm{1.92}$
EDF-RL-EE	$62.0$	$83.0$	$6.20$	$10.42$	$1.84$
EDF-RL-APF-EE	$\bm{75.0}$	$\bm{96.0}$	$\bm{6.79}$	$\bm{6.45}$	$\bm{4.09}$
EDF-RL-FB	$56.0$	$78.0$	$9.82$	$17.22$	$2.50$

TABLE IV: Simulation results over 100 tests

Panel of Figure 10 — Fig. 6 : Real-world experimental setups: (a) two obstacles in predefined positions; (b) a single dynamic obstacle introduced during execution.

Panel of Figure 11 — Fig. 7 : Time sequence (left to right) of the EDF-RL-APF-FB policy on the real robot, successfully avoiding two obstacles under the fixed-position condition, with corresponding rows (top: IDMP, middle: simulation for visual reference, bottom: real robot).

Condition	Reaching goal w/o collision (%)
EDF-RL-EE	60
EDF-RL-APF-EE	40
EDF-RL-FB	70
EDF-RL-APF-FB	70

TABLE V : Real-world results for the fixed-obstacle condition over 10 tests

Panel of Figure 13 — Fig. 8 : Trajectories of the EDF-RL-APF-FB policy under four conditions: no obstacle, fixed obstacles, lateral dynamic interference, and vertical dynamic interference.

Robot Collision Avoidance

Robot collision avoidance develops sensing, planning, and control methods that keep robots safely separated from people, obstacles, and other robots.

See all 162 papers

E2DRL: End-to-End Differentiable Reinforcement Learning for 3D Obstacle Avoidance in Quadruped Robots
Oct 14, 2026Cheng Fan, Bingyang Zhou, Gang Yang +7Reinforcement Learning Robot Collision Avoidance
Department of Electrical and Computer Engineering, National University of Singapore, Singapore.
CLASH: Collision Learning via Augmented Sim-to-real Hybridization to Bridge the Reality Gap
Feb 21, 2026Haotian He, Ning Guo, Siqi Shi +2Sim-to-Real Transfer Robotics Simulation
School of Mathematical Sciences, Peking University. · School of Artificial Intelligence, Shanghai Jiao Tong University. · School of Mathematical Sciences, Beijing Normal University.
Zero-Shot Reactive Obstacle Avoidance for Generative Robot Policies
Sep 28, 2026Weihang Guo, Lydia E. KavrakiCollision Avoidance Robot Collision Avoidance
Department of Computer Science, Rice University, Houston, TX, USA · Ken Kennedy Institute at Rice University

Oct 14, 2026J/K move · Enter open · S save

E2DRL: End-to-End Differentiable Reinforcement Learning for 3D Obstacle Avoidance in Quadruped Robots

Cheng Fan, Bingyang Zhou, Gang Yang, Zixuan Zhuang+6

Department of Electrical and Computer Engineering, National University of Singapore, Singapore.

Reinforcement Learning Robot Collision Avoidance RL for Legged Locomotion Visuomotor Policy Learning Dynamic Obstacle Avoidance

Open paper AskPDF arXiv

Differentiable reinforcement learning shows strong promise for robot policy learning, but its application to legged locomotion has remained limited to blind walking or simulated 2D tasks. Extending it to end-to-end visual obstacle avoidance is challenging because hard collision constraints and early termination upon contact corrupt first-order gradients, trapping optimizers in local minima. We present E2DRL, which addresses this with a soft differentiable obstacle-interaction formulation: penetration penalties across multiple body keypoints with velocity-conditioned stop-gradient shaping to provide the first-order training signals for 3D obstacle avoidance. To realize a deployable perception-control system, a privileged teacher trained via Soft Analytic Policy Optimization (SAPO) in a differentiable simulator is distilled into a recurrent depth-based student via DAgger, then blended with a base locomotion policy using a terrain-aware gating module for deployment. In simulation, E2DRL achieves higher sample efficiency than PPO and demonstrates superior velocity-tracking performance and 3D obstacle-avoidance capability compared with both PPO and Duality-based DCBF NMPC. We present the first sim-to-real validation of end-to-end differentiable RL for 3D obstacle avoidance on a Unitree Go2 in static and dynamic environments.

Abstract

Robot Collision Avoidance

Explore similar work

E2DRL: End-to-End Differentiable Reinforcement Learning for 3D Obstacle Avoidance in Quadruped Robots

CLASH: Collision Learning via Augmented Sim-to-real Hybridization to Bridge the Reality Gap

Zero-Shot Reactive Obstacle Avoidance for Generative Robot Policies