Positive Safety Margins from Artificial Potential Field Reward Augmentation for Size-Robust Obstacle Avoidance under Partial Observability
Organizations: The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong SAR, China
Abstract
Collision-free reaching is central to manipulation, yet real cells often leave the size and exact position of an obstacle unobserved, which excludes reinforcement learning controllers that take obstacle geometry as input. A reward limited to a collision penalty does not depend on the clearance outside the obstacle, so under a step cost its learned policy skims the obstacle surface and fails once the obstacle radius exceeds its training value. We investigate whether a single analytic reward term can induce a safety margin without exposing the obstacle to the policy. We augment the reward of a simulated Franka Emika Panda reaching task with a horizontal artificial potential field that penalizes proximity within a safety margin around each of two pillars (fixed nominal layout, random drift). Under hidden observation, policies trained with this term maintain a safe success rate above 88% across a radius sweep from 0.03 m to 0.055 m (30 seeds), while policies without it fall below 10% at the maximum tested radius; the term induces a minimum clearance of 47.9 mm against 9.6 mm without it and eliminates training collisions at least eight times earlier (median steps). The advantage is statistically significant at all out-of-distribution radii and holds across three reinforcement learning solvers and under full observation. Its safe success rate is comparable to that of a classical potential-field planner given the pillar positions. The margin is a fixed buffer rather than an adaptation to the radius, and it lengthens the path by about 19%.
Figures & tables
| Term | Meaning | Default value |
|---|---|---|
| dense distance to goal | (native) | |
| success bonus | ||
| goal-progress reward | ||
| action-magnitude smoothness | ||
| horizontal APF potential | ||
| over-the-top height penalty |
| Configuration | Removed terms |
|---|---|
| ours | none |
| no_apf | |
| no_progress | |
| vanilla | all auxiliary terms |
| no_xy_apf |
| Group | Obstacle radius (m) | |||||
|---|---|---|---|---|---|---|
| 0.030 | 0.035 | 0.040 | 0.045 | 0.050 | 0.055 | |
| ours | 94.3 | 94.3 | 94.3 | 94.3 | 93.5 | 89.4 |
| no_apf | 92.5 | 69.3 | 32.9 | 22.8 | 17.2 | 9.5 |
| no_progress | 93.7 | 93.7 | 93.7 | 93.5 | 91.8 | 88.3 |
| vanilla | 95.3 | 68.1 | 30.2 | 22.7 | 17.2 | 9.7 |
| no_xy_apf | 83.6 | 64.6 | 39.8 | 27.7 | 16.6 | 7.5 |
| Group | Clearance (mm) | Center distance (mm) |
|---|---|---|
| ours | ||
| no_apf | ||
| no_progress | ||
| vanilla | ||
| no_xy_apf |
| Reward | Penalty | Clearance (mm) | Safe success at (m) | Seeds | |||
|---|---|---|---|---|---|---|---|
| 0.030 | 0.040 | 0.055 | at 0.055 | ||||
| no_apf | 92.5 | 32.9 | 9.5 | ||||
| no_apf | 94.8 | 24.4 | 8.8 | ||||
| no_apf | 90.2 | 55.6 | 18.0 | ||||
| ours | 94.3 | 94.3 | 89.4 | ||||
| no_apf | ours | ||||
|---|---|---|---|---|---|
| (m) | Mean (%) | 95% CI | Mean (%) | 95% CI | |
| 0.030 | 92.5 | 94.3 | (n.s.) | ||
| 0.035 | 69.3 | 94.3 | |||
| 0.040 | 32.9 | 94.3 | |||
| 0.045 | 22.8 | 94.3 | |||
| 0.050 | 17.2 | 93.5 | |||
| Obstacle radius (m) | ||||||
|---|---|---|---|---|---|---|
| Method | 0.030 | 0.035 | 0.040 | 0.045 | 0.050 | 0.055 |
| Classical APF (sees pillars) | 90.0 | 90.0 | 90.0 | 90.0 | 90.0 | 90.0 |
| RL ours | 94.3 | 94.3 | 94.3 | 94.3 | 93.5 | 89.4 |
| RL no_apf | 92.5 | 69.3 | 32.9 | 22.8 | 17.2 | 9.5 |
| Solver | Group | Obstacle radius (m) | |||||
|---|---|---|---|---|---|---|---|
| 0.030 | 0.035 | 0.040 | 0.045 | 0.050 | 0.055 | ||
| SAC | ours | 85.5 | 85.4 | 85.1 | 85.1 | 85.1 | 85.1 |
| SAC | no_apf | 99.2 | 70.5 | 18.1 | 10.4 | 7.5 | 2.9 |
| TD3 | ours | 88.7 | 88.7 | 88.7 | 88.7 | 88.2 | 85.5 |
| TD3 | no_apf | 100.0 | 68.2 | 18.3 | 14.4 | 12.3 | 5.1 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Item | Setting |
|---|---|
| Environment | PandaReach-v3 (dense reward), steps per episode |
| Action | , scaled to a maximum step of |
| Success threshold | ; discount factor |
| Pillars | , height , nominal centers and ; random walk along with a step drawn uniformly from , reflected at from the nominal center; reset to the nominal centers at every episode |
| Training radius | |
| Goal sampling | , , , rejected if within of a nominal pillar center |
| Clearance | Path | Safe success (%) | Collision (%) | Seeds | |||
|---|---|---|---|---|---|---|---|
| (mm) | (mm) | (m) | |||||
| (default) | |||||||
Robot Collision Avoidance
Robot collision avoidance develops sensing, planning, and control methods that keep robots safely separated from people, obstacles, and other robots.
See all 185 papers