Organizations: School of Computation, Information and Technology, Technical University of Munich, Germany. · Beijing Information Science and Technology University, China.
Real-world robot deployment requires adapting learned policies to unseen obstacles without collecting new demonstrations or retraining. Existing policy-level guidance often applies obstacle corrections independently at individual horizon steps, which may address local obstacle violations but fail to coordinate the action sequence into a complete, task-directed detour. How obstacle responses should be propagated across the action horizon to coordinate complete, task-directed avoidance motion remains underexplored. We introduce an inferencetime obstacle-guidance framework for pretrained relative-action diffusion policies. It rolls out cumulative relative actions into a workspace trajectory, evaluates obstacle interactions along the trajectory with an SDF-based obstacle model, and propagates each obstacle response to all contributing action increments during denoising. Real-robot experiments across seven obstacleconstrained scenarios spanning four manipulation tasks show that our method achieves 105/140 (75.0%) collision-free task successes, compared with 8/140 (5.7%) for absolute-action pointwise guidance. Ablations show that the relative-action horizoncoupled formulation provides the larger observed improvement across both obstacle models.
Figures & tables
Fig. 1: Overview of the proposed framework. Boxes denote processing stages, and labeled arrows indicate the variables passed between them. The upper path shows guided DDIM sampling, and the lower path shows horizon-coupled obstacle guidance. Details are provided in Sec. III-A .
Fig. 2: Real-robot experimental system and workspace.
Fig. 3: Our four real-world manipulation tasks span obstacle-constrained pick-and-place, pouring with end-effector roll, pen insertion with pitch adjustment, and sequential sock placement and drawer closing. Superimposed robot poses visualize each execution sequence.
Method
(a) Obstacle-Configuration Evaluation
(b) Cross-Task Evaluation
All 7
Rect.
2 Obs.
3 Obs.
4 Obs.
Overall
Pouring
Pen Insertion
Sock + Drawer
Overall
Abs.-NoG
2/20
0/20
0/20
0/20
2/80
0/20
0/20
2/20
2/60
4/140
Rel.-NoG
0/20
0/20
0/20
0/20
0/80
0/20
0/20
0/20
0/60
0/140
Abs.-Guide
2/20
1/20
1/20
0/20
4/80
0/20
0/20
4/20
4/60
8/140
Ours
18/20
16/20
12/20
8/20
54/80
20/20
16/20
15/20
51/60
105/140
TABLE I: Collision-free task successes across (a) obstacle configurations and (b) manipulation tasks.
Method
Pick-and- Place
Pouring
Pen Insertion
Sock + Drawer
Abs.-NoG
20/20
20/20
13/20
20/20
Rel.-NoG
20/20
20/20
18/20
19/20
TABLE II: Task successes in obstacle-free reference settings.
Fig. 4: Executed end-effector trajectories of Abs.-Guide (top) and Ours (bottom) across rectangular-, two-, and three-obstacle layouts. Purple dashed contours denote the primitive envelopes utilized by Abs.-Guide . Blue and red trajectories indicate successful and failed trials, and crosses mark terminal positions.
Fig. 5: Ablation of obstacle modeling and guidance formulation. (a) Collision-free task successes for the four combinations. Executed end-effector trajectories under (b) primitive-envelope relative-action horizon-coupled guidance and (c) SDF absolute-action point-wise guidance. Purple dashed contours denote the primitive envelope.
Imitation learning has achieved impressive results in robotic manipulation, yet most existing approaches assume clean backgrounds and lack explicit mechanisms for obstacle-aware motion generation. Extending such policies to cluttered, real-world scenes with unstructured obstacles remains a key generalization challenge. We present ObstaDiff, a decomposed diffusion-policy framework with a lightweight obstacle-aware visual encoder. ObstaDiff extracts a structured target-obstacle-background representation, enabling the downstream alignment policy to generate end-effector trajectories toward a target-centered bottleneck pose while reasoning about surrounding obstacles. We evaluate ObstaDiff on 61 real-robot greenhouse trials per method (366 executions in total). ObstaDiff achieves 75.41% average task success and 8.20% average obstacle collision rate, outperforming representative imitation-learning baselines and improving generalization in cluttered agricultural scenes.
Jiawen Wang, Kevin Yao, Khalid Jawed
Department of Mechanical and Aerospace Engineering, University of California, Los Angeles · Department of Computer Science, University of California, Los Angeles
We propose NUDGE (Nudge Update via Differentiable GEometry), a training-free obstacle-avoidance procedure that can be incorporated in any robot policy based on diffusion or flow matching, including diffusion policies and vision-language-action models. Our work injects gradients from a signed distance field, a function returning each point's distance to the nearest obstacle, into the policy at inference time to steer it away from obstacles. It supports any common action parameterization, from absolute or relative joint poses to end-effector poses, through a differentiable joint-trajectory decoder. Experiments show that NUDGE preserves the policy's task distribution and runs reactively in real time.
Weihang Guo, Lydia E. Kavraki
Department of Computer Science, Rice University, Houston, TX, USA · Ken Kennedy Institute at Rice University
Robots operating in human-centered environments must safely navigate among multiple dynamic obstacles to avoid collisions with people and surrounding infrastructure. Control barrier functions (CBFs) provide an effective mechanism for safety filtering, and recent CBF-based reinforcement learning (RL) methods embed such safety information into learned policies. However, executing only safety-filtered actions during training can restrict policy exploration, a limitation that becomes particularly consequential in dynamic scenes where safety depends on relative robot-obstacle motion. We propose DODGER, a safety-guided RL framework that directly executes policy-generated actions to drive training rollouts while using CBF-filtered references and constraint violations to shape the policy toward collision-avoidance behavior. We evaluate DODGER through a Dubins-car safety analysis and demonstrate goal-directed navigation among multiple dynamic obstacles in full-order humanoid simulation and real-world humanoid experiments using LiDAR-based perception, without a runtime safety filter.
Sanghyuk Park, Kwanwoo Lee, Taekyung Kim +2
Department of Intelligence and Information, Seoul National University, Republic of Korea · Department of Robotics, University of Michigan, Ann Arbor, MI, USA · Department of Mechanical Engineering, Yonsei University, Republic of Korea +1