Non-Prehensile Multi-Object Placement with Mobile Robots via Diffusion-Based Trajectory Planning

Lance Pharand; Megnath Ramesh; Mark Chiu; Yash Vardhan Pant; Stephen L. Smith

cs.ROOct 8, 2026

Non-Prehensile Multi-Object Placement with Mobile Robots via Diffusion-Based Trajectory Planning

Authors: Lance Pharand, Megnath Ramesh, Mark Chiu, Yash Vardhan Pant, Stephen L. Smith

Organizations: Electrical and Computer Engineering Department at the University of Waterloo, ON, Canada

Abstract

Non-prehensile manipulation with mobile robots has primarily targeted placement in coarse goal regions. Precision placement has been studied with manipulators on benchmarks such as Push-T, where an omnidirectional end effector can readily re-approach an object from any side. For a mobile robot, correcting the same placement errors may require wide repositioning maneuvers around the object. We formalize multi-object pushing with a mobile robot as a multi-target assignment problem at tight tolerances. We propose a diffusion planner with a causal factorization: each robot denoising step is followed by an object step conditioned on the current clean robot estimate. This factorization allows us to sample multiple object outcomes for a single robot plan, capturing the multi-modality of pushing. We also design a set of pushing-specific costs for mobile robots, used for both guidance and best-of- $N$ sampling at inference. On Push-T, our planner attains the highest success rate of the methods we compare, while having the lowest rate of implausible object motion. Across three multi-box pushing environments resembling warehouse scenarios, it raises the fraction of boxes placed by $21.5$ percentage points over an unguided baseline, and replans $1.8\times$ faster than a planner that denoises the two trajectories together. Lastly, we deploy the guided planner on a TurtleBot 3 Burger with object shapes absent from the training data, where it places $85.0\%$ of objects within tolerance against Diffusion Policy's $37.5\%$ .

Figures & tables

Fig. 1 : Robot completes a successful push during an Out-of-distribution (OOD) experiment with new shapes. Robot demo videos and code available on our project website

Fig. 2 : Architecture diagrams of the Joint prediction and Causal factorization planners. The Joint planner denoises both robot and object trajectories together. In contrast, the Causal planner makes two passes through a shared transformer decoder. The Robot phase runs first and Tweedie’s formula (Eq. ( 1 )) turns this output into a clean robot estimate $\hat{\bm{\tau}}^{R}_{0}$ . The Object phase denoises $\bm{\tau}^{O}_{t}$ while being conditioned on $\hat{\bm{\tau}}^{R}_{0}$ . The resulting chain $\bm{\tau}^{R}_{t}\to\hat{\bm{\tau}}^{R}_{0}\to\hat{\bm{\tau}}^{O}_{0}$ is differentiable through both decoder passes.

Term	Role	Penalty
$C_{\text{place}}^{\star}$	Push	Asymmetric endpoint error along the push axis (overshoot $=25\times$ undershoot) plus squared lateral error. Object endpoint near target.
$C_{\text{pred}}^{\star}$	Push	Penalizes the object’s predicted landing position at the horizon end, assuming it keeps its current along-axis velocity. Lookahead to predict overshoot.
$C_{\text{coll}}^{(o)}$	Safety	Margin violation of the predicted object trajectory against placed objects and obstacles, through the barrier $\Psi(x){=}e^{x}{-}x{-}1$ . Object penetrates margin.
$C_{\text{coll}}^{(r)}$	Safety	Same barrier on the predicted robot trajectory. Robot penetrates margin.
$C_{\text{retreat}}^{\star}$	Retreat	Penalizes robot proximity to the target object when retreat is needed. Bad approach, or target object is placed.

TABLE I : Summary of the five guidance cost terms. Starred (

\star

) terms are novel contributions of this work, while the rest are inspired by [ 18 ] . Each penalty is followed in italics by the condition under which it is active.

Panel of Table 4 — TABLE II : Push-T simulation results with each method at its own best-of- $N$ draws per replan, selected by success rate over $N\in\{1,8,16,32\}$ : $N{=}16$ for Diffusion Forcing [100 DDIM] and Joint [100 DDIM], $N{=}32$ for Causal [100 DDIM], and $N{=}8$ for Causal [50 DDIM]. Causal consistently has the highest success rate across all batch sizes. Each row pools $75$ episodes ( $25$ episodes $\times$ $3$ seeds: $42$ , $1000$ , $2000$ ). Max and Final are reported with their $95\%$ confidence intervals (CI), and SR with its Wilson $95\%$ CI. Latency is roughly constant across the different batch sizes for each model. All models have around $17.8$ M parameters. Rows are color-coded: baselines , ablations , ours (proposed) . Bolded entries mark the best per column.

Panel of Table 5 — TABLE III : Simulation results across 3 previously unseen environments at two goal tolerances ( $4$ cm, $2$ cm), pooled over $150$ episodes per cell ( $3$ seeds $\times$ $50$ episodes). All diffusion methods used 100 DDIM steps, unless otherwise stated, and are trained at a tolerance of $2$ cm. Diffusion Policy uses FiLM [ 30 ] for conditioning whereas Joint and Causal use AdaLN [ 26 ] . For best of N sampling, the sample that minimizes the cost function in Sec. IV-D is selected. SPL and BPF are both reported out of $100$ and with their $95\%$ CIs. Bolded entries mark the best BPF per (environment, tolerance). Causal w/ MC DPS is run at $K=16$ . Rows are color-coded: baselines , ablations , ours (proposed) .

Panel of Figure 6 — Fig. 3 : Failure modes.

Explore similar work

Disentangled Point Diffusion for Precise Object Placement
Apr 13, 2026Lyuxing He, Eric Cai, Shobhit Aggarwal +2Robotic Manipulation Generalization in Robotic Manipulation
The authors are with Carnegie Mellon University, Pittsburgh, USA. · The author is with ABB Inc., USA.
MultiPush: Learning to Rearrange with Teams of Car-Like Pushers
Sep 22, 2026Jeeho Ahn, Christoforos MavrogiannisMulti-Robot Systems Multi-Robot Motion Planning
Robotics Department, University of Michigan, Ann Arbor, USA
Trajectory-Level Mode Guidance for Controllable Diffusion-Based Multi-Robot Motion Planning
Sep 29, 2026Tianyou Yu, Shengze Cai, Chao XuMulti-Robot Motion Planning Trajectory Generation
Institute of Cyber-System and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China

Apr 13, 2026J/K move · Enter open · S save

Disentangled Point Diffusion for Precise Object Placement

Lyuxing He, Eric Cai, Shobhit Aggarwal, Jianjun Wang+1

The authors are with Carnegie Mellon University, Pittsburgh, USA. · The author is with ABB Inc., USA.

Robotic Manipulation Generalization in Robotic Manipulation Point Cloud Generation

Open paper AskPDF arXiv

Recent advances in robotic manipulation have highlighted the effectiveness of learning from demonstration. However, while end-to-end policies excel in expressivity and flexibility, they struggle both in generalizing to novel object geometries and in attaining a high degree of precision. An alternative, object-centric approach frames the task as predicting the placement pose of the target object, providing a modular decomposition of the problem. Building on this goal-prediction paradigm, we propose TAX-DPD, a hierarchical, disentangled point diffusion framework that achieves state-of-the-art performance in placement precision, multi-modal coverage, and generalization to variations in object geometries and scene configurations. We model global scene-level placements through a novel feed-forward Dense Gaussian Mixture Model (GMM) that yields a spatially dense prior over global placements; we then model the local object-level configuration through a novel disentangled point cloud diffusion module that separately diffuses the object geometry and the placement frame, enabling precise local geometric reasoning. Interestingly, we demonstrate that our point cloud diffusion achieves substantially higher accuracy than a prior approach based on SE(3)-diffusion, even in the context of rigid object placement. We validate our approach across a suite of challenging tasks in simulation and in the real-world on high-precision industrial insertion tasks. Furthermore, we present results on a cloth-hanging task in simulation, indicating that our framework can further relax assumptions on object rigidity.

Abstract

Explore similar work

Disentangled Point Diffusion for Precise Object Placement

MultiPush: Learning to Rearrange with Teams of Car-Like Pushers

Trajectory-Level Mode Guidance for Controllable Diffusion-Based Multi-Robot Motion Planning