Masked Generative Motion Planning with Geometry-Guided Token Search

Lipeng Zhuang; Yingdong Ru; Shiyu Fan; Edmond S. L. Ho; Gerardo Aragon Camarasa; Paul Henderson

cs.ROOct 7, 2026

Masked Generative Motion Planning with Geometry-Guided Token Search

Authors: Lipeng Zhuang, Yingdong Ru, Shiyu Fan, Edmond S. L. Ho, Gerardo Aragon Camarasa, Paul Henderson

Organizations: University of Glasgow, Glasgow, United Kingdom

Abstract

Generative motion planners typically use learned trajectory priors for initial generation, while leaving test-time repair to local continuous refinement. We introduce Masked Generative Motion Planning (MGMP), which extends the learned prior from efficient parallel generation to structural repair. A masked generative transformer generates discrete trajectory candidates in parallel, and Geometry-Guided Token Search (GGTS) uses scene geometry to target where to edit and which prior-supported alternatives to evaluate. This turns refinement into an efficient search over discrete motion alternatives, enabling route-level restructuring beyond local trajectory deformation. MGMP achieves 96% success on Ring Maze and 82% repair success on Controlled Route Invalidation on Kuka, exceeding the strongest external baselines by 23 and 25 percentage points, respectively. It further generalizes to unseen layouts, additional obstacles, unseen geometries, single- and dual-arm planning, and real-world Baxter tasks.

Figures & tables

Figure 1: Overview of MGMP . We replace continuous trajectory generation and refinement with discrete masked generation and Geometry-Guided Token Search, enabling efficient planning and structural route repair. We evaluate it across challenging simulated and real-world settings, including basin escape, unseen geometries, additional obstacles, dual-arm planning, and Baxter tasks.

Figure 2: Overview of GGTS. We use geometric attribution to select edit positions, first-order scores to screen token replacements, and sequential verification to accept improving candidates, followed by optional continuous polish.

Methods	Ring Maze			Controlled Route Invalidation on Kuka
	SR $\uparrow$	VSF $\uparrow$	RT $\downarrow$	RS $\uparrow$	Avg. Dev.	Max. Dev.	Changed Span
CHOMP	$0.73$ ±0.09	$0.23$ ±0.10	$0.56$ ±0.14	$0.46$ ±0.15	$0.82$ ±0.19	$1.30$ ±0.32	$31.20$ ±9.40
TrajOpt	$0.66$ ±0.04	$0.22$ ±0.03	$2.64$ ±0.01	$0.53$ ±0.14	$0.88$ ±0.20	$1.61$ ±0.38	$34.80$ ±10.10
STOMP	$0.23$ ±0.01	$0.05$ ±0.03	$0.45$ ±0.07	$0.57$ ±0.15	$0.96$ ±0.22	$1.66$ ±0.35	$39.60$ ±10.80
MPD	$0.14$ ±0.06	$0.02$ ±0.02	$0.21$ ±0.06	$0.38$ ±0.13	$0.83$ ±0.20	$1.43$ ±0.35	$31.90$ ±9.60
PBD	$0.67$ ±0.10	$0.26$ ±0.09	$0.16$ ±0.05	$0.44$ ±0.14	$0.93$ ±0.22	$1.65$ ±0.39	$37.20$ ±10.30

Table 1: Results on Ring Maze and Controlled Route Invalidation on Kuka. SR, VSF, RT, and RS denote success rate, valid-sample fraction, running time, and repair success rate, respectively. For Controlled Route Invalidation, trajectory-deviation statistics are reported over successful repairs.

Figure 3: Planning efficiency and qualitative comparison. (a) Valid-sample fraction versus wall-clock time across three settings. (b) Dual-Kuka results under unseen obstacle geometries (top) and additional obstacles (bottom); PBD collides (red), while MGMP is collision-free (blue).

Figure 4: (a) Qualitative comparison on Ring Maze. Top and bottom rows show two planning problems. Dashed/solid curves denote initial/refined trajectories, and green/red denote collision-free/colliding solutions. (b) Example of Controlled Route Invalidation on Kuka.

Methods	Maze2D-6			Concave Maze2D			7-DoF Kuka			14-DoF DualKuka
	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$
RMP	$0.44$ ±0.13	–	$3.09$ ±0.59	$0.38$ ±0.13	–	$3.42$ ±0.64	$0.81$ ±0.14	–	$1.24$ ±0.72	$0.65$ ±0.18	–	$2.25$ ±0.88
RRT*	$0.99$ ±0.01	–	$0.54$ ±0.13	$0.98$ ±0.03	–	$1.24$ ±0.22	$0.64$ ±0.17	–	$2.31$ ±0.78	$0.44$ ±0.17	–	$3.49$ ±0.64
P-RRT*	$0.99$ ±0.02	–	$0.86$ ±0.18	$0.96$ ±0.05	–	$1.60$ ±0.27	$0.63$ ±0.18	–	$2.46$ ±0.80	$0.42$ ±0.17	–	$3.58$ ±0.64
BIT*	$\mathbf{1.00}$ ±0.01	–	$0.15$ ±0.02	$\mathbf{1.00}$ ±0.01	–	$0.29$ ±0.04	$\mathbf{0.97}$ ±0.02	–	$0.39$ ±0.25	$0.85$ ±0.14	–	$2.01$ ±0.76
CHOMP	$0.99$ ±0.01	$0.64$ ±0.07	$0.35$ ±0.03	$0.99$ ±0.01	$0.55$ ±0.02	$0.24$ ±0.08	$0.89$ ±0.04	$0.62$ ±0.02	$1.50$ ±0.08	$0.81$ ±0.05	$0.49$ ±0.04	$1.74$ ±0.12

Table 2: Results on unseen obstacle layouts. SR, VSF, and PT denote success rate, valid-sample fraction, and planning time in seconds. Results are reported as mean

\pm

standard deviation.

Methods	Kuka Add.-Obs.			DualKuka Add.-Obs.			Kuka Unseen Geo.			DualKuka Unseen Geo.
Methods	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$	SR $\uparrow$	VSF $\uparrow$	PT $\downarrow$
RMP	$0.77$ ±0.22	–	$1.40$ ±1.12	$0.48$ ±0.19	–	$3.05$ ±0.88	$0.77$ ±0.15	–	$1.47$ ±0.78	$0.65$ ±0.17	–	$2.21$ ±0.80
RRT*	$0.65$ ±0.23	–	$2.24$ ±1.03	$0.30$ ±0.16	–	$4.01$ ±0.56	$0.60$ ±0.15	–	$2.58$ ±0.73	$0.46$ ±0.19	–	$3.44$ ±0.74
P-RRT*	$0.66$ ±0.23	–	$2.38$ ±1.02	$0.29$ ±0.17	–	$4.14$ ±0.54	$0.57$ ±0.15	–	$2.84$ ±0.70	$0.43$ ±0.18	–	$3.58$ ±0.70
BIT*	$0.95$ ±0.10	–	$0.73$ ±0.62	$0.63$ ±0.21	–	$3.44$ ±0.81	$\mathbf{0.91}$ ±0.10	–	$0.99$ ±0.58	$0.83$ ±0.19	–	$1.88$ ±0.96
CHOMP	$0.77$ ±0.02	$0.52$ ±0.04	$2.16$ ±0.05	$0.68$ ±0.07	$0.26$ ±0.06	$2.95$ ±0.01	$0.65$ ±0.02	$0.33$ ±0.01	$2.20$ ±0.06	$0.57$ ±0.08	$0.34$ ±0.05	$2.68$ ±0.08

Table 3: Results on additional-obstacle and unseen-geometry environments. SR, VSF, and PT denote success rate, valid-sample fraction, and planning time in seconds, respectively.

Variant	Ring Maze			7-DoF Kuka
Variant	SR $\uparrow$	VSF $\uparrow$	RT $\downarrow$	SR $\uparrow$	VSF $\uparrow$	RT $\downarrow$
MGT (Conf.-Refine)	0.69	0.20	0.03	0.84	0.58	0.07
MGT + EmbOpt-30	0.75	0.29	0.08	0.90	0.74	0.30
Conf.-only Remask	0.84	0.49	0.08	0.92	0.78	0.43
Geom.-only Remask	0.93	0.64	0.08	0.95	0.84	0.44
Prob.-only Screening	0.88	0.55	0.07	0.94	0.81	0.41

Table 4: Ablation results on Ring Maze and 7-DoF Kuka. SR, VSF, and RT denote success rate, valid-sample fraction, and runtime.

Figure 5: Real-world tasks: (a) button pressing; (b) shelf placing; (c) drawing.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Maze2D	7-DoF Kuka	14-DoF DualKuka
Codebook size	8192	8192	8192
Code dimension	16	256	256
Encoder width	16	256	256
Encoder depth	4	4	4
Downsampling rate	2	2	1
Stride size	2	2	2

Appendix

Table 6: VQ-VAE hyperparameters.

Parameter	Maze2D	7-DoF Kuka	14-DoF DualKuka
GGTS refinement rounds	2	2	2
Token updates per round	2	2	2
Proposed code candidates per update	64	16	16
Candidates retained after screening	32	2	2
Maximum verification remask ratio	0.4	0.4	0.4
Polishing iterations	5	5	5

Appendix

Table 8: Test-time GGTS and continuous polishing hyperparameters. During verification, the number of remasked tokens is capped at 40% of the trajectory tokens; for the Kuka setting, this corresponds to at most four tokens. Continuous polishing uses momentum-accumulated raw gradients with no separate learning rate. The SDF and SG weights are switched to their post-convergence values once the SDF loss falls below

5{\times}10^{-4}

Figure 6: Object-level mesh obstacles used for evaluating generalization to unseen obstacle geometries. These objects replace the box obstacles used during training, creating test environments with irregular and previously unseen obstacle shapes.

(a) CHOMP: optimization iterations
Iterations	10	20	30	50
SR (%) $\uparrow$	73.30	73.30	73.30	73.30
VSF (%) $\uparrow$	22.30	23.30	23.70	23.30

Appendix

Table 9: Effect of refinement budget on Ring Maze. SR and VSF denote success rate and valid-sample fraction, respectively.

Figure 7: Qualitative results on unseen obstacle layouts in Maze2D. We compare BIT*, PBD, and MGMP on Maze2D-6 and Concave Maze2D. MGMP generates collision-free trajectories that reach the goal while avoiding unseen obstacle placements.

Figure 8: Qualitative results on unseen obstacle layouts in 7-DoF Kuka. Each column compares trajectories generated by BIT*, PBD, and MGMP under the same obstacle configuration. MGMP generates feasible joint-space trajectories that avoid cluttered obstacles.

Figure 9: Additional qualitative results on unseen obstacle layouts in 7-DoF Kuka. MGMP produces collision-free plans across different start-goal pairs and obstacle placements.

Figure 10: Qualitative results on unseen obstacle layouts in 14-DoF DualKuka. The examples compare BIT*, PBD, and MGMP in higher-dimensional dual-arm planning scenes. MGMP generates feasible trajectories while avoiding both workspace obstacles and inter-arm collisions.

Figure 11: Additional qualitative results on unseen obstacle layouts in 14-DoF DualKuka. MGMP remains robust across different obstacle placements and start-goal configurations.

Figure 12: Qualitative results on additional-obstacle 14-DoF DualKuka environments. Compared with the base setting, these scenes include more obstacles at test time. MGMP generates collision-free trajectories in the more constrained workspace.

Figure 13: Additional qualitative results on additional-obstacle 14-DoF DualKuka environments. MGMP remains effective when the test scene contains more constraints than those observed during training.

Figure 14: Qualitative results on unseen obstacle geometries in 14-DoF DualKuka. Cuboid training obstacles are replaced by unseen object-level mesh obstacles. MGMP adapts to these geometry shifts and generates collision-free trajectories around irregular objects.

Figure 15: Additional qualitative results on unseen obstacle geometries in 14-DoF DualKuka. MGMP maintains feasible planning performance under realistic mesh obstacles that differ from the box obstacles used during training.

Variant	Position Selection	Candidate Shortlist	Exact Verification	Commit	Continuous Opt.
MGT (Conf.-Refine)	Confidence	MGT prob.	–	–	–
MGT + EmbOpt-30	–	–	–	–	EmbOpt-30
Conf.-only Remask	Confidence	Eq. (2), $M{\rightarrow}k$	top- $k$	Sequential	Polish
Geom.-only Remask	Geometry	Eq. (2), $M{\rightarrow}k$	top- $k$	Sequential	Polish
Probability-only Screening	Geom.+Conf.	MGT prob., $M{\rightarrow}k$	top- $k$	Sequential	Polish
Independent Exact Search	Geom.+Conf.	Eq. (2), $M{\rightarrow}k$	top- $k$	Independent	Polish

Appendix

Table 10: Refinement configurations used in the ablation study. For GGTS variants, discrete refinement consists of position selection, candidate shortlisting, exact verification, and token commitment, followed by optional continuous polishing.

(a) Candidate shortlist size $k$				(b) Refinement schedule				(c) Continuous polishing
$k$	SR $\uparrow$	VSF $\uparrow$	RT $\downarrow$	Schedule	SR $\uparrow$	VSF $\uparrow$	RT $\downarrow$	Steps	SR $\uparrow$	VSF $\uparrow$	RT $\downarrow$
1	0.95	0.84	0.38	$1\times 1$	0.93	0.79	0.34	0	0.94	0.83	0.40
2	0.97	0.87	0.44	$1\times 4$	0.95	0.83	0.40	3	0.96	0.86	0.42
4	0.97	0.88	0.56	$\mathbf{2\times 2}$	0.97	0.87	0.44	5	0.97	0.87	0.44
16	0.98	0.89	1.28	$4\times 1$	0.96	0.85	0.52	10	0.97	0.88	0.48

Appendix

Table 11: Additional analysis of GGTS on 7-DoF Kuka. (a–c) Sensitivity to the candidate shortlist size, refinement schedule, and continuous polishing budget. The MGT proposal size is fixed to

M=16

, and

k

denotes the number of candidates retained for exact verification. For the refinement schedule,

1\times 4

2\times 2

, and

4\times 1

use the same nominal refinement budget, while

1\times 1

serves as a minimal-refinement reference. (d) Comparison of first-order screening and exact verification. Default settings are shown in bold.

Figure 16: Real-world tabletop setups for Baxter experiments. Figures (a)–(c) show the 3D trajectory setting used for shelf placing and button pressing, where the robot plans obstacle-avoiding motions through the workspace. Panel (d) shows the 2D tabletop drawing setting, where the robot follows a near-surface trajectory while holding a pen.

Figure 17: Qualitative results for the real-world pen-drawing task. Rows (a) and (b) show successful executions, where the robot follows the planned 2D tabletop trajectory and draws a line to the target while avoiding obstacles. Row (c) shows a failure case caused by collision with an obstacle during execution. Row (d) shows final drawn trajectories.

Figure 18: Qualitative results for the real-world button-pressing task. Rows (a) and (b) show successful executions. Row (c) shows a failure case: the robot does not reach the correct pre-press goal pose.

Figure 19: Qualitative results for the real-world shelf-placing task under three target shelf modes. MGMP successfully generates obstacle-avoiding trajectories that reach valid release poses. Failure cases are mainly caused by obstacle collisions or unsuitable approach trajectories that prevent successful placement.

Explore similar work

ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation
Sep 8, 2026Yiran Wang, Zeyu Zhang, Ling Shao +1Text-to-Motion Generation Human Motion Generation
University of Sydney, Camperdown 2006, Australia · School of Computer Science, Peking University, Beijing 100871, China · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Beijing 101408, China
GASP: GPU-Accelerated Safe Planner for Real-Time Collision-Aware Motion Generation with Latent Trajectory Sampling
Aug 5, 2026Colin Merk, Stefanos Charalambous, Peter Dürr +1Constrained Motion Planning Trajectory Generation
Sony, Zurich, Switzerland.
Sum of Costs Diffusion with Dynamic Guidance for Motion Planning
May 23, 2026Aysu Aylin Kaplan, Özgür ErkentTrajectory Generation Diffusion-Based Planning
Computer Engineering Department, Hacettepe University, Ankara, Turkey

Sep 8, 2026, cs.CVJ/K move · Enter open · S save

ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation

Yiran Wang, Zeyu Zhang, Ling Shao, Hao Tang

University of Sydney, Camperdown 2006, Australia · School of Computer Science, Peking University, Beijing 100871, China · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Beijing 101408, China

Text-to-Motion Generation Human Motion Generation Motion Representation Learning

Open paper AskPDF arXiv

Text-to-motion (T2M) generation maps natural language to human joint movements, aiding gaming, VR, and robotics. Retrieval-Augmented Text-to-Motion (RAG-T2M) improves generation on complex descriptions by conditioning on retrieved motion-text pairs. However, existing RAG-T2M models face two challenges: coarse-grained retrieval and fusion mechanisms overlook the hierarchical, spatial-temporal topology of human motion, and a representation gap exists because retrieved evidence resides in a semantic space separate from the generator's latents. To address the first, we present ReMoMask, a structure-aware RAG framework coupling Hierarchical Bidirectional Momentum (HBM) contrastive learning to align global and part-level features with text; Semantic Spatial-Temporal Attention (SSTA) for topology-aware fusion; and Topology Structured Masking (TSM) to force robust part-level grounding via adaptive masking. To address the second, we introduce ReMoMask-2, which rebuilds the retrieval database directly within the generator's pre-quantization latent space and aligns text queries via a distilled lightweight projector, allowing the generator to directly consume the retrieved motion's semantic content. Extensive experiments on HumanML3D, KIT-ML, and SnapMoGen demonstrate our retriever achieves state-of-the-art accuracy, while ReMoMask-2 attains the lowest FID on KIT-ML and SnapMoGen; notably, its single mask-transformer stage surpasses ReMoMask's full two-stage pipeline and delivers the fastest inference.

Abstract

Explore similar work

ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation

GASP: GPU-Accelerated Safe Planner for Real-Time Collision-Aware Motion Generation with Latent Trajectory Sampling

Sum of Costs Diffusion with Dynamic Guidance for Motion Planning