Learning Task and Motion Plans from Real Demonstrations with Hybrid Flow Matching

Zuleika Redondo Garcia; Andreu Matoses Gimenez; Javier Alonso-Mora

cs.ROOct 3, 2026

Learning Task and Motion Plans from Real Demonstrations with Hybrid Flow Matching

Authors: Zuleika Redondo Garcia, Andreu Matoses Gimenez, Javier Alonso-Mora

Organizations: Department of Cognitive Robotics, Delft University of Technology, 2628 CD Delft, The Netherlands

Abstract

Long-horizon mobile manipulation requires a task plan and the motion that executes it. Generative planners trained on demonstration produce both in one pass, requiring neither a symbolic domain nor search. To date, however, they have relied on thousands of scripted demonstrations of fixed-base arms and executed open loop. This paper presents a hybrid flow matching planner: a single network generates the symbolic plan with masked discrete flow matching and the motion trajectory with continuous flow matching. Unlike prior generative planners, we aim to learn from a much smaller set of demonstrations and to execute the plan in closed loop. Two properties of the data compensate for the small dataset. A demonstration resumed from any of its intermediate actions is itself a demonstration, which multiplies the training samples and enables replanning after every action. Objects of the same kind are interchangeable, which turns demonstrations of one goal into demonstrations of every permuted goal. Our base implementation produces valid plans on 68% of held-out scenes; a training and generation scheme for the discrete plan raises this to 76%, and replanning after every action raises the task completion rate from 40% to 53% in a kinematic simulation. The planner matches the task completion rate of motion-only flow matching policies while additionally providing the symbolic plan, and it outperforms previous hybrid diffusion formulations on both task completion and plan validity. We validate the planner on a real mobile manipulator. Videos and project page: https://andreumatoses.github.io/research/hybrid-flow-planning

Figures & tables

Fig. 1: The planner on the real robot building the tower R on Y on G. Four frames of one run: the start and end of the first half of the task (top) and of the second half (bottom). Each frame shows the plan the planner generated from that scene, with the action in progress highlighted, and the generated waypoints of the base and end effector. The planner generates the waypoints of the whole task; for readability only those of the half shown are drawn.

Fig. 2: The running example. Three cubes R, Y, and G are placed on tables T1–T3, with approach regions RG1–RG10 located along the table sides. The observation

o

contains the poses of the mobile manipulator and the cubes. A plan consists of a sequence of symbolic actions, such as Pick G , while the goal

g

specifies the desired final configuration: R on Y on G on T1, illustrated by the ghost cubes.

Fig. 3: The network. The symbolic plan tokens and the waypoints get a token embedding or a linear layer, a learned modality and positional embedding, and form one sequence for the transformer. Its cross-attention reads the memory: one token per flow time (Fourier features) and one per observation component (an MLP), each with a type embedding. The token head gives $p_{\theta}$ at every symbolic position and the velocity head $v_{\theta}$ at every waypoint.

Fig. 4: Plan suffixes of one demonstration. Top: the end-effector path over the scene, colored per action (blue Move_to , yellow Pick Y and Place Y , red Pick R and Place R ); the grey curves mark the boundary jitter around the action boundaries. Bottom: the plan

\sigma

and the suffixes that start at the observations

o_{1},\ldots,o_{M}

; each suffix is a training sample.

Fig. 5: Symbolic permutation of the same demonstration. Renaming the cubes (R to G, Y to R, G to Y) in the goal, the plan, and the observation gives a demonstration of another tower order with the same trajectory.

Fig. 6: The 109 (+12 validation) demonstrations. Left: top view with the tables T1–T3 and the approach regions RG1–RG10: base paths (grey), end-effector paths (violet), start poses (arrows), and initial cube positions (dots). Right: one demonstration represented by observations $o_{1},\ldots,o_{8}$ at the action boundaries, together with the corresponding actions, wrist-camera views, and the distribution of plan lengths at the bottom right.

Fig. 7: A run on the real robot with the goal R on Y on G on table T1 (cubes R red, Y yellow, G green). After every action the policy observes the new scene and samples a new plan and trajectory. Here every action succeeds, so the plan stays the same from one planning step to the next and the run looks like an open-loop execution.

Fig. 8: Success against the number of actions left to the goal, on the 12 held-out demonstrations (mean and standard deviation over three training seeds, 50 samples per task). Values of

d

with few demonstrations are grouped: 13–18, 11–12, and 9–10 hold 850, 750, and 900 tasks, the other values 550 to 600 each. Left: the sampled plan is valid. Middle: the tower stands after the whole sampled trajectory is executed. Right: the tower stands when the policy replans after every action. Hybrid diffusion is the formulation of Høeg et al. [ 13 ] , trained on our suffix samples or on whole demonstrations. Motion-only FM is flow matching on the trajectory alone; it has no plan and no left panel. *The goal-conditioned policy builds any of the six tower orders and is scored on the recorded one.

	Symbolic plan valid † (%)		Task completed ‡ (%)
Policy	all $d$	$d\geq 13$	open loop	closed loop
Motion-only FM	–	–	39 $\pm$ 2	52 $\pm$ 2
Hybrid diffusion [ 13 ] :
whole demonstrations	0 8 $\pm$ 1	0 1	0 0	0 0
our suffix samples	62 $\pm$ 1	0 5	0 6 $\pm$ 0	0 9 $\pm$ 0
Ours	76 $\pm$ 1	33	40 $\pm$ 3	53 $\pm$ 3

TABLE I: Performance in percent on the 12 held-out demonstrations, averaged over 3 training seeds and 50 samples per task.

Fig. 9: The same setup with a failure. The grasp of R misses; at the next planning step, the policy sees R still on the table, plans the pick again, and finishes the tower. Replanning after every action turns a missed grasp into a retry instead of a failed run.

Long-Horizon Robotic Manipulation

Long-horizon robotic manipulation studies how robots plan, remember, and recover across extended sequences of physical tasks.

See all 431 papers

Flow Motion Policy: Manipulator Motion Planning with Flow Matching Models
Apr 8, 2026Davood Soleymanzadeh, Xiao Liang, Minghui ZhengRobot Motion Planning Robot Motion Generation
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA · Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843 USA
ConFlow: Constraints-Guided Learning with Flow Matching for Motion Generation
Jul 15, 2026Nutan Chen, Jianxiang Feng, Marvin Alles +1Constrained Generative Modeling Flow Matching
LS Wiiri Robot Innovation Center · Independent Researcher · Technical University of Munich +1
HDFlow: Hierarchical Diffusion-Flow Planning for Long-horizon Tasks
May 6, 2026Nandiraju Gireesh, Yuanliang Ju, Chaoyi Xu +3Hierarchical Planning Long-Horizon Robotic Manipulation
1Peking University · University of Toronto. · 2Galbot

Apr 8, 2026J/K move · Enter open · S save

Flow Motion Policy: Manipulator Motion Planning with Flow Matching Models

Davood Soleymanzadeh, Xiao Liang, Minghui Zheng

J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA · Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843 USA

Robot Motion Planning Robot Motion Generation

Open paper AskPDF arXiv

Open-loop end-to-end neural motion planners have recently been proposed to improve motion planning for robotic manipulators. These methods enable planning directly from sensor observations without relying on a privileged collision checker during motion planning. However, existing planners produce a single path for a given planning problem and cannot exploit their open-loop nature to propose multiple motion plans. To address this limitation, we introduce Flow Motion Policy, an open-loop neural motion planner that uses flow matching to generate a batch of motion plan proposals by learning a distribution over motion plans conditioned on the planning observation. At inference time, it samples multiple candidate motion plans to enable efficient best-of-

N

inference while avoiding iterative collision checking during planning. We benchmark the Flow Motion Policy against representative sampling-based, optimization-based and neural motion planning methods. Evaluation results demonstrate that Flow Motion Policy improves planning success and efficiency, highlighting the effectiveness of stochastic generative policies for end-to-end motion planning and best-of-

N

sampling. Project website: https://davoodsz.github.io/FlowMotionPolicy.github.io/

Abstract

Long-Horizon Robotic Manipulation

Explore similar work

Flow Motion Policy: Manipulator Motion Planning with Flow Matching Models

ConFlow: Constraints-Guided Learning with Flow Matching for Motion Generation

HDFlow: Hierarchical Diffusion-Flow Planning for Long-horizon Tasks