TOPS: Reshaping Flow Source and Transport for Multi-Task Robot Manipulation

Tuo An; Chuhao Zhou; Jiahua Dong; Jingliang Li; Jiaqi Bai; Jindou Jia; Jianfei Yang

cs.ROOct 27, 2026

TOPS: Reshaping Flow Source and Transport for Multi-Task Robot Manipulation

Authors: Tuo An, Chuhao Zhou, Jiahua Dong, Jingliang Li, Jiaqi Bai, Jindou Jia, Jianfei Yang

Organizations: MARS Lab, Nanyang Technological University

Abstract

Flow-based policies for language-conditioned multi-task manipulation commonly share a single $\textit{Gaussian}$ source prior and a common linear transport path across tasks. However, reliably executing multiple tasks with distinct motion requirements remains challenging for a shared policy. We introduce $\textbf{TOPS}$ ( $\textbf{T}$ ask- $\textbf{O}$ riented $\textbf{P}$ riors and $\textbf{S}$ chedules), which adapts the flow source prior and interpolation schedule to each task without modifying the network architecture. TOPS first constructs multimodal task sources from reverse-mapped expert action sequences, allowing language instructions to select where generation begins. Guided by source-target covariance mismatch, it then slows the transition toward expert actions early in denoising and accelerates it later along directions requiring expansion. Under a $\textit{Gaussian}$ endpoint model, our analysis shows that this schedule adjustment can lower an upper bound on velocity-prediction risk. Experiments on LIBERO, LIBERO-Plus, and CALVIN demonstrate improved performance across in-distribution, perturbation and cross-environment settings, alongside better data efficiency. Using one integration step, TOPS raises LIBERO-10 success from $53.44\%$ to $69.67\%$ compared with ten-step Standard Flow Matching (FM), while reducing inference time to approximately one-sixth of the baseline. TOPS also yields consistent gains when transferred to the large-scale flow-based $π_{0.5}$ policy across all four evaluated settings. Real-world experiments further demonstrate improved task success, including an increase from $20\%$ with standard FM to $80\%$ with TOPS on the upper-drawer opening task.

Figures & tables

Figure 1: Task-structured reverse images and the fitted source. (a) t-SNE of reverse images from the ten LIBERO-10 tasks: tasks occupy distinct, partially overlapping regions. (b) The same embedding with one task highlighted per view, showing sub-clusters within individual tasks. (c) Samples from fitted per-task source in the same coordinates, reproducing the structure in (a).

Figure 2: Overview of TOPS. (a) We construct task-specific sources by fitting MPPCA models to reverse-mapped expert actions. An instruction selector maps the input language to an index $\hat{\tau}$ , identifying the source $q_{\hat{\tau}}$ and its associated schedule. (b) In the selected task’s Gaussian OT eigenbasis, we apply slow-then-fast interpolation along expansion directions and linear interpolation elsewhere, then map the interpolated states and velocity targets back to action space.

	LIBERO			CALVIN
Method	ID	OOD	$t$ (ms)	D $\to$ D	ABC $\to$ D	$t$ (ms)
Standard FM ( Lipman et al., 2022 )	53.44 $\pm$ 2.93	27.11 $\pm$ 0.34	49.4	1.90 $\pm$ 0.05	0.70 $\pm$ 0.03	56.2
OT-CFM ( Tong et al., 2024 )	30.17 $\pm$ 2.72	19.02 $\pm$ 0.12	49.4	1.28 $\pm$ 0.05	0.46 $\pm$ 0.02	58.9
2-ReFlow ( Liu et al., 2023b )	37.17 $\pm$ 3.12	20.22 $\pm$ 0.35	7.1	1.74 $\pm$ 0.05	0.66 $\pm$ 0.03	8.6
RF-distill ( Liu et al., 2023b )	29.17 $\pm$ 0.94	17.56 $\pm$ 0.06	4.8	1.75 $\pm$ 0.05	0.70 $\pm$ 0.03	5.8
MeanFlow ( Geng et al., 2025 )	59.00 $\pm$ 2.83	34.73 $\pm$ 0.32	5.7	1.89 $\pm$ 0.02	0.86 $\pm$ 0.02	5.4

Table 1: Main results and inference cost. LIBERO: success rate (%); CALVIN: mean completed chain length (out of five); higher is better for both.

t

: per-call latency (ms). Results are mean

\pm

sample s.d. across three independent training runs, after averaging evaluation repeats within each run. LIBERO-Plus means cover 2,100 variants across six categories, weighted by variant count; Camera is excluded because all evaluated methods score zero. Appendix A.1 provides category and chain-depth breakdowns.

Figure 3: Data efficiency. Performance with 5%–50% of the training demonstrations on LIBERO-10, LIBERO-Plus, CALVIN D

\to

D, and CALVIN ABC

\to

D. TOPS matches or exceeds the strongest baseline in 14 of the 20 benchmark–data-budget configurations.

Figure 4: Transfer to the large-scale flow-based

\pi_{0.5}

policy. Both variants use the same 14K-step training budget. The baseline uses ten Euler inference steps, while TOPS uses one schedule-aware step (NFE=2). Detailed results are provided in App. A.1.1 .

Figure 5: Real-world evaluation. Left: example scenes from the three task suites. Right: per-task success rates (%) and suite averages (Avg.). Each task is evaluated over 20 rollouts.

Figure 6: Ablations of TOPS. Left: LIBERO-10 success (%) versus component count

M_{\tau}

. Middle and right: path ablations on CALVIN D

\to

D and ABC

\to

D (mean completed chain length). FM denotes Standard FM; Rand. dir. replaces expansion directions with random ones; Trig. uses trigonometric interpolation. The dark bar denotes full TOPS. Error bars: LIBERO-10 s.d. across seeds (training for FM; evaluation for

M_{\tau}

variants) and CALVIN s.e.m. over 1,000 chains.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Interpolation scheduling for few-step generation. The left three columns compare linear (standard), fast-then-slow, and slow-then-fast (ours) scheduling, respectively. The two rows show generated endpoints after one and two integration steps, corresponding to NFE = 2 and 4, respectively. Stars mark the source mean, dashed outlines indicate the target capture regions, and the rightmost column compares the generated $y$ -marginals with the target distribution (gray). With two steps, capture reaches 84%, 40%, and 99%, respectively. With one step, fast-then-slow scheduling leaves 56% of samples in the central band, compared with 22% for linear and 13% for ours. Slow-then-fast scheduling achieves the highest capture at both integration budgets, supporting our scheduling design.

Perturbation	$\pi_{0.5}$	$\pi_{0.5}$ + TOPS
Lighting	77.5	94.2
Object layout	74.1	84.3
Background texture	79.0	81.9
Sensor noise	46.8	51.8
Camera viewpoint	38.4	40.0
Robot initial state	74.4	76.1

Appendix

Table 2: LIBERO-Plus perturbation breakdown for

\pi_{0.5}

and

\pi_{0.5}

+ TOPS. Success rates (%) are reported for each perturbation category and overall.

Setting	Method	$\geq 1$	$\geq 2$	$\geq 3$	$\geq 4$	$\geq 5$	Len
D $\rightarrow$ D	$\pi_{0.5}$	87.4	79.2	72.3	66.7	61.7	3.673
	$\pi_{0.5}$ + TOPS	90.7	83.7	78.4	73.0	68.3	3.941
ABC $\rightarrow$ D	$\pi_{0.5}$	83.4	69.1	59.0	50.6	41.2	3.033
	$\pi_{0.5}$ + TOPS	91.4	80.9	73.7	67.3	61.1	3.744

Appendix

Table 3: CALVIN chain-depth results for

\pi_{0.5}

and

\pi_{0.5}

+ TOPS. Columns

\geq k

report the percentage of five-task chains whose first

k

subtasks are completed successfully; Len denotes average completed chain length.

Method	Type	Robot	Lang.	Object	Backgr.	Sensor	Light	Weighted
		(393)	(383)	(312)	(289)	(449)	(274)	(2100)
Standard FM	Ref.	54.0	54.0	21.7	13.6	4.7	8.2	27.1 $\pm$ 0.3
OT-CFM	Coupl.	41.1	40.8	10.0	4.5	6.2	3.3	19.0 $\pm$ 0.1
2-ReFlow	Few-step	38.1	42.0	17.4	9.6	4.5	4.4	20.2 $\pm$ 0.4
RF-distill	Few-step	33.1	35.0	15.3	8.4	3.6	6.1	17.6 $\pm$ 0.1
MeanFlow	Few-step	76.1	72.7	22.1	10.9	8.4	4.9	34.7 $\pm$ 0.3

Appendix

Table 4: LIBERO-Plus results by perturbation category. Success rates (%), with variant counts in parentheses. Type denotes the method category. Weighted is the variant-count-weighted mean over the six reported categories (Camera excluded, since all methods are zero). Uncertainty follows the seed and aggregation convention of Tab. 1 . Bold and underlined values indicate the highest and second-highest means in each column, respectively.

		D $\to$ D (in-distribution)						ABC $\to$ D (unseen env.)
Method	Type	Len	$\geq$ 1	$\geq$ 2	$\geq$ 3	$\geq$ 4	$\geq$ 5	Len	$\geq$ 1	$\geq$ 2	$\geq$ 3	$\geq$ 4	$\geq$ 5
Standard FM	Ref.	1.90 $\pm$ .05	72.9	52.2	32.2	20.3	11.9	0.70 $\pm$ .03	42.8	16.8	7.0	2.5	0.5
OT-CFM	Coupl.	1.28 $\pm$ .05	59.8	34.6	18.8	9.8	4.9	0.46 $\pm$ .02	32.9	10.0	2.8	0.6	0.1
2-ReFlow	Few-step	1.74 $\pm$ .05	72.0	47.5	28.6	17.1	8.9	0.66 $\pm$ .03	42.3	15.8	5.5	2.4	0.4
RF-distill	Few-step	1.75 $\pm$ .05	71.0	48.3	28.9	17.0	9.9	0.70 $\pm$ .03	44.1	17.7	6.5	1.7	0.2
MeanFlow	Few-step	1.89 $\pm$ .02	71.4	49.2	32.9	22.0	13.7	0.86 $\pm$ .02	50.5	23.9	8.6	2.7	0.6

Appendix

Table 5: CALVIN long-horizon chains. Average completed chain length and the percentage of the 1000 chains whose first

k

subtasks all succeeded (

\geq k

); higher is better throughout. Type denotes the method category.

Figure 8: Action-coordinate structure of OT directions on LIBERO-10. (a,b): covariance-cost-weighted squared loadings for expansion (

s_{\tau j}>1

) and non-expansion (

s_{\tau j}\leq 1

) directions, respectively, defined in Eq. 14 . Each task’s map is independently normalized to sum to one before averaging across the ten tasks; both heatmaps use the same color scale. Columns index chunk timesteps and rows index action coordinates. (c): loading shares summed over position, rotation, and gripper coordinates. Grey bars indicate the uniform-coordinate reference determined by group dimensionality. The figure analyzes raw Gaussian -OT eigenvectors in normalized action space.

Figure 9: Empirical characterization of expansion directions. (a) Within-task sorted OT eigenvalues: curves show task medians and bands show interquartile ranges; the dashed line marks

s_{\tau j}=1

. (b) Expansion fraction

k_{\tau}/D

versus covariance-cost share

R_{\mathrm{exp}}^{\tau}

. (c) Top-

k_{\tau}

subspace overlap, averaged over 20 random data splits with the source and full-data dimension fixed. (d) Symmetric overlap between the fitted-source and isotropic-source expansion subspaces, each selected by

s_{\tau j}>1

. Each point in (b–d) represents a task within a diagnostic setting. Black bars in (c,d) show task medians; grey markers show analytical random-subspace expectations. All spectra use the regularized covariance estimates specified above.

Integrator	NFE $=2$	NFE $=10$	NFE $=20$
Explicit Euler	$0.00\pm 0.00$	$35.67\pm 1.43$	$55.00\pm 2.16$
Explicit midpoint	$0.00\pm 0.00$	$57.67\pm 1.84$	$57.50\pm 2.48$
Schedule-aware (ours)	$\mathbf{68.33\pm 2.09}$	$\mathbf{72.17\pm 4.48}$	$\mathbf{72.67\pm 2.01}$

Appendix

Table 6: Solver ablation on LIBERO-10 with a fixed TOPS checkpoint. Success rate (%, higher is better), reported as mean

\pm

population standard deviation over three evaluation seeds. All rows use the same

M_{\tau}=3

checkpoint and differ only in the inference solver. Columns align the number of velocity-network evaluations (NFE), rather than the number of integration steps.

Hyperparameter	Value
TOPS warm-up / continuation epochs	$300\,/\,300$
Total training epochs, all methods	600
Batch size	32
Optimizer	AdamW, $\beta_{1}=0.95$ , $\beta_{2}=0.999$
Learning rate	$10^{-4}$
Weight decay	$10^{-6}$

Appendix

Table 7: Main hyperparameters for simulation experiments.

Hyperparameter	Value
Robot platform	Galaxea R1 Lite
Actuated degrees of freedom	Right arm (6) + gripper (1)
Control and recording rate	$15$ Hz
Cameras	Head-mounted and right wrist-mounted RGB
Image resolution	$256\times 256$ (pad to square, then resize)
Proprioception / action dimension	$7$ / $7$

Appendix

Table 8: Main hyperparameters for real-world experiments. Hyperparameter settings not listed here are identical to those in Tab. 7 .

Task	Method	1 step	5 steps	10 steps
Lift	Standard FM	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
	TOPS scheduling	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$
Can	Standard FM	$90.7\pm 5.0$	$92.7\pm 3.1$	$94.0\pm 0.0$
	TOPS scheduling	$\mathbf{95.3\pm 2.3}$	$\mathbf{95.3\pm 4.2}$	$\mathbf{96.0\pm 2.0}$
Square	Standard FM	$78.7\pm 3.1$	$80.7\pm 6.1$	$82.0\pm 6.9$
	TOPS scheduling	$\mathbf{81.3\pm 3.1}$	$\mathbf{86.7\pm 5.8}$	$\mathbf{88.0\pm 4.0}$

Appendix

Table 9: Single-task success rates (%) on Robomimic. TOPS scheduling retains the same Gaussian source as Standard FM. Results are mean

\pm

sample standard deviation across three evaluation seeds, each evaluated on the same 50 initial states, for one training seed. Both methods use two network evaluations per integration step.

	1 integration step (NFE = 2)		2 integration steps (NFE=4)
Schedule	Capture $\uparrow$	Mid-band $\downarrow$	Capture $\uparrow$	Mid-band $\downarrow$
Linear	24.87	22.39	84.16	0.89
Fast-then-slow	16.05	55.91	40.23	2.31
Slow-then-fast (ours)	34.10	12.93	98.57	0.73

Appendix

Table 10: Results on the two-dimensional bimodal generation task. All entries are percentages computed from the same 30,000 source samples reused across schedules. Fig. 7 displays the same results rounded to whole percentages.

Encoder	Task representation	Top-1 (%)	Top-3 (%)
CLIP	Canonical task names	32.7	66.8
CLIP	Leave-one-out prototypes	51.4	96.1
MiniLM	Leave-one-out prototypes	81.3	98.3
MiniLM	Learned classifier (out-of-fold)	98.6	100.0

Appendix

Table 11: Language-based task-selection accuracy over 358 distinct instruction formulations for 34 known tasks. Prototype evaluations exclude each query from its support set. The learned classifier is evaluated out of fold, with each instruction classified by a model trained on the remaining folds.

Figure 10: Empirical support for the slow-then-fast scheduling principle. (a) Gaussian -predicted versus empirically estimated slow-then-fast/linear ratios of time-integrated directional risk terms; each point is an expansion direction, colored by $\log_{10}s_{\tau j}$ . (b) Expansion-subspace prediction error of full-state MLP regressors, normalized by the linear baseline after summing over tasks. Gaussian -optimal optimizes the Gaussian bound on expansion directions. (c) Endpoint-speed ratios of numerical optima reconstructed from empirical conditional variances, with the theoretical relation shown as a dashed line. Panel (c) evaluates these reconstructed optima; the bounded TOPS schedule is evaluated in (a,b).

Flow-Based Policy Learning

See all 41 papers

Aug 18, 2026

Task-Prototype Guided Flow Matching for Few-Shot Generalization in Vision-Language Robot Manipulation

Vision-language robot manipulation policies can follow semantic instructions, but adapting them to a new procedure from only a few demonstrations remains difficult because language underspecifies contact timing, motion phases, corrective behavior, and execution style. This paper presents Task-Prototype Guided Flow Matching (TP-Flow), a few-shot manipulation framework that converts support demonstrations into structured task-prototype tokens and uses them to guide both the initial flow prior and the velocity field. TP-Flow employs symmetric cross-attention with learnable queries to extract phase-level prototypes, parameterizes a task-adaptive initial distribution, and injects prototype information through gated adaptive normalization. It is trained with an episodic support-query objective and prototype contrastive regularization, so few-shot adaptation is simulated during training while nuisance information is suppressed. On the LEROBOT-ARM-SO101 platform, TP-Flow achieves 66.8%, 79.6%, and 82.1% success rates under 1-, 4-, and 6-shot settings, with a 75.5% few-shot AUC. At 1-shot, it improves over CFM, Pooled-Demo CFM, and In-Context Flow by 29.8, 14.5, and 9.9 percentage points. It also improves held-out target-group generalization across novel-object transfer, goal recombination, long-horizon composition, and contact/correction tasks. TP-Flow maintains real-time execution with six online prototype tokens, 54.3 ms latency, 3.9 GB peak memory, and a 10 Hz control rate, while reducing the noisy-support success drop to 6.2%. Theoretical diagnostics show that prototype distance aligns with action-distribution distance, the adaptive prior reduces transport cost, and gated modulation keeps measured trajectory deviations below the derived ODE bound. The code repository is omitted for anonymous review.

Yizhao Wang, Guantao Zhang, Jingbo Wang

School of Computer Science, Henan Institute of Science and Technology

Generalization in Robotic Manipulation Robotic Manipulation Language-Conditioned Robot Manipulation Flow Matching VLMs for Robotics

Ask PDF

Oct 1, 2026

NarrativeFlow: Flow-Based Vision-Language-Action Model Using Robot Velocity Fields

We focus on language-conditioned flow-based manipulation, where robot flows (robot velocity fields) serve as embodiment-agnostic, motion-centric representations for leveraging data collected from multiple robot platforms. This task is crucial because language-conditioned manipulation is essential for practical robotic systems, yet scaling robot foundation models remains limited by the labor-intensive collection of embodiment-specific data. Existing methods either coarsely approximate robot flows with sparse keypoint displacements, or cannot handle language-conditioned manipulation. To address this limitation, we propose NarrativeFlow, which models robot flows as continuous velocity fields using a flow-matching formulation conditioned on language. Accordingly, NarrativeFlow generates robot flows that are physically consistent with real-world manipulation. To validate NarrativeFlow, we have conducted experiments on standard datasets for language-conditioned manipulation. The experimental results show that NarrativeFlow outperforms representative baseline methods on standard evaluation metrics. Furthermore, through real-world experiments, we show that NarrativeFlow achieves higher success rates than baseline methods across multiple manipulation tasks. The project page is available at https://shota0520.github.io/NarrativeFlow-project-page/

Shota Kobayashi, Koki Seno, Daichi Yashima +1

Keio University, Japan

Language-Conditioned Robot Manipulation Cross-Embodiment Robot Learning Robot Motion Generation Vision-Language-Action Models Motion Representation Learning

Ask PDF

Jun 19, 2026

VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation

Multi-task robot manipulation policies are challenging to learn from demonstration because traditionally a single network must select among qualitatively different action modes from a multimodal demonstration distribution, conditioned on language and visual context. A wrong mode selection means executing the wrong task or an action infeasible in the scene. Tokenizing continuous actions into a learned discrete codebook separates these modes at the representation level, offering structural advantages for multi-task learning. We propose VQActFlow, a multi-task manipulation policy that tokenizes action chunks and generates code sequences via Variational Flow Matching. VQActFlow maintains an explicit preference over action modes throughout generation. Inference-time guidance acts on this preference to steer mode commitment. We instantiate this with classifier-free guidance over language conditioning, which steers the policy toward the instructed action mode, and a learned codebook critic that supplies a complementary feasibility signal. We evaluate VQActFlow on three platforms: the LIBERO simulation benchmarks, a Unitree G1 humanoid performing whole-body pick-and-place, and an ALOHA-style bimanual platform performing contact-rich tasks. Across these benchmarks, VQActFlow outperforms both continuous and discrete baselines.

Zhigen Zhao, Mark Leggiero, Yipu Chen +5

Institute for Robotics and Intelligent Machines (IRIM), Georgia Institute of Technology, Atlanta, GA 30332

Latent Action Learning Language-Conditioned Robot Manipulation Multi-Task Robotic Manipulation Vector Quantization Robotic Manipulation Robot Imitation Learning

Ask PDF

Abstract

Flow-Based Policy Learning

Explore similar work

Task-Prototype Guided Flow Matching for Few-Shot Generalization in Vision-Language Robot Manipulation

NarrativeFlow: Flow-Based Vision-Language-Action Model Using Robot Velocity Fields

VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation