cs.ROSep 30, 2026

Cue the Flow: Steering Flow-Matching Policies for Open-World Delivery Manipulation

Authors: Haoxuan Wang, Griffin Galimi, Junhua Huang, Selina Song, Wayne Wu, Yan Yan, Bolei Zhou

Organizations: University of Illinois Chicago · University of California, Los Angeles

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Open-world goods delivery requires mobile manipulators to follow free-form user instructions and manipulate potentially novel objects. Existing dual-system approaches use high-level grounding models to convert language into grounded visual prompts, but their low-level controllers can remain brittle under noisy perception, dynamic scenes, and contact-rich interactions. We instead use a pretrained flow-matching vision-language-action model as the low-level control interface, leveraging its reactivity and robustness to environmental changes while treating the grounding output as a spatial cue for policy steering. Our key insight is that the pretrained VLA already provides a strong manipulation prior, while the spatial cue supplies the missing target information needed to guide actions under novel language--object mappings. Concretely, we introduce a lightweight cue-conditioned adapter. The adapter is first trained with contrastive objectives to produce salient and spatially discriminative cue representations, and is then supervised to predict a diagonal affine transformation over the generated action chunk, aligning policy steering with the cued target. Across tabletop and mobile-base settings, our method improves instruction following and manipulation success on both in-domain and out-of-domain objects, achieving up to near 2×2\times improvement in average task success rate with negligible inference overhead.

Figures & tables

Figure 1: (a.1) The policy is trained on in-context demonstrations collected in a fixed environment. (a.2) In real-world delivery, however, the robot must execute manipulation under changing environments and novel language–object mappings. (b) We focus on improving policy generalization to these novel mappings. A grounding module converts the open-ended instruction into a spatial cue, which conditions a contrastive adapter to predict affine transformations over the frozen flow-matching policy’s action output, steering the generated action chunk toward the target object.
Figure 2: Illustration of adapter learning.
Figure 3: Qualitative rollouts under the unseen-object with novel-language setting (Setting III). The leftmost column shows the System 2 spatial cue, where the red heatmap indicates the target object. For each method, two temporal keyframes are shown. The Base policy produces untargeted behavior, while VP-VLA either oscillates or reaches the wrong object. Our method follows the spatial cue and successfully manipulates the specified target in both task examples.
Figure 4: Extension to a complete delivery pipeline. We show example rollouts that combine manual navigation with autonomous manipulation. The red heatmap indicates the correctly grounded target object. After the mobile base is manually navigated to the workspace, our policy performs autonomous cue-conditioned manipulation to complete the task.
Figure 5
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Robot hardware setup.
Evaluation SettingPrompt Templates
Setting I & IIPick the <descriptor> bag and put it into the box Grab <descriptor> bag and place it in the box Take the <descriptor> bag and drop it into the box Move the <descriptor> bag into the box Put <descriptor> bag into the box Place the <descriptor> bag in the box
Setting IIIPut bag with order number <value> into the box Grab bag with number <value> into the box Pick order number <value> bag and place it into the box
Appendix
Table 3: List of prompts used for evaluation. For seen-object evaluations, <descriptor> refers to objects observed during training, i.e., “black” or “plastic.” For unseen-object evaluations, it refers to novel descriptors such as “brown” or “red.” In the novel-language setting, <value> denotes ordinal identifiers, i.e., 1, 2, or 3, used to evaluate compositional reference grounding under previously unseen instruction patterns.
Figure 8: Representative evaluation configurations across the three experimental settings. Rows correspond to the evaluation regimes described in the main paper: Setting I uses seen objects with paraphrased language, Setting II uses unseen objects with known language, and Setting III uses unseen objects with novel language. Columns show different spatial layouts used during evaluation, varying object placement, distractor arrangement, and scene geometry. In Setting I, Configurations 0–1 correspond to the black-bag task, while Configurations 2–3 correspond to the plastic-bag task. All methods are evaluated on the same prompts and spatial configurations.
Figure 9: Example demonstrations of policy steering across the three evaluation settings. The red heatmap indicates the correctly grounded target object.

Explore similar work

CardsList

  1. Guided Action Flow: Value-Guided Sampling for Frozen Vision-Language-Action Policies

    Jul 2, 2026Liuhaichen Yang, Zhuang Jiang, Chenchao Sheng +8Flow-Matching Vision-Language-ActionGuidance

    Department of Computer Science, University College London. · Department of Mechanical Engineering, University College London.

  2. World Pilot: Steering Vision-Language-Action Models with World-Action Priors

    Jun 10, 2026Zefu Lin, Rongxu Cui, Junjia Xu +4Efficient World-Action ModelWorld Models

    Institute of Automation, Chinese Academy of Sciences (CASIA) · Nanjing University · Beihang University

  3. Flow Control: Steering Vision-Language-Action Models with Simple Real-Time Inputs

    Jun 8, 2026Jonathan C. Kao, Jason Chan, Andy WangFlow-Matching Vision-Language-ActionVision-Language-Action Framework

    Departments of Electrical and Computer Engineering, Computer Science, and Neurobiology, University of California, Los Angeles · Department of Computer Science, University of California, Los Angeles