Organizations: Zhejiang University · DEEP Robotics · Shanghai Artificial Intelligence Laboratory · Northeastern University, China · Beijing Zhongguancun Academy
During steady pouring, images and robot pose may remain nearly unchanged while transferred mass accumulates, making the return decision ambiguous for a whole-task vision-language-action (VLA) policy. We present FlowVLA, which separates motor generation from physical-completion estimation. A shared π0.5 action expert learns approach, pour, and return with boundary-aware supervision. A target-independent estimator fuses visual, joint, wrist-wrench, and recurrent force-belief histories to maintain an estimate of transferred mass. The controller initiates return when this estimate plus predicted residual outflow reaches the requested mass. In coffee-pot/water experiments, FlowVLA achieves 13.1 g goal-balanced mean absolute error (MacroMAE), lower than all tested VLA baselines. Across all attempts, 53.7% both complete the task and finish within +/-10% of the requested mass. The same model interpolates to held-out 150/250-g targets without retraining. Independently trained models further evaluate the formulation in three additional container-fluid settings.
Figures & tables
Fig. 1: Quantitative robotic pouring task. The robot performs grasping, pouring, return, and placement while targeting a requested transferred mass without online scale feedback.
Fig. 2: The architecture of FlowVLA. The action backbone fuses 30-Hz images and robot state with 500-Hz force history and a 20-Hz recurrent belief. The physical-completion module estimates transferred mass and return tail, whose sum determines when to return.
Fig. 3: Left: views from head and wrist cameras during approach, grasp, steady pouring, return, and placement. Right: (a) coffee pot, (b) top-handle kettle, (c) soy-sauce dispenser, and (d) oil dispenser.
Fig. 4: Flow-labeled demonstration datasets for the four domains (1,684 episodes total). Left: episode mean flow versus duration; diamonds are task means and error bars span the central 95% episode ranges. Right: duration distributions; thick lines mark interquartile ranges.
Method
Grasp
Post-grasp success
E2E
Nm
Nmass (100/200/300)
Per-goal MAE (g)
Pooled
Macro
≤10%
E2E+ ±10%
100/200/300
Trajectory-replay controls
TR-LT-ID
100.0%
100.0%
100.0%
37
37 (12/15/10)
66.86/95.48/124.33
94.0
95.6
5.4%
5.4%
TR-LT-Cal
100.0%
100.0%
100.0%
38
38 (13/12/13)
13.40/21.17/25.04
19.8
19.9
57.9%
57.9%
External baselines
π0.5
65.2%
33.3%
21.7%
45
15 (7/6/2)
69.34/54.97/75.40
64.4
66.6
13.3%
2.9%
TABLE I: Primary coffee-pot/water results at 100/200/300 g. TR-LT replays held-out paths in their original layouts; learned policies use randomized initial source positions (Sec. IV-A ).
Fig. 5: Automatic 300-g trial. (a) Predicted terminal mass using fixed and learned tails, offline electronic-scale reference, and frames at onset, mid-pour, and return. (b) Reconstructed flow and completion-head prediction. (c) Return decision.
Component
Variant
Na
Nm
Nmass 100/200/300
Per-goal MAE (g) 100/200/300
Pooled
Bias
Macro
Grasp
Post-grasp success
E2E
Full
Mt+Tt≥g
123
84
79 (28/26/25)
6.96/13.81/18.66
12.92
+2.65
13.15
68.3%
94.0%
64.2%
Action
No boundary-aware supervision
59
39
33 (13/11/9)
13.39/18.57/28.84
19.33
+12.16
20.27
66.1%
84.6%
55.9%
State
− hard-zero gate
53
37
35 (12/12/11)
12.47/13.46/24.82
16.69
−5.73
16.92
69.8%
94.6%
66.0%
Trigger
Mt≥g
61
40
39 (13/12/14)
10.18/20.11/26.89
19.24
+6.93
19.06
65.6%
97.5%
63.9%
Mt+0.2ft≥g
59
40
38 (12/13/13)
7.26/13.86/26.24
16.01
+2.74
15.79
67.8%
95.0%
64.4%
TABLE II: System-level mechanism ablations: mass accuracy and manipulation success.
Container / fluid
Goals (g)
Na
Nm
Nmass
MAE (g)
Macro Rel.
Grasp
Post-grasp
E2E
E2E+ ±10%
MAE
success
Coffee pot / water
150/250
31
22
10/10
10.94/19.71
7.6%
71.0%
90.9%
64.5%
54.8%
Top-handle kettle / water
200/400
32
25
12/12
34.54/50.73
15.0%
78.1%
96.0%
75.0%
25.0%
Soy-sauce dispenser / saturated brine
100
31
18
11
9.05
9.1%
58.1%
61.1%
35.5%
25.8%
Oil dispenser / cooking oil
50/100
47
30
14/13
4.23/11.86
10.2%
63.8%
90.0%
57.4%
31.9%
TABLE III: FlowVLA on held-out targets and additional container–fluid tasks. Held-out targets use the unchanged primary model; additional tasks use independently trained domain-specific models.
Fig. 6: Median flow (lines) and central 95% episode ranges (shading), with time normalized from onset (0%) to cutoff (100%). Peak P95 is the 95th percentile of episode peak flow.
We introduce flow control of vision-language-action (VLA) models, a simple and effective way to steer VLA actions in real time through generic inputs, such as a keyboard. This method can be used out-of-the-box and does not require retraining or fine-tuning VLAs. It enables relatively crude user inputs to steer a VLA to align with user intent. Flow control writes the user input into the initial condition of the VLA's flow matching action expert. Because the flow is deterministic, integration retains information about the user input while moving it toward the action distribution learned during training, so that the generated actions reflect the user's intent (high fidelity) and stay close to the VLA's own unsteered actions (high conformity). We demonstrate that flow control has many desirable properties: (1) flow control accurately and responsively steers robot actions with user inputs, (2) it is robust to suboptimal user inputs, (3) it enables users to steer VLAs to achieve significantly higher success rates and faster task completion, and (4) fine-tuning a VLA on flow control trajectories improves the autonomous policy. Together, these results provide a simple and intuitive way for users to help steer VLA actions, increasing task performance. Project page and videos: https://jcykao.github.io/flowcontrol/.
Jonathan C. Kao, Jason Chan, Andy Wang
Departments of Electrical and Computer Engineering, Computer Science, and Neurobiology, University of California, Los Angeles · Department of Computer Science, University of California, Los Angeles
Vision-language-action (VLA) models leverage pretrained vision-language representations for robot control, yet simply adding historical frames does not reliably capture recent physical change. This is especially problematic in multi-stage manipulation, where visually similar states may require different actions depending on prior execution. To address this challenge, we present TemporalFlow-VLA, which learns compact execution history through physically grounded temporal supervision. Using recorded robot states, robot geometry, and calibrated cameras, we construct robot-surface temporal flow as a training-only target and supervise two execution-aligned temporal queries that provide structured history to the action expert. The geometric supervision path is not evaluated at deployment. TemporalFlow-VLA achieves 97.63 +/- 0.26% average success on LIBERO, including 96.60 +/- 0.87% on LIBERO Long, and 85.5%/84.2% Clean/Randomized success across 12 RoboTwin tasks. It shows its clearest advantage over prior methods on longer-horizon, multi-stage manipulation. Controlled history interventions show that action prediction depends on both historical content and temporal order. With asynchronous feature caching, temporal conditioning maintains single-frame-level server-side sampling latency without additional historical-encoding overhead. Overall, TemporalFlow-VLA provides a compact, physically grounded interface for exploiting ordered execution history without explicit motion estimation or geometric processing at deployment.
Jiarui Yang, Yehao Lu, Yuning Su +9
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Zhejiang University, Hangzhou, China. · Simon Fraser University, Burnaby, BC, Canada. +1
Vision-language-action models (VLAs) combine vision-language backbones with expressive generative action heads trained via flow matching on large-scale robotic datasets. Despite their strong empirical performance in robotic manipulation, VLAs lack mechanisms to quantify confidence in their predictions and to detect when their actions may be unreliable. This presents a critical limitation for real-world deployment in non-stationary environments, where models inevitably encounter scenarios outside their pretraining distribution and may fail without warning. To address this, we derive an efficient method for quantifying epistemic uncertainty in flow-matching models by leveraging velocity-field disagreement (VFD) across a small ensemble. We successfully use this uncertainty estimate for failure detection during deployment and active fine-tuning of flow-based VLAs. To this end, we propose SAVE, a framework for uncertainty-guided active multitask fine-tuning that reduces the number of costly expert demonstrations required to adapt VLAs to new tasks. Through extensive experiments on the LIBERO benchmark, we demonstrate that VFD yields better-calibrated uncertainty estimates predictive of downstream performance, that VFD achieves strong performance in detecting failures, and that uncertainty-guided data acquisition with SAVE requires at least 22% fewer samples than baselines. In summary, our work shows that quantifying epistemic uncertainty in flow-based VLAs improves both failure awareness and adaptation. Project website: tum-lsy.github.io/uq_vla/.