FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

Junhyeong Go; Mingyu Jung; Woobin Im; Jongbin Ryu

cs.ROOct 12, 2026

FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

Authors: Junhyeong Go, Mingyu Jung, Woobin Im, Jongbin Ryu

Organizations: Ajou University · Samsung Electronics

Abstract

Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.

Figures & tables

Figure 1: Overview of Training-only 3D flow supervision. Reference-anchored 3D points are tracked over time to construct 3D flow targets, from which top- $K$ motion-salient points are selected. Point-Level Flow predicts scene-level motion, while TCP-Level Flow aggregates it into end-effector motion, bridging scene-level dynamics and control-relevant motion.

Method	Spatial (%)	Object (%)	Goal (%)	Long (%)	Avg. (%)
Diffusion Policy (RSS’23)	78.3	92.5	68.3	50.5	72.4
OpenVLA (CoRL’24)	84.7	88.4	79.2	53.7	76.5
$\pi_{0}$ (RSS’25)	96.8	98.8	95.8	85.2	94.2
CoT-VLA (CVPR’25)	87.5	91.6	87.6	69.0	83.9
$\pi_{0}$ -FAST (RSS’25)	96.4	96.8	88.6	60.2	85.5
SpatialVLA (RSS’25)	88.2	89.9	78.6	55.5	78.1

Table 1: Results on LIBERO Benchmark. For Seer [ 26 ] , we follow the original LIBERO-Long evaluation protocol, averaging the top-3 checkpoints over 20 rollouts per task.

	Camera	Robot	Language	Light	Background	Noise	Layout	Avg. (%)
OpenVLA	0.8	3.5	23.0	8.1	34.8	15.2	28.5	15.6
WorldVLA	0.1	27.9	41.6	43.7	17.1	10.9	38.0	25.0
UniVLA	1.8	46.2	69.6	69.0	81.0	21.2	31.9	42.9
Seer	22.1	47.7	86.3	12.3	2.0	22.9	50.0	36.1
Seer + FlowVLA	24.6	54.6	87.0	25.1	8.1	25.5	57.2	41.1
$\pi_{0.5}$	21.4	39.1	68.4	86.7	83.3	36.1	84.6	57.5

Table 2: Zero-shot evaluation on LIBERO-Plus. Results demonstrate the robustness of policies trained on LIBERO benchmarks under unseen environmental perturbations.

Supervision Signal	Avg. (%)
None (Baseline)	80.4
2D optical flow (CoTracker [ 13 ] )	84.5
Static 3D point cloud	85.0
Dynamic 3D flow (Ours)	87.8

Table 4: Effect of supervision signal.

	Base Tasks				3D-aware Tasks
	Cup & Plush	Fruit	Stack Cups	Cabinet	Basketball	Hang Cup	Ring Insertion
Seer	60.0	66.7	73.3	60.0	60.0	46.7	66.7
Seer + FlowVLA	73.3	73.3	93.3	73.3	80.0	86.7	73.3

Table 5: Real-world results . We evaluate each task over 15 trials and report success rates (%).

Figure 3: Task visualization. All policies are trained with only 40 demonstrations per task.

Method	Viewpoint	Height
Seer	0	46.7
Seer + FlowVLA	46.7	73.3

Table 6: Test-time variation results.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: LIBERO-Plus Benchmark. LIBERO-Plus evaluates robustness under distribution shifts across seven perturbation categories: lighting, camera viewpoint, sensor noise, background, language instruction, scene layout, and robot initialization.

Top- $K$	1024	2048	3072
Success (%)	86.0	87.8	85.2

Appendix

Table 7: Ablation on the number of Top-

K

Method	Baseline	VGGT	SpatialTrackerV2	VDPM
Success (%)	80.4	86.8	88.0	87.8

Appendix

Table 8: Ablation on reconstruction models for 3D flow supervision.

Noise ratio (%)	0	5	20	Baseline
Success (%)	87.8	84.7	84.0	80.4

Appendix

Table 9: Noise sensitivity analysis.

Category	Simulation	Real-world
Evaluation Trials	20 per task	15 per task
Training Demonstrations	50 per task	40 per task
Test-time Randomization	-	position perturbations
Batch Size	512	512
Learning Rate	1e-3	1e-3
LR Schedule	cosine	cosine

Appendix

Table 10: Experimental protocol and hyperparameter configuration for Seer [ 26 ] .

Figure 6: Base task visualization. Overview of the four long-horizon base tasks used in our real-world experiments: Fruit, Cup & Plush, Stack Cups, and Cabinet.

Figure 7: 3D-aware task visualization. Overview of the three 3D-aware tasks used in our real-world experiments: Basketball, Ring Insertion, and Hang Cup.

Method	Success
FlowVLA	7/15
$-$ w/o coordinate alignment	4/15
$-$ w/o magnitude-based selection	0/15
$-$ w/o TCP-level loss	2/15
$-$ w/o point-level loss	4/15

Appendix

Table 11: Component analysis under viewpoint variation.

Method	Basketball	Hang Cup	Background
$\pi_{0.5}$	10/15	9/15	5/15
$\pi_{0.5}$ + FlowVLA	13/15	14/15	8/15

Appendix

Table 12: Real-world results with

\pi_{0.5}

. Evaluation on 3D-aware tasks and test-time dynamic-background variation.

Figure 8: 3D point composition analysis. Motion-based selection increases the proportion of object points, while TCP aggregation further reduces background points.

Figure 9: VDPM 3D point trajectory visualization on the real-world Fruit task.

Figure 10: VDPM 3D point trajectory visualization on the LIBERO-10 task.

Figure 11: Qualitative visualization of magnitude-based point selection.

Figure 12: Qualitative visualization of predicted TCP trajectory.

Figure 13: Qualitative visualization of TCP-level aggregation weights.

Figure 14: Failure case visualization on Stack Cups and Cabinet .

Figure 15: Failure case visualization on Basketball with viewpoint variation .

Figure 16: Failure case visualization on Ring Insertion with height variation .

Explore similar work

Mar 26, 2026

LaMP: Learning Vision-Language-Action Policy with 3D Scene Flow as Latent Motion Prior

We introduce LaMP, a dual-expert Vision-Language-Action framework that embeds dense 3D scene flow as a latent motion prior for robotic manipulation.Existing VLA models regress actions directly from 2D semantic visual features, forcing them to learn complex 3D physical interactions implicitly.This implicit learning strategy degrades under unfamiliar spatial dynamics.LaMP addresses this limitation by aligning a flow-matching Motion Expert with a policy-predicting Action Expert through gated cross-attention.Specifically, the Motion Expert generates a one-step partially denoised 3D scene flow, and its hidden states condition the Action Expert without full multi-step reconstruction.We evaluate LaMP on the LIBERO, LIBERO-Plus, and SimplerEnv-WidowX simulation benchmarks as well as real-world experiments.LaMP consistently outperforms evaluated VLA baselines across LIBERO, LIBERO-Plus, and SimplerEnv-WidowX benchmarks, achieving the highest reported average success rates under the same training budgets. On LIBERO-Plus OOD perturbations, LaMP shows improved robustness with an average 9.7% gain over the strongest prior baseline.Our project page is available at https://summerwxk.github.io/lamp-project-page/.

Xinkai Wang, Chenyi Wang, Yifu Xu +7

Southeast University, China · Shanghai Innovation Institute, China · Shanghai Jiao Tong University, China +1

Language-Conditioned Robot Manipulation Vision-Language-Action Models

Ask PDF

Jul 7, 2026

Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation

Recently, Vision-Language-Action (VLA) models have demonstrated strong generalization across diverse tasks. However, effective robotic manipulation in physical environments fundamentally requires geometric understanding and spatial reasoning. While some VLA approaches attempt to incorporate 3D information, they are constrained by limited data availability and geometric information loss in current 3D encoding pipelines, and fail to jointly capture 3D geometry and temporally structured actions in dynamic environments. To address these limitations, we introduce Lift3D-VLA, a unified VLA framework that equips models with explicit 3D point cloud reasoning and enables temporally coherent action generation. First, building upon our previous work Lift3D, an enhanced 2D model-lifting strategy is proposed to geometrically align 3D points with pretrained 2D positional embeddings. This design enables direct point-cloud encoding within the VLA vision encoder while minimizing spatial information loss. Based on explicit 3D inputs, we propose Geometry-Centric Masked Autoencoding (GC-MAE), a dual-objective self-supervised framework that reconstructs the current point cloud while predicting its future geometric evolution. This formulation allows the 2D vision encoder to internalize both 3D structure and physical dynamics. To fully exploit 3D representations, we further design layer-wise temporal action modeling, which leverages multiple layers of the LLM to collaboratively predict action chunks, enabling temporally consistent predictions. Across 22 simulated tasks and 8 real-world manipulation tasks, Lift3D-VLA achieves 10.8% and 11.1% higher mean success rates on MetaWorld and RLBench than the best-performing prior VLA methods, and outperforms the strongest real-world baseline by 4 percentage points, while exhibiting stronger generalization to out-of-distribution perturbations.

Jiaming Liu, Qingpo Wuwu, Nuowei Han +8

State Key Laboratory of Multimedia Information Processing and National Engineering Research Center of Visual Technology, School of Computer Science, Peking University, Beijing, China · CUHK, Shatin, Hong Kong · AI2Robotics, Beijing, China

Robotic Manipulation Vision-Language-Action Models 3D Representation Learning

Ask PDF

Sep 12, 2026

GeomVLA: Unifying Scene, Motion, and Action in 3D

We present GeomVLA, a Vision-Language-Action (VLA) model that unifies perception, latent scene motion prediction, and action generation within a shared robot-centric 3D coordinate frame. Our approach lifts pretrained VLM features into spatially grounded 3D scene tokens using depth and camera calibration, while retaining the semantic representations learned during VLM pretraining. We further introduce a 3D Scene Trajectory Denoiser, a task-conditioned module that learns a latent representation of how scene points are expected to move in 3D. Rather than executing the predicted trajectory as an open-loop plan, GeomVLA extracts intermediate motion tokens from the trajectory denoiser and uses them to condition a 3D flow-based action denoiser through geometry-aware attention. GeomVLA achieves state-of-the-art performance on CALVIN, competitive performance on LIBERO and RoboTwin2.0, and outperforms strong baselines in real-world manipulation settings without robot-action pretraining. Extensive ablations show that future-motion reasoning alone is insufficient: the primary gains are associated with maintaining geometric consistency among scene representation, motion prediction, and robot actions throughout the perception-to-action pipeline.

Ziyin Xiong, Nikolaos Gkanatsios, Moritz Reuss +1

Carnegie Mellon University · NVIDIA

Robotic Manipulation 3D Scene Representation Vision-Language-Action Models VLMs for Robotics

Ask PDF