FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Model
Authors: Xiaoxu Xu, Hao Li, Jinhui Ye, Yilun Chen, Jia Zeng, Xinyi Chen, Linning Xu, Dahua Lin, +2 more
Organizations: Beihang University. · Shanghai Artificial Intelligence Laboratory. · University of Science and Technology of China. · The Chinese University of Hong Kong.
Predictive foresight is important to intelligent embodied agents. Since the motor execution of a robot is intrinsically constrained by its visual perception of environmental geometry, effectively anticipating the future requires capturing this tightly coupled visuomotor interplay. While recent vision-language-action models attempt to incorporate future guidance, they struggle with this joint modeling. Existing explicit methods divert capacity to task-irrelevant visual details, whereas implicit methods relying on sparse frame pairs disrupt temporal continuity. By heavily relying on visual reconstruction, these methods become visually dominated, entangling static scene context with dynamic action intent. We argue that effective joint visuomotor predictive modeling requires both temporal continuity and visually-conditioned supervision decoupling. To this end, we propose FutureVLA, featuring a novel Joint Visuomotor Predictive Architecture. FutureVLA is designed to extract joint visuomotor embeddings by first decoupling visual and motor information, and then jointly encoding generalized physical priors. Specifically, in the pretraining stage, we leverage heterogeneous manipulation datasets and introduce a Joint Visuomotor Gating mechanism to structurally separate visual state preservation from temporal action modeling. It allows the motor stream to focus on continuous physical dynamics while explicitly querying visual tokens for environmental constraints, yielding highly generalizable joint visuomotor embeddings. Subsequently, in the post-training stage, we employ a latent embeddings alignment strategy, enabling diverse downstream VLA models to internalize these temporal priors without modifying their inference architectures. Extensive experiments demonstrate that FutureVLA consistently improves VLA frameworks.
Figures & tables
Figure 1: Comparison of future guidance paradigms for VLA models. (a) Explicit guidance predicts future video frames. (b) Implicit guidance learns latent vectors to reconstruct changes between sparsely sampled frames. (c) Ours processes continuous multi-frame clips and structurally decouples the latent representation into a visual stream and a motor stream. This visually-conditioned decoupling extracts joint visuomotor embeddings, yielding consistent performance improvements across diverse benchmarks.
Figure 2: Overview of the FutureVLA framework. (a) Joint Visuomotor Pretraining: Continuous video clips are processed by a frozen 3D-VAE into temporal tokens and structurally decoupled into two streams. Visual tokens reconstruct the initial frame Ot , while motor tokens, supervised by action chunks, utilize the Joint Visuomotor Gating module (c) based on gated cross-attention, where the motor stream iteratively queries spatial affordances from the visual tokens, yielding physically grounded joint visuomotor embeddings. (b) Joint Visuomotor Embedding Guided VLA Post-training: The frozen model provides joint visuomotor embeddings as future-aware temporal priors. Through latent embedding alignment, the downstream VLA’s intermediate representations are forced to internalize these dynamics.
Settings
Methods
Pick Coke Can
Move Near
Open/Close Drawer
Put in Drawer
Avg.
Visual Matching
OpenVLA-OFT [ 25 ]
72.3
69.6
47.2
0.9
47.5
π0 [ 4 ]
87.3
35.0
72.6
16.0
52.7
GR00T-N1.5 [ 3 ]
51.7
54.0
27.8
7.4
35.2
Villa-X [ 12 ]
98.7
75.0
59.3
5.6
59.6
FutureVLA-GT
92.3
74.2
68.5
85.2
80.1
FutureVLA-OT
97.0
83.8
55.6
74.1
77.6
Table 1: Comparison of different VLA models across four tasks in two SimplerEnv settings on the Google robot. “GT” indicates GR00T-style model, “OT” denotes OFT-style model.
Methods
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
OpenVLA-OFT [ 25 ]
12.5
4.2
4.2
100.0
30.2
π0 [ 4 ]
25.0
16.7
12.5
29.2
20.9
GR00T-N1.5 [ 3 ]
75.3
54.3
57.0
61.3
61.9
UniVLA [ 9 ]
52.8
55.6
2.8
80.6
47.9
Villa-X [ 12 ]
48.3
24.2
19.2
71.7
40.8
FutureVLA-GT
83.3
75.0
29.2
100.0
71.9
Table 2: Comparison of different VLA models across four tasks in the SimplerEnv setting on the WidowX robot.
Methods
Object
Spatial
Goal
Long
Avg.
OpenVLA-OFT [ 25 ]
88.4
84.7
79.2
53.7
76.5
π0 [ 4 ]
98.8
96.8
95.8
85.2
94.2
GR00T-N1.5 [ 3 ]
97.6
94.4
93.0
90.6
93.9
WorldVLA [ 11 ]
96.2
87.6
83.4
60.0
81.8
DreamVLA [ 49 ]
94.0
97.5
89.5
89.5
92.6
UniVLA [ 9 ]
96.8
96.5
95.6
92.0
95.2
Table 3: Comparison of different VLA models on the LIBERO benchmark.
Figure 3: Visualization in (a) and performance comparison in (b) for four real-robot manipulation tasks. In (a), from top to bottom: (1) Make a burger : The robot first picks up the burger slice on the right and places it on the bread, then picks up the bread slice on the left and places it on top of the burger slice. (2) Insert roses into a pot : The robot picks up the roses from the table and inserts them into the pot. (3) Scoop beans with a spoon : The robot grasps the spoon in the center, scoops beans from the left container, and transfers them into the bowl on the right. (4) Erase handwriting on a whiteboard : The robot erases the black markings on the whiteboard.
Methods
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
FutureVLA-GT wo/ JVPM
79.2
58.3
20.8
91.7
62.5
FutureVLA-GT w/ JVPM
83.3
75.0
29.2
100.0
71.9
FutureVLA-OT wo/ JVPM
66.7
58.3
12.5
79.2
54.2
FutureVLA-OT w/ JVPM
75.0
54.2
25.0
100.0
63.6
Table 4: Ablation of Joint Visuomotor Predictive Modeling guidance on the WidowX robot. “w/ JVPM” and “wo/ JVPM” denote whether latent embedding alignment is applied during post-training.
MT
VT
JVG
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
(a)
79.2
58.3
20.8
91.7
62.5
(b)
✓
66.7
54.2
16.7
95.8
58.4
(c)
✓
✓
66.7
62.5
37.5
95.8
65.6
(d)
✓
✓
✓
83.3
75.0
29.2
100.0
71.9
Table 5: Ablation studies of Joint Visuomotor Gate on the WidowX robot. “MT” represents motor tokens, “VT” indicates visual tokens and “JVG” denotes Joint Visuomotor Gating.
Figure 9
Figure 6: Performance comparison of different future guidance styles on the WidowX robot under a unified training framework. Explicit future guidance is evaluated using the WorldVLA-style, while implicit future guidance is evaluated using two representative paradigms: LAPA-style and Villa-X-style.
Figure 7: Visualization of video clips associated with similar latent embeddings. In (a), both clips exhibit a grasping action followed by a front-to-back translation and downward movement. In (b), both clips show a grasping action with a upper-left translation.
Figure 8: Probability density distributions of LAPA-style, Villa-X-style, and our method over the similarity between latent embeddings and executed action chunks. Vertical lines denote the corresponding distribution means.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: The composition of datasets used for Joint Visuomotor Pretraining.
Methods
Cam.
Rob.
Lang.
Light
Back.
Noise
Layout
Total
OpenVLA [ 26 ]
0.8
3.5
23.0
8.1
34.8
15.2
28.5
15.6
OpenVLA-OFT [ 25 ]
56.4
31.9
79.5
88.7
93.3
75.8
74.2
69.6
π0 [ 4 ]
13.8
6.0
58.8
85.0
81.4
79.0
68.9
53.6
WorldVLA [ 11 ]
0.1
27.9
41.6
43.7
17.1
10.9
38.0
25.0
UniVLA [ 9 ]
1.8
46.2
69.6
69.0
81.0
21.2
31.9
43.9
FutureVLA-GT
59.7
66.0
88.2
97.4
97.8
77.3
82.6
79.7
Appendix
Table 6: Comparison of different VLA models on the LIBERO-Plus benchmark. For simplicity, we abbreviate Camera/Robot/Language/Background as Cam./Rob./Lang./Back., respectively. “GT” indicates GR00T-style model, “OT” denotes OFT-style model.
Settings
Methods
Pick Coke Can
Move Near
Open/Close Drawer
Put in Drawer
Avg.
Visual Matching
FutureVLA-GT wo/ JVPM
97.6
69.1
66.2
60.1
73.2
FutureVLA-GT w/ JVPM
92.3
74.2
68.5
85.2
80.1
FutureVLA-OT wo/ JVPM
93.0
70.4
54.2
29.6
61.8
FutureVLA-OT w/ JVPM
97.0
83.8
55.6
74.1
77.6
Variant Aggregation
FutureVLA-GT wo/ JVPM
92.7
68.1
61.9
62.4
71.2
FutureVLA-GT w/ JVPM
95.7
72.9
52.4
81.2
75.6
Appendix
Table 7: Ablation of JVPM guidance on the Google robot.
Methods
Object
Spatial
Goal
Long
Avg.
FutureVLA-GT wo/JVPM
98.6
98.0
97.0
92.0
96.4
FutureVLA-GT w/JVPM
99.8
98.8
98.6
96.0
98.3
FutureVLA-OT wo/JVPM
98.4
98.0
96.6
93.6
96.6
FutureVLA-OT w/JVPM
99.2
99.6
99.2
94.8
98.2
Appendix
Table 8: Ablation of JVPM guidance on the LIBERO.
Methods
Cam.
Rob.
Lang.
Light
Back.
Noise
Layout
Total
FutureVLA-GT wo/ JVPM
48.8
60.3
87.1
95.5
95.2
75.1
78.3
75.4
FutureVLA-GT w/ JVPM
59.7
66.0
88.2
97.4
97.8
77.3
82.6
79.7
FutureVLA-OT wo/ JVPM
47.1
59.8
87.3
95.9
93.9
73.2
78.9
74.8
FutureVLA-OT w/ JVPM
50.4
68.8
90.0
97.9
95.4
72.6
83.9
78.2
Appendix
Table 9: Ablation of JVPM guidance on the LIBERO-Plus.
Methods
Make Burger
Insert Rose
Scoop Beans
Eraser Handwriting
Avg.
FutureVLA-GT wo/ JVPM
80.0
33.3
46.7
33.3
48.3
FutureVLA-GT w/ JVPM
100.0
46.7
60.0
73.3
70.0
FutureVLA-OT wo/ JVPM
40.0
26.7
40.0
26.7
33.3
FutureVLA-OT w/ JVPM
60.0
40.0
60.0
46.7
51.7
Appendix
Table 10: Ablation of JVPM guidance on real-world robotic tasks.
λ
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
0.5
70.8
75.0
29.2
95.8
67.7
1.0
83.3
75.0
29.2
100.0
71.9
1.5
79.2
70.8
29.2
100.0
69.8
Appendix
Table 11: Impact of the reconstruction loss weight λ on the WidowX robot.
VQ-GAN
3D-VAE
Cost time (s)
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
✓
3.53
79.2
62.5
20.8
95.8
64.6
✓
3.28
83.3
75.0
29.2
100.0
71.9
Appendix
Table 12: Effect of different video encoding strategies on performance and computational efficiency. “Cost time” denotes the average training time per step.
Reconstruction Target
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
Last-frame
75.0
75.0
29.2
95.8
68.8
First-frame
83.3
75.0
29.2
100.0
71.9
Appendix
Table 13: Impact of different reconstruction targets on the WidowX robot.
Figure 10: Visualization of visual perturbations applied to video frames. The left shows the original frame, and the right illustrates different types of data perturbations.
Methods
LAPA-style
Villa-X-style
Ours
Embedding MSE
0.3047
0.0188
0.0054
Origin Action MSE
0.0691
0.0399
0.0371
Noise Action MSE
0.0752
0.0435
0.0398
Action Bias
0.0061
0.0036
0.0027
Appendix
Table 14: Performance comparison of LAPA-style, Villa-X-style, and our method under visual perturbations. “Embedding MSE” measures the discrepancy between latent embeddings before and after noise injection. “Origin Action MSE” and “Noise Action MSE” denote the error between predicted actions and ground-truth actions under clean and perturbed observations, respectively. “Action Bias” quantifies the action deviation induced by visual perturbations.
Frame number
Selected frame indices
2
1, 1, 17, 17, 17
5
1, 6, 10, 14, 17
9
1, 3, 5, 7, 9, 11, 13, 15, 17
17
1–17
Appendix
Table 15: Frame index selection strategy under different frame numbers.
Vision-Language-Action (VLA) models have achieved impressive results in visuomotor policy learning, yet remain fundamentally reactive, mapping current observations and language to actions without explicit forward prediction of world dynamics. Existing visual foresight methods predict future visual states but lack explicit motion guidance: they show where to go but not how to get there. We argue that future feature prediction and sparse point tracking are naturally complementary: the former provides the goal state, while the latter captures the continuous motion path toward it. We propose FoMoVLA, a framework that augments VLA representations with explicit spatio-temporal supervision by jointly learning future feature foresight and sparse 2D point tracking, enhancing the continuous action policy. FoMoVLA introduces compact foresight tokens to decode future feature states, decodes sparse temporal 2D point trajectories to model compact geometric motion, and couples both through a lightweight future-conditioned cross-attention module that enables consistent reasoning between anticipated states and point dynamics. Extensive experiments on LIBERO, RoboCasa GR-1 Tabletop, and LIBERO-Plus demonstrate state-of-the-art performance and strong zero-shot generalization. Project page is available at https://liauto-research.github.io/FoMoVLA.
Wei Li, Peijin Jia, Yuan Ma +9
LiAuto · University of Chinese Academy of Sciences · Tsinghua University
Vision-Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared global action features may fail to establish timestep-specific correspondence between actions and local visual changes. To address this issue, we propose MotionWeave, a motion-centric future-dynamics framework for action-chunk prediction with two modules: the Action-Induced Motion Grounder (AIMG) and the Horizon Residual Composer (HRC). Specifically, AIMG conditions on action and proprioceptive representations to construct horizon-specific queries that localize interaction regions associated with each future action timestep from current visual tokens. HRC extracts differences between interaction representations at adjacent horizons, encodes them as temporal motion cues, and injects them into action tokens through a gated residual. During training, robot-arm masks rendered from future frames are used to construct KL-based motion-grounding supervision, while inference uses only the current observation. On six MetaWorld tasks, MotionWeave achieves a 75.3% average success rate, an absolute gain of 8.6% over π0 (66.7%), especially on sustained-interaction tasks. Our code is available at https://github.com/autu-mn/MotionWeave.
Jingqiu Wang, Yan Wang
School of Data Science and Engineering East China Normal University, Shanghai, China
Vision-language-action (VLA) models provide a unified framework for visual understanding, language grounding, and robot control, yet anticipating how the scene will evolve beyond the current observation can provide additional information for control. Action supervision alone does not explicitly encourage such anticipation, while future-image prediction entangles task-relevant motion with largely static appearance. To address these limitations, we introduce Evo-World, a lightweight VLA model that augments policy learning with future motion prediction through optical-flow supervision, thereby providing a task-relevant predictive signal focused on motion rather than appearance. We further design the training procedure to preserve these predictive representations during policy learning, regulating representation diversity while decoupling representation learning from action optimization. Across simulation benchmarks, Evo-World demonstrates strong performance, achieving a tier-averaged success rate of 91.1% on Meta-World MT50 , with a 13.0% relative improvement over the non-predictive baseline. On real-world AgileX robots, Evo-World outperforms the evaluated baselines across single-arm and bimanual manipulation tasks while using only 0.87B parameters and delivering lower GPU memory usage and higher inference efficiency. These results demonstrate that future motion prediction through optical-flow supervision provides an effective and lightweight approach to enhancing the predictive capabilities of VLA models.
Zile Zhong, Yuxin Du, Tao Lin +4
School of AI, Shanghai Jiao Tong University · Zhejiang University · University of Electronic Science and Technology of China +1