MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies

Jingqiu Wang; Yan Wang

cs.ROSep 30, 2026

MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies

Authors: Jingqiu Wang, Yan Wang

Organizations: School of Data Science and Engineering East China Normal University, Shanghai, China

Abstract

Vision-Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared global action features may fail to establish timestep-specific correspondence between actions and local visual changes. To address this issue, we propose MotionWeave, a motion-centric future-dynamics framework for action-chunk prediction with two modules: the Action-Induced Motion Grounder (AIMG) and the Horizon Residual Composer (HRC). Specifically, AIMG conditions on action and proprioceptive representations to construct horizon-specific queries that localize interaction regions associated with each future action timestep from current visual tokens. HRC extracts differences between interaction representations at adjacent horizons, encodes them as temporal motion cues, and injects them into action tokens through a gated residual. During training, robot-arm masks rendered from future frames are used to construct KL-based motion-grounding supervision, while inference uses only the current observation. On six MetaWorld tasks, MotionWeave achieves a 75.3% average success rate, an absolute gain of 8.6% over π0 (66.7%), especially on sustained-interaction tasks. Our code is available at https://github.com/autu-mn/MotionWeave.

Figures & tables

Figure 1: Overview of MotionWeave. The current observation $I_{t}$ and instruction $l$ are tokenized into visual tokens and text tokens, concatenated with a learnable action query placeholder $A_{0}$ , and jointly processed by LLM to produce action tokens $A$ ; a separate proprioceptive encoder maps $S_{t}$ to $P$ . The Action-Induced Motion Grounder (AIMG) combines $A$ and $P$ into horizon-specific queries that attend over visual tokens $V$ to produce interaction tokens $M$ with spatial attention $\alpha$ . The Horizon Residual Composer (HRC) injects adjacent-horizon differences of $M$ into $A$ via gate $g$ to obtain $A^{\prime}$ , decoded by Action DiT into a 4-step chunk. Robot-arm masks [ 8 ] supervise $\alpha$ via KL loss in training only.

Figure 2: Qualitative rollouts on six tasks. Each row shows key frames from observation to successful completion.

Method	Source	Pick Place	Disassemble	Stick Pull	Assembly	Shelf Place	Hand Insert	Avg.
$\pi_{0}$ [ 1 ]	RSS’25	72.0	52.0	68.0	76.0	64.0	68.0	66.7
DreamVLA [ 22 ]	NeurIPS’25	60.0	68.0	72.0	70.0	56.0	64.0	65.0
WoG [ 16 ]	ICML’26	28.0	60.0	68.0	24.0	64.0	60.0	50.7
Fast-WAM [ 21 ]	arXiv’26	44.0	56.0	16.0	64.0	56.0	60.0	49.3
MotionWeave (ours)	—	60.0	92.0	76.0	80.0	76.0	68.0	75.3

Table 1: Success rates (%) on six MetaWorld tasks. All methods share the evaluation random seeds, and bold denotes the best result for each task. The “Source” column reports the publication status of each baseline.

Variant	Avg.
Baseline	58.0
+ AIMG w/o $\mathcal{L}_{\mathrm{mot}}$	62.0
+ AIMG	70.0
+ AIMG + HRC	75.3

Table 2: Component ablation results for MotionWeave. We report macro-average success (%) over the six tasks. Each row adds one component relative to the preceding configuration.

Figure 3: Hyperparameter curves in average success (%). (a) Horizon query count peaks at 4 with 75.3%, aligned with the action chunk length

H=4

, falling back to 73.3%/70.0% at 6/8 queries. (b) Motion loss weight

\lambda_{\mathrm{motion}}

peaks at 0.05 with 75.3%, falling back to 72.7%/68.7% at 0.10/0.20.

Explore similar work

FoMoVLA: Bridging Visual Foresight and Motion Guidance for Vision-Language-Action Models
Jul 16, 2026Wei Li, Peijin Jia, Yuan Ma +9Photometric Supervision Foresight
LiAuto · University of Chinese Academy of Sciences · Tsinghua University
ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
Oct 6, 2026Yuan Xu, Yixiang Chen, Qisen Ma +10
CASIA · UCAS · Alibaba Group +1
Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation
Jun 1, 2026Shahram Najam Syed, Arthur Jakobsson, Haoran Hao +1Future Video Prediction Latent World Models
Robotics Institute, Carnegie Mellon University, Pittsburgh, USA

Jul 16, 2026, cs.CVJ/K move · Enter open · S save

FoMoVLA: Bridging Visual Foresight and Motion Guidance for Vision-Language-Action Models

Wei Li, Peijin Jia, Yuan Ma, Xuefeng Jiang+8

LiAuto · University of Chinese Academy of Sciences · Tsinghua University

Photometric Supervision Foresight Visuomotor Policy Geometric Guidance Spatial Supervision Cross-Attention

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have achieved impressive results in visuomotor policy learning, yet remain fundamentally reactive, mapping current observations and language to actions without explicit forward prediction of world dynamics. Existing visual foresight methods predict future visual states but lack explicit motion guidance: they show where to go but not how to get there. We argue that future feature prediction and sparse point tracking are naturally complementary: the former provides the goal state, while the latter captures the continuous motion path toward it. We propose FoMoVLA, a framework that augments VLA representations with explicit spatio-temporal supervision by jointly learning future feature foresight and sparse 2D point tracking, enhancing the continuous action policy. FoMoVLA introduces compact foresight tokens to decode future feature states, decodes sparse temporal 2D point trajectories to model compact geometric motion, and couples both through a lightweight future-conditioned cross-attention module that enables consistent reasoning between anticipated states and point dynamics. Extensive experiments on LIBERO, RoboCasa GR-1 Tabletop, and LIBERO-Plus demonstrate state-of-the-art performance and strong zero-shot generalization. Project page is available at https://liauto-research.github.io/FoMoVLA.

Abstract

Explore similar work

FoMoVLA: Bridging Visual Foresight and Motion Guidance for Vision-Language-Action Models

ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models

Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation