Evo-World: A Lightweight Vision-Language-Action Model with Future Motion Prediction via Optical-Flow Supervision
Authors: Zile Zhong, Yuxin Du, Tao Lin, Haoxiao Wang, Wei Dai, Binhua Zuo, Bo Zhao
Organizations: School of AI, Shanghai Jiao Tong University · Zhejiang University · University of Electronic Science and Technology of China · QuicRobot
Vision-language-action (VLA) models provide a unified framework for visual understanding, language grounding, and robot control, yet anticipating how the scene will evolve beyond the current observation can provide additional information for control. Action supervision alone does not explicitly encourage such anticipation, while future-image prediction entangles task-relevant motion with largely static appearance. To address these limitations, we introduce Evo-World, a lightweight VLA model that augments policy learning with future motion prediction through optical-flow supervision, thereby providing a task-relevant predictive signal focused on motion rather than appearance. We further design the training procedure to preserve these predictive representations during policy learning, regulating representation diversity while decoupling representation learning from action optimization. Across simulation benchmarks, Evo-World demonstrates strong performance, achieving a tier-averaged success rate of 91.1% on Meta-World MT50 , with a 13.0% relative improvement over the non-predictive baseline. On real-world AgileX robots, Evo-World outperforms the evaluated baselines across single-arm and bimanual manipulation tasks while using only 0.87B parameters and delivering lower GPU memory usage and higher inference efficiency. These results demonstrate that future motion prediction through optical-flow supervision provides an effective and lightweight approach to enhancing the predictive capabilities of VLA models.
Figures & tables
Figure 1: Prediction targets for a VLA policy. (a) An action-only policy; (b) prediction of future appearance; and (c) prediction of future motion with optical-flow supervision. The two predictive branches condition action generation on latent representations. The flow illustration depicts the training target. The action-correlation labels express the motivation for motion supervision qualitatively.
Figure 2: Evo-World architecture. (a) The Motion Representation Module predicts a latent motion representation from DINOv2 image features and CLIP instruction features. In the dashed box, DINOv2 encodes an optical-flow RGB rendering as a target used only during training. (b) The InternVL3-1B vision-language backbone provides visual and linguistic context. (c) The current and predicted representations are concatenated and detached from action gradients, then projected into FiLM scale and shift parameters. These parameters modulate the backbone features to condition the flow-matching action expert for action generation.
Figure 3: Optical-flow supervision on Meta-World and RoboTwin. Panels (a) and (b) pair optical-flow renderings (top) with corresponding RGB observations (bottom) from Meta-World and RoboTwin, respectively. Colored regions emphasize motion, while nearly static regions appear close to white. Panel (c) shows the motion-to-color mapping: hue indicates displacement direction, and weaker motion produces colors closer to white.
Figure 4: Successful real-world executions of Evo-World on an AgileX robot. The rows show floral arrangement (top), a single-arm task, and mug hanging (middle) and microwave placement (bottom), two bimanual collaboration tasks.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1: Effect of SIGReg on latent representations. PCA projections for Evo-World (left pair) and the variant without SIGReg (right pair). Each pair shows zpred followed by zt . Colors indicate task difficulty, and black crosses mark the plotted means. For each representation, the corresponding panels use the same displayed axis scales. The annotations report the variance reduction factors when SIGReg is removed.
Figure A2: Example task sequences from Meta-World. From top to bottom: placing a basketball into a basket, opening a hinged door, opening a drawer, and opening a window. Each row shows six successive observations from left to right, with the corresponding language instruction below.
Figure A3: Example task sequences from RoboTwin. From top to bottom: pouring balls from a small container into a larger one, handing a microphone from one arm to the other, placing food items on a tray with both arms, and stacking a green block on a red block. Each row shows six successive observations from left to right, with the corresponding language instruction below.
Figure A4: Real-world experimental setup. Left: the teleoperation apparatus, with leader arms controlling the two follower arms and cameras providing an elevated view and left and right wrist views. Right: the Intel RealSense D435 camera and AgileX PiPER robot used in the setup. We collect 100 teleoperated demonstrations per task with varied object positions and use the collected data to separately fine-tune OpenVLA-OFT, π0 , and Evo-World.
Vision-Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared global action features may fail to establish timestep-specific correspondence between actions and local visual changes. To address this issue, we propose MotionWeave, a motion-centric future-dynamics framework for action-chunk prediction with two modules: the Action-Induced Motion Grounder (AIMG) and the Horizon Residual Composer (HRC). Specifically, AIMG conditions on action and proprioceptive representations to construct horizon-specific queries that localize interaction regions associated with each future action timestep from current visual tokens. HRC extracts differences between interaction representations at adjacent horizons, encodes them as temporal motion cues, and injects them into action tokens through a gated residual. During training, robot-arm masks rendered from future frames are used to construct KL-based motion-grounding supervision, while inference uses only the current observation. On six MetaWorld tasks, MotionWeave achieves a 75.3% average success rate, an absolute gain of 8.6% over π0 (66.7%), especially on sustained-interaction tasks. Our code is available at https://github.com/autu-mn/MotionWeave.
Jingqiu Wang, Yan Wang
School of Data Science and Engineering East China Normal University, Shanghai, China
Vision-Language-Action (VLA) models have achieved impressive results in visuomotor policy learning, yet remain fundamentally reactive, mapping current observations and language to actions without explicit forward prediction of world dynamics. Existing visual foresight methods predict future visual states but lack explicit motion guidance: they show where to go but not how to get there. We argue that future feature prediction and sparse point tracking are naturally complementary: the former provides the goal state, while the latter captures the continuous motion path toward it. We propose FoMoVLA, a framework that augments VLA representations with explicit spatio-temporal supervision by jointly learning future feature foresight and sparse 2D point tracking, enhancing the continuous action policy. FoMoVLA introduces compact foresight tokens to decode future feature states, decodes sparse temporal 2D point trajectories to model compact geometric motion, and couples both through a lightweight future-conditioned cross-attention module that enables consistent reasoning between anticipated states and point dynamics. Extensive experiments on LIBERO, RoboCasa GR-1 Tabletop, and LIBERO-Plus demonstrate state-of-the-art performance and strong zero-shot generalization. Project page is available at https://liauto-research.github.io/FoMoVLA.
Wei Li, Peijin Jia, Yuan Ma +9
LiAuto · University of Chinese Academy of Sciences · Tsinghua University
Learning to predict how the world evolves can provide vision-language-action (VLA) policies with predictive context for long-horizon control, but its effectiveness depends on what future representation is modeled and how it conditions action generation. We introduce PLaW-VLA, which models task-relevant future states in a pretrained prediction-oriented representation space, reducing the need to predict control-irrelevant visual details. Built on a Mixture-of-Transformers architecture, PLaW-VLA conditions action generation on observation history, current task semantics, and predicted future states through structured causal attention. Experiments show a +11.8 percentage-point (pp) gain over reactive policies on RoboTwin Hard Horizon III and a +1.77 pp gain over reconstruction-oriented latent prediction on zero-shot LIBERO-Plus, supporting improved long-horizon control and generalization under distribution shift, respectively. By avoiding low-level visual reconstruction, PLaW-VLA lowers the burden of future prediction, enabling a lightweight latent world model with parallel future prediction and about 1/19 the inference latency of generative world-action modeling at comparable policy performance.
Yu Liu, Hetian Guo, Tianlv Huang +11
Jilin University · Astribot · Harbin Institute of Technology, Shenzhen +2