ForeTime-VLA: Causal Future-Token Distillation from a World Action Model for Conveyor-Belt Manipulation

Siyuan Ma; Yutian Zhang; Boshi Zhang; Qinglian Wu; Jiaqi Zhai; Dong Wei; Xiaojin Huang

cs.AIAug 21, 2026

ForeTime-VLA: Causal Future-Token Distillation from a World Action Model for Conveyor-Belt Manipulation

Authors: Siyuan Ma, Yutian Zhang, Boshi Zhang, Qinglian Wu, Jiaqi Zhai, Dong Wei, Xiaojin Huang

Organizations: Tsinghua University, Beijing, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Harbin Institute of Technology, Harbin, China · Hangzhou Yunshenchu Technology Co., Ltd. (DEEP Robotics), Hangzhou, China

Abstract

Manipulating moving objects requires a policy to anticipate contact events, yet vision-language-action (VLA) policies are commonly fine-tuned from the current observation alone. World action models (WAMs) learn predictive dynamics, but running a video-scale teacher or explicitly imagining future frames at deployment is costly. We introduce ForeTime-VLA, a dense pi0.5 policy that distills a future-aware, action-equivalent representation from a frozen Fast-WAM-derived teacher while remaining causal at inference. Offline, current and future video latents are compressed into a whitened 64-D target. Online, an eight-frame history encoder predicts this target together with manipulation phase and normalized time-to-transition. Four future tokens and one phase token condition the VLM prefix, while the predicted future and transition horizon condition the action expert. Training retains the original flow-matching action target and adds cosine, relational geometry, phase, time-to-transition, and action-equivalence objectives. On a deduplicated conveyor-belt dataset, we compare 40k-step checkpoints on 768 matched windows per split. Test MAE decreases from 0.134119 to 0.130593 (2.63%; paired-bootstrap 95% CI: 0.82-4.48% improvement), and test L2 decreases by 3.02%, at a 2.46-2.93% latency cost. In quantitative real-robot evaluation, ForeTime-VLA achieves 81.1% stationary and 58.9% slow-moving grasp success, exceeding the next-best reference by 12.2 and 22.2 percentage points, respectively. Across three belt speeds, it completes 44/90 grasps versus 23/90 for pi0.5, including 11/30 versus 2/30 at fast speed. The agreement between offline orientation gains and reduced real-robot contact-pose failures supports causal future-token distillation as an effective way to improve dynamic manipulation without deploying the world-model teacher.

Figures & tables

Fig. 1: ForeTime-VLA overview. Left: the offline teacher encodes the current observation and sampled future frames with a frozen video VAE, then maps them to a compact action-equivalent target. Center: the deployable student uses eight causal observations to predict future, phase, and transition horizon variables. These predictions condition both the slow $\pi_{0.5}$ VLM prefix and the fast action-expert suffix, while the original noisy-action and target-flow construction is retained. Right: training combines action flow, future alignment, relational geometry, transition, phase, and auxiliary action-reconstruction losses. Future frames and the WAM are absent at test time.

Fig. 2: Offline action-equivalent future teacher. The frozen latent pipeline extracts current and future features $(c_{t},f_{t})$ , and the adapter also receives robot state $s_{t}$ . The privileged encoder uses $(c_{t},f_{t},s_{t})$ , whereas the causal branch uses $(c_{t},s_{t})$ . Both codes share an action decoder; the teacher additionally reconstructs the future feature difference. Teacher–causal alignment, variance/covariance regularization, and pairwise action geometry prevent collapse and retain action relevance. The selected 64-D teacher code is whitened and cached as $z_{t}^{T}$ .

Fig. 3: Dataset structure and realized coverage. Left: the collection taxonomy spans stationary grasp, mobile tracking, navigate-then-grasp, and concurrent-grasp settings; outer labels give canonical episode counts. Upper right: episode and aligned-frame coverage by scene. Lower right: object composition, the duplicate-free split, and the recorded multimodal contract. Aggregate statistics exclude 28 exact duplicate files; the remaining 458 episodes contain 96,512 aligned frames and are split 402/31/25 for train/validation/test.

Split	Model	MSE	MAE	L2
Val.	$\pi_{0.5}$ baseline	0.256338	0.138628	1.153725
	ForeTime-VLA	0.249276	0.134522	1.128938
	GR00T N1.6	0.278784	0.162035	1.285486
	StarVLA	0.258356	0.163372	1.262874
	SmolVLA	0.257650	0.147754	1.182720
Test	$\pi_{0.5}$ baseline	0.248680	0.134119	1.104541

TABLE I: Offline MSE, MAE, and L2 over 768 matched windows per split. Lower is better. The lower panel reports paired relative improvement and 95% CIs for ForeTime-VLA versus the

\pi_{0.5}

baseline.

Split	Group	Baseline	ForeTime-VLA
Val	base XY	0.016555	0.016176 (+2.30%)
	base yaw	0.008367	0.008367 (+0.00%)
	EE position	0.023643	0.023479 (+0.69%)
	EE RPY	0.478830	0.459112 (+4.12%)
	arm	0.095593	0.094838 (+0.79%)
Test	base XY	0.015229	0.015642 ( $-2.71$ %)

TABLE II: Pooled action-group MAE. Parentheses show relative improvement.

Method	Overall MAE $\downarrow$	Transition MAE $\downarrow$	Transition EE-RPY MAE $\downarrow$
$\pi_{0.5}$ baseline	0.122124	0.150140	0.501144
ForeTime-VLA w/o future condition	0.128787	0.159982	0.498349
ForeTime-VLA w/o phase/time condition	0.134846	0.160672	0.497727
ForeTime-VLA slow path only	0.136006	0.156587	0.494265
ForeTime-VLA fast path only	0.127140	0.161749	0.479751
ForeTime-VLA	0.118354	0.145320	0.482656

TABLE III: Offline inference-time component ablation. Overall metrics use 96 matched test windows; transition metrics use 48 matched phase-1 windows. Lower is better. All ForeTime-VLA variants share weights and differ only by component masking at inference.

Stationary grasp task
Method	Grasp success
$\pi_{0.5}$	65.56%
GR00T N1.6	46.67%
StarVLA	68.89%
SmolVLA	63.33%
ForeTime-VLA	81.11%

TABLE IV: Real-robot outcomes for stationary and slow-speed moving grasp tasks. All entries are percentages; higher success and lower failure are better.

Fig. 4: Real-robot grasp completions across belt speeds. Successful grasps out of 30 trials; labels give completed/total trials.

Fig. 5: Qualitative real-robot grasp and phase prediction. Top: grasp of the moving target. Bottom: causal phase probabilities; at $9.8$ s, grasp has 87.3% confidence before transport rises after acquisition.

Visuomotor Policy Learning

Visuomotor policy learning trains robots to turn camera observations, often with language and proprioception, into actions for manipulation and navigation.

See all 275 papers

Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation
Jun 1, 2026Shahram Najam Syed, Arthur Jakobsson, Haoran Hao +1World Models for Robotics Robotic Manipulation
Robotics Institute, Carnegie Mellon University, Pittsburgh, USA
MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies
Sep 30, 2026Jingqiu Wang, Yan WangVisuomotor Policy Learning Robotic Manipulation
School of Data Science and Engineering East China Normal University, Shanghai, China
Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation
Sep 30, 2026Chuyao Fu, Xiaowei Chi, Yuhan Rui +14World Models Action-Conditioned World Models
Southern University of Science and Technology · MUKA Robotics · Hong Kong University of Science and Technology +3

Jun 1, 2026J/K move · Enter open · S save

Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation

Shahram Najam Syed, Arthur Jakobsson, Haoran Hao, Jeffrey Ichnowski

Robotics Institute, Carnegie Mellon University, Pittsburgh, USA

World Models for Robotics Robotic Manipulation Vision-Language-Action Models Dynamic Robotic Manipulation VLMs for Robotics

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models generalize across static manipulation but fail when objects move during task execution. They map the current observation to an action and assume the scene is stationary between observation and execution, so at any non-trivial object speed the resulting latency exceeds the time available to grasp. We close this gap with AHEAD (Anticipatory Horizon Extrapolation with Adaptive Dynamics), a predict-then-act wrapper that augments a frozen VLA with a motion-aware latent world model. A small world model trained on manipulation video forecasts future patch tokens in the VLA's feature space, conditioned on per-token velocity and acceleration from optical flow. A language-and-motion saliency mask concentrates prediction on task-relevant patches, and the model rolls forward for an adaptive horizon, halting when prediction uncertainty crosses a threshold. The frozen action decoder then receives the predicted future tokens in place of the current ones. AHEAD adds 4.9M parameters to a frozen 7B OpenVLA and reaches 79 to 97% success across 20 dynamic simulation scenarios where the strongest baseline reaches 31 to 58%. On a physical UFactory xArm 7, AHEAD succeeds on 29/30 to 30/30 on three conveyor and rolling-ball tasks, 23/30 on paddle interception, and 19/30 on projectile catching where every baseline scores 0/30.

Abstract

Visuomotor Policy Learning

Explore similar work

Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation

MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies

Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation