PLaCA: Learning Patch-Level Future Latent Change for Action Generation

Zhan Yi; Yuchen Wang; Chao Li; Lei Hong; Boyuan Yang

cs.ROOct 20, 2026

PLaCA: Learning Patch-Level Future Latent Change for Action Generation

Authors: Zhan Yi, Yuchen Wang, Chao Li, Lei Hong, Boyuan Yang

Organizations: School of Robotics and Automation, Nanjing University, Suzhou, Jiangsu, China

Abstract

Vision-Language-Action (VLA) models generate robot actions from visual observations and language instructions, but effective manipulation also requires anticipating how the environment will change. Existing world-action models explore future image prediction and latent dynamics modeling. However, generating complete future images can be computationally expensive, while highly compressed latent representations may obscure local changes. To address these challenges, we propose PLaCA, a world-to-action framework built around future latent change. PLaCA directly predicts spatially structured patch-level future latent change from the current multimodal input, preserving local information about expected environmental changes. We introduce patch-level change-aware supervision to emphasize regions undergoing larger changes and a future-to-action interface that explicitly incorporates predicted future latent change into action generation. Experiments on LIBERO-Long show that PLaCA achieves a 79.0% success rate, improving upon the Action-only baseline by 5.0 percentage points. On CALVIN, PLaCA improves the average sequence length from 2.784 to 3.564. These improvements are achieved while updating only 2.95% of the model parameters during the second training stage, demonstrating the effectiveness of future latent change modeling for downstream robotic manipulation.

Figures & tables

Figure 1: Motivation for spatially structured future latent change. (a) In robotic manipulation, future changes are often concentrated around the robot arm and manipulated objects, while much of the background remains unchanged. (b) Highly compressed state representations do not explicitly preserve the spatial locations of these changes, whereas patch-level future latent change retains their spatial structure and provides future information for action generation.

Figure 2: Overview of our world-to-action framework. The shared multimodal backbone produces action-query representations, from which a future-change decoder predicts patch-level future latent change. During training, a frozen visual encoder provides change-aware supervision using current and future observations. The predicted future latent change is then incorporated into the action representation through a gated future-to-action interface to generate the full action chunk. Future observations are not required during inference.

Figure 3: Patch-level future latent change and change-aware supervision. (a) A frozen visual encoder extracts patch-level features from current and future observations. Their differences at corresponding patch locations define the ground-truth future latent change $\Delta Z_{t}^{*}$ , preserving the spatial structure across camera views. (b) The magnitude of each ground-truth change vector is used to compute patch-wise supervision weights, assigning larger relative weights to patches with greater changes. (c) The predicted future latent change $\widehat{\Delta Z}_{t}$ is supervised against $\Delta Z_{t}^{*}$ using weighted regression and directional losses. All patches contribute to the training objective with different weights.

Method	Params (B)	LIBERO-Long $\uparrow$
OpenVLA [ 25 ]	7.0	53.7
UniAct [ 54 ]	0.5	46.5
UniAct + ACT Head [ 54 , 53 ]	$\sim$ 0.5	70.0
RoboMonkey [ 26 ]	$\sim$ 14	56.5
PDF [ 50 ]	$7.0^{\dagger}$	59.0
Dita (OXE) [ 18 ]	0.334	63.8

Table 1: Comparison on LIBERO-Long. Params denotes the reported model size. Results from prior methods are taken from their respective papers.

\dagger

PDF uses a 7B OpenVLA backbone with an additional 9M trainable parameters.

Figure 4: Per-task success rates on LIBERO-Long. Comparison between the Action-only baseline and PLaCA across all 10 tasks, with 20 closed-loop rollouts per task.

Method	ASL $\uparrow$
Action-only	2.784
PLaCA (Ours)	3.564

Table 2: Results on CALVIN ABC

\rightarrow

D. ASL denotes the average number of consecutively completed subtasks per five-task sequence.

Variant	Patch Grid	Supervision	F2A	Success $\uparrow$
Action-only	–	–	–	74.0
Action-only (+40 epochs)	–	–	–	72.5
PLaCA ( $7\times 7$ )	$7\times 7$	Change-aware	✓	77.5
Patch-change $\rightarrow$ Action	$14\times 14$	Uniform	✓	76.0
Future-change auxiliary	$14\times 14$	Change-aware	–	76.5
Absolute future latent	$14\times 14$	Change-aware	✓	74.5

Table 3: Ablation studies of PLaCA on LIBERO-Long. Patch Grid denotes the spatial resolution of the future latent representation. Uniform and Change-aware indicate the patch-wise supervision weighting scheme. F2A denotes the future-to-action interface.

Temporal Modeling for VLA Models

See all 27 papers

WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
Oct 13, 2026Haomin Wang, Anqiong Ji, Ke Wang +4World Models for Robotics World Action Models
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Baidu AI Cloud · Awaken Robotics +1
$Δ$ VLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation
Mar 9, 2026Yijie Zhu, Jie He, Rui Shao +4Vision-Language-Action Models VLMs for Robotics
Harbin Institute of Technology, Shenzhen, Shenzhen 518055, China · Great Bay University, Dongguan 523000, China · Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China +1
SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models
Sep 27, 2026Tianfu Li, Haoxuan Xu, Wenbo Chen +7World Model Learning Visuomotor Policy Learning
The Hong Kong University of Science and Technology (Guangzhou). · The Hong Kong University of Science and Technology. · Nanyang Technological University. +1

Oct 13, 2026J/K move · Enter open · S save

WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction

Haomin Wang, Anqiong Ji, Ke Wang, Zhongshan Liu+3

AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Baidu AI Cloud · Awaken Robotics+1

World Models for Robotics World Action Models Vision-Language-Action Models Latent World Models

Open paper AskPDF arXiv

Vision-language-action (VLA) policies excel at generalist robotic manipulation but generate actions without explicitly modeling their environmental consequences. World action models (WAMs) incorporate future prediction, yet most rely on costly video generation, while latent alternatives either use predicted futures only as auxiliary targets or infer latent actions from suboptimal visual transitions through costly large-scale video pretraining. We introduce WAVE-0 (World-model-Augmented VLA Execution), a framework that bridges VLAs and WAMs through a lightweight latent world model. By connecting the semantic understanding of pretrained VLAs with explicit future prediction, WAVE-0 allows anticipated action consequences to directly guide action generation. Built on a pretrained VLA, its 257M world model translates VLM-generated latent action plans into predicted future representations that condition the action expert, unifying latent planning, future prediction, and action generation. The world model is trained directly on ground-truth robot actions with a counterfactual ranking objective, requiring neither pixel reconstruction nor large-scale video pretraining. WAVE-0 achieves success rates of 91.47% on RoboTwin 2.0 and 98.7% on LIBERO, along with 88.0%, 61.4%, and 75.0% on LIBERO-Plus, LIBERO-Pro, and LIBERO-Para, respectively. On real-world tasks, it improves the success rate of

π_{0.5}

from 38.3% to 73.3%. These results demonstrate strong manipulation performance and generalization across diverse distribution shifts.

Abstract

Temporal Modeling for VLA Models

Explore similar work

WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction

ΔΔΔVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation

SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models

$Δ$ VLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation