PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation

Xinyu Guo; Chao Jing; Li Yi; Xingyu Chen; Zhengxing Wu

cs.ROOct 12, 2026

PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation

Authors: Xinyu Guo, Chao Jing, Li Yi, Xingyu Chen, Zhengxing Wu

Organizations: Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Zhongguancun Academy · Fudan University ·

Abstract

Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for world modeling. However, existing unified models that directly fuse them through shared attention at multiple layers can make it harder to draw on their specialized capabilities. We introduce PINT, a unified model that coordinates semantic reasoning, world modeling, and action generation through a shared chunk-level Predictive Intent while preserving expert-specific pathways. PINT inserts learnable Intent Tokens into the VLM sequence to form this representation, conditioning both the VGM and an Action Expert (AE). It replaces only the VGM's prompt conditioning and retains the VGM at inference, where the AE reads its representations through one-way attention. Rather than prescribing what Predictive Intent should encode, PINT learns it jointly from the native prediction objectives of the VLM, VGM, and AE, which provide complementary views of the same upcoming task execution. Training uses two stages: Stage 1 freezes the VGM so that future video prediction shapes Predictive Intent with task-relevant future dynamics, while Stage 2 fixes the learned Intent and adapts the VGM and AE. Without embodied pretraining, PINT achieves strong in-distribution (ID) performance and substantially improves out-of-distribution (OOD) generalization, reaching 15.5% on RoboTwin 2.0-Hard compared with 1.8% for Fast-WAM and 3.2% for StarVLA, and 79.5% on LIBERO-Plus compared with 51.5% and 75.0%, respectively. Across five real-world bimanual tasks, PINT reaches 68% ID and 24% OOD success, compared with 35%/9% for pretrained Motus and 67%/35% for pretrained $π_{0.5}$ .

Figures & tables

Figure 2: PINT architecture and attention design. Left: PINT inserts learnable Intent Tokens into the VLM sequence to form Predictive Intent, which conditions both the VGM and AE while preserving their expert-specific pathways and inputs. The AE receives layer-wise current-frame K/V from the VGM. Right: attention patterns among the VLM, VGM, and AE. OBS, IT, and AR denote the observation prefix, Intent Tokens, and training-only autoregressive suffix of action tokens. Img and

N_{1}

denote the VGM image and noise inputs, while State and

N_{2}

denote the AE state and action-noise inputs.

Figure 3: Future Video Prediction across ID/OOD Scenes and Training Stages. We visualize predictions from the VGM trained in Fast-WAM [ 8 ] and PINT after Stage 1 and Stage 2 on the same ID and OOD inputs. With the VGM frozen and no direct text conditioning, instruction semantics reach the VGM only through Predictive Intent, already yielding coarse task dynamics. Stage 2 fixes the learned Intent and adapts the VGM for more accurate motion.

Method	LIBERO					LIBERO-Plus
Method	Spatial	Object	Goal	Long	Avg.	Cam.	Robot	Lang.	Light	BG	Noise	Layout	Total
With Embodied Pretraining
$\pi_{0}$ [ 5 ]	96.8	98.8	95.8	85.2	94.2	13.8	6.0	58.8	85.0	81.4	79.0	68.9	53.6
$\pi_{0}$ -FAST [ 15 ]	96.4	96.8	88.6	60.2	85.5	65.1	21.6	61.0	73.2	73.2	74.4	68.8	61.6
RIPT-VLA [ 36 ]	98.6	98.6	99.0	93.8	97.5	55.2	31.2	77.6	88.4	91.6	73.5	74.2	68.4
OpenVLA-OFT [ 37 ]	97.6	98.4	97.9	94.5	97.1	56.4	31.9	79.5	88.7	93.3	75.8	74.2	69.6

Table 1: Performance on LIBERO and LIBERO-Plus. Success rates (%) on LIBERO (ID) and LIBERO-Plus (OOD). Best and second-best results in each group are bolded and underlined.

Method	PT	Easy	Hard	Avg.
With Embodied Pretraining
RDT [ 43 ]	$\checkmark$	34.5	13.7	24.1
$\pi_{0}$ [ 5 ]	$\checkmark$	46.4	16.3	31.4
GalaxeaVLA [ 44 ]	$\checkmark$	62.7	12.7	37.7
ABot-M0 [ 45 ]	$\checkmark$	57.4	30.4	43.9
X-VLA [ 46 ]	$\checkmark$	68.0	20.9	44.5

Table 2: Performance on RoboTwin 2.0. Success rates (%) under Easy and Hard, with all methods trained on clean demonstrations. PT denotes embodied pretraining. Best and second-best in each group are bolded and underlined.

\dagger

denotes our reproduction.

Eval.	Method	Turn On	Stack	Place	Store	Fold	Avg.
Eval.	Method	Light	Cups	Pens	Block	Towel	Success
	$\pi_{0.5}$	90	85	65	75	20	67
ID	Motus	50	60	30	35	0	35
	PINT	90	90	60	85	15	68
	$\pi_{0.5}$	30	70	20	45	10	35
OOD	Motus	5	20	5	15	0	9

Table 3: Real-world robot manipulation results. Success rates (%) under ID/OOD evaluation.

\pi_{0.5}

[ 2 ] and Motus [ 1 ] use embodied pretraining, while PINT does not. Best and second-best are bolded and underlined.

Setting	Easy	Hard
Prompt-conditioned VGM	62.4	11.1
Split Intent	65.1	11.8
PINT	68.6	15.5

Table 4: Ablation studies of PINT. All variants are trained on the 50-task clean demonstrations from RoboTwin 2.0 and evaluated under the Easy and Hard settings. We report success rates (%).

Figure 4: Predictive Intent visualization. We project ground-truth action chunks into a two-dimensional t-SNE space and color each point using a three-dimensional PCA projection of its corresponding Predictive Intent. Representative behaviors from highlighted regions are shown.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Camera views	VGM composition	State dim.	Action dim.
RoboTwin 2.0	Head + left wrist + right wrist	Head above two wrist views, $384\times 320$	14	14
LIBERO	Main + wrist	Main and wrist views side by side, $224\times 448$	8	7
Real-world ALOHA	Head + left wrist + right wrist	Same layout as RoboTwin 2.0, $384\times 320$	14	14

Appendix

Table 5: Benchmark-specific input configuration. Image sizes are reported as height

\times

width. The VLM receives camera views independently, whereas the VGM operates on a composed multi-view frame.

Module	Stage 1	Stage 2
VLM + Intent Tokens	$10^{-5}\rightarrow 10^{-6}$	Frozen
Intent $\rightarrow$ VGM projector	$10^{-5}\rightarrow 10^{-6}$	$10^{-4}\rightarrow 10^{-6}$
Intent $\rightarrow$ AE projector	$10^{-5}\rightarrow 10^{-6}$	$10^{-4}\rightarrow 10^{-6}$
AE + state encoder	$10^{-5}\rightarrow 10^{-6}$	$10^{-4}\rightarrow 10^{-6}$
VGM DiT	Frozen	$10^{-4}\rightarrow 10^{-6}$
Video VAE	Frozen	Frozen

Appendix

Table 6: Module-wise learning rates. Each entry reports the peak learning rate

\rightarrow

final learning rate after cosine decay.

Hyperparameter	RoboTwin 2.0	LIBERO	Real-World ALOHA
Stage 1 epochs	5	10	5
Stage 2 epochs	5	10	5
Stage 1 batch size / GPU	16	16	16
Stage 2 batch size / GPU	12	12	12
Stage 1 global batch size	128	128	128
Stage 2 global batch size	96	96	96

Appendix

Table 7: Training budget across benchmarks. Global batch sizes correspond to 8 GPUs with no gradient accumulation.

Figure 5: Experimental overview. PINT is evaluated on LIBERO [ 50 ] , LIBERO-Plus [ 14 ] , RoboTwin 2.0 [ 12 ] , and five real-world bimanual tasks under ID/OOD settings.

Domain randomization
Field	Easy / ID	Hard / OOD
Background randomization	Off	On
Table clutter	Off	On
Clean background rate	1.00	0.02
Table-height perturbation	0	0.03
Lighting randomization	Off	On

Appendix

Table 8: RoboTwin 2.0 Easy and Hard evaluation configurations. The Easy setting uses the clean environment, while the Hard setting applies the official domain randomization.

Task	Demonstrations
Turn On Light	50
Stack Cups	50
Place Pens	150
Store Block	150
Fold Towel	300
Total	700

Appendix

Table 9: Real-world training data for PINT. Number of demonstrations collected for each ALOHA manipulation task.

Method	LIBERO source	LIBERO-Plus source
$\pi_{0}$	OpenVLA-OFT [ 37 ]	Official LIBERO-Plus leaderboard
$\pi_{0}$ -FAST	OpenVLA-OFT [ 37 ]	Official LIBERO-Plus leaderboard
RIPT-VLA	RIPT-VLA [ 36 ]	Official LIBERO-Plus leaderboard
OpenVLA-OFT	OpenVLA-OFT [ 37 ] , Table I	Official LIBERO-Plus leaderboard
MemoryVLA++	MemoryVLA++ [ 38 ] , Table I	MemoryVLA++ [ 38 ] , Table V
Motus	MoWAM [ 52 ] , Table I	MoWAM [ 52 ] , Table III

Appendix

Table 12: Result provenance for Table 1 . Each entry identifies the source of the corresponding LIBERO or LIBERO-Plus result.

Method	Source	Note
RDT	Official RoboTwin 2.0 leaderboard	Clean-to-clean / clean-to-randomized
$\pi_{0}$	Official RoboTwin 2.0 leaderboard	Clean-to-clean / clean-to-randomized
GalaxeaVLA	Official RoboTwin 2.0 leaderboard	Clean-to-clean / clean-to-randomized
ABot-M0	Official RoboTwin 2.0 leaderboard	Clean-to-clean / clean-to-randomized
X-VLA	Official RoboTwin 2.0 leaderboard	Clean-to-clean / clean-to-randomized
X-WAM	Official RoboTwin 2.0 leaderboard	Clean-to-clean / clean-to-randomized

Appendix

Table 13: Result provenance for Table 2 . Leaderboard values refer to the official RoboTwin 2.0 leaderboard snapshot used for this submission.

	LIBERO	LIBERO-Plus
Setting	Avg.	Cam.	Robot	Lang.	Light	BG	Noise	Layout	Total
Prompt-conditioned VGM	95.9	53.2	39.1	78.7	88.3	45.7	69.8	70.9	63.4
w/o Stage 1 Video Loss	97.1	53.5	65.3	80.2	88.7	88.8	83.1	75.1	75.2
PINT	98.0	59.2	73.7	84.4	94.0	91.4	80.8	81.2	79.5

Appendix

Table 14: Ablations on LIBERO and LIBERO-Plus. Success rates (%) on LIBERO (ID) and across the seven LIBERO-Plus distribution shifts. The LIBERO-Plus Total is computed over all 10,030 evaluation trials rather than by averaging the seven category-level success rates.

Benchmark	Metric	Eval. 1	Eval. 2	Eval. 3	Mean $\pm$ Std.
RoboTwin 2.0	Easy	68.34	68.66	68.90	68.63 $\pm$ 0.28
RoboTwin 2.0	Hard	15.08	15.52	15.86	15.49 $\pm$ 0.39
LIBERO	Avg.	98.2	97.8	98.0	98.00 $\pm$ 0.20
LIBERO-Plus	Total	79.5	79.5	79.4	79.47 $\pm$ 0.06

Appendix

Table 15: Repeated evaluation of PINT. Each row uses a fixed trained checkpoint and three evaluation seeds. We report the mean and sample standard deviation across evaluations.

Method	Easy $\uparrow$	Hard $\uparrow$	Avg. $\uparrow$	Latency (ms) $\downarrow$
Fast-WAM	72.4	1.8	37.1	266
PINT	68.6	15.5	42.1	331
PINT-VA	71.9	18.1	45.0	776

Appendix

Table 16: Performance and inference efficiency. RoboTwin 2.0 success rates (%) and mean end-to-end latency for generating one 32-step action chunk. All latency measurements use the same NVIDIA A100 80GB GPU, BF16 precision, batch size 1, and 10 action denoising steps.

	RDT		$\pi_{0}$		Evo-1		PINT		PINT-VA
Task	Easy	Hard	Easy	Hard	Easy	Hard	Easy	Hard	Easy	Hard
Adjust Bottle	81	75	90	56	100	40	99.67	40.67	99	50
Beat Block Hammer	77	37	43	21	82	8	87.67	13.67	95	19
Blocks Ranking RGB	3	0	19	5	76	21	82.67	3.67	83	6
Blocks Ranking Size	0	0	7	1	58	7	34.00	0.00	53	2
Click Alarmclock	61	12	63	11	98	18	84.67	22.33	93	17

Appendix

Table 17: Per-task performance on RoboTwin 2.0. Success rates (%) on all 50 tasks under the Easy and Hard settings. PINT results are averaged over three repeated evaluations.

Panel of Figure 6 — Figure 6: Expanded closed-loop visualization corresponding to Fig. 3 . We show closed-loop future-video predictions under ID (Easy / clean) and OOD (Hard / randomized) conditions for placing the container on the plate. Rows compare the VGM from our Fast-WAM reproduction with PINT after Stage 1 and Stage 2.

Figure 7: Additional open-loop future-video predictions on RoboTwin 2.0 (part I). For each task, the left and right panels show ID (Easy / clean) and OOD (Hard / randomized) examples, respectively. Rows show the ground-truth future frames, the VGM trained in Fast-WAM, PINT after Stage 1, and PINT after Stage 2. Columns show the main-camera view at $t{+}8$ , $t{+}16$ , $t{+}24$ , and $t{+}32$ .

Figure 8: Additional open-loop future-video predictions on RoboTwin 2.0 (part II). Same visualization format as Fig. 7 .

Figure 9: Qualitative real-world rollouts on individual tasks. We show representative executions of Turn On Light, Stack Cups, Place Pens, Store Block, and Fold Towel under both in-distribution and out-of-distribution conditions. For each task, the top row shows the in-distribution rollout and the bottom row shows the corresponding out-of-distribution rollout. Frames are ordered temporally from left to right.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,672 papers

Jul 6, 2026

InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization

Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through future prediction. In practice, existing designs tend to erode the semantics of the pretrained backbone, suffer interference among heterogeneous objectives, and learn future prediction from scratch in pixel space, leaving the dynamics priors of pretrained video generators unexploited. We present InternVLA-A1.5, which builds the policy on a native VLM backbone that keeps training on VQA and subtask prediction, and attaches a lightweight unified expert for continuous action generation. Future prediction is recast as a latent-querying problem, where a small set of learnable foresight tokens condenses the task-relevant future into a compact latent code under the supervision of a frozen pretrained video generation model, so the policy inherits world-model dynamics priors without ever learning pixel-level generation. The video branch is discarded at inference, keeping real-time control. Pretrained on 1.2M robot episodes and 3M multimodal samples, InternVLA-A1.5 achieves the best overall results on all six simulation benchmarks. In the real world, the preserved semantics deliver the strongest compositional generalization on held-out instruction bindings, and the two designs together sustain long-horizon execution.

Haoxiang Ma, Junhao Cai, Xiaoxu Xu +26

Physical Intelligence Team, Shanghai AI Laboratory

Generalization in Robotic Manipulation Visuomotor Policy Learning AI Control Vision-Language-Action Models Latent World Models Compositional Generalization

Ask PDF

Apr 13, 2026

AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps

Pretrained video generation models provide strong priors for robot control, but existing unified world action models still struggle to decode reliable actions without substantial robot-specific training. We attribute this limitation to a structural mismatch: while video models capture how scenes evolve, action generation requires explicit reasoning about where to interact and the underlying manipulation intent. We introduce AIM, an intent-aware unified world action model that bridges this gap via an explicit spatial interface. Instead of decoding actions directly from future visual representations, AIM predicts an aligned spatial value map that encodes task-relevant interaction structure, enabling a control-oriented abstraction of future dynamics. Built on a pretrained video generation model, AIM jointly models future observations and value maps within a shared mixture-of-transformers architecture. It employs intent-causal attention to route future information to the action branch exclusively through the value representation. We further propose a self-distillation reinforcement learning stage that freezes the video and value branches and optimizes only the action head using dense rewards derived from projected value-map responses together with sparse task-level signals. To support training and evaluation, we construct a simulation dataset of 30K manipulation trajectories with synchronized multi-view observations, actions, and value-map annotations. Experiments on RoboTwin 2.0 benchmark show that AIM achieves a 94.0% average success rate, significantly outperforming prior unified world action baselines. Notably, the improvement is more pronounced in long-horizon and contact-sensitive manipulation tasks, demonstrating the effectiveness of explicit spatial-intent modeling as a bridge between visual world modeling and robot control.

Liaoyuan Fan, Zetian Xu, Chen Cao +3

INFIFORCE Intelligent Technology Co., Ltd., Hangzhou, China · The University of Hong Kong, Hong Kong SAR, China · Shanghai Jiao Tong University, Shanghai, China

World Action Models Robotic Manipulation Long-Horizon Robotic Manipulation Robotic RL

Ask PDF

Oct 7, 2026

Video Prediction Policy 2: Predict Better, Act Better

World action models (WAMs) have emerged as an important class of generalist robot policies, aiming to transfer video prediction priors to action learning. However, we find that existing WAMs frequently produce incorrect motion predictions in open-ended environment, leading to erroneous actions. We attribute this limitation to two factors: (1) base video models are not optimized for manipulation, and (2) naively incorporating action components into video models can substantially degrade their generalization capabilities. We introduce Video Prediction Policy 2 (VPP2), a WAM that enables strong zero-shot generalization in both video prediction and action generation. First, we curate a large-scale, diverse dataset of manipulation videos to continue pretraining the base video foundation model. We annotate video clips with detailed captions and perform event-level video pretraining to promote generalization across open-ended manipulation tasks. Second, we post-train and distill the video model into a single-step visual planner with fixed prediction horizon. Finally, we introduce action module via a mixture-of-transformers (MoT) architecture to learn implicit inverse dynamics model. Experiments demonstrate three key results: (1) VPP2-14B outperforms Cosmos3-64B by 11.0% points in video prediction instruction-following success rate on open-ended tasks; (2) VPP2 surpasses the strongest baseline by 18.5% points in success rate on real-world zero-shot ALOHA manipulation tasks; and (3) following benchmark-specific post-training, VPP2 achieves the highest success rates among evaluated methods on the challenging LIBERO-Pro, LIBERO-OOD, and RoboDojo benchmarks.

Yanjiang Guo, Haodong Yan, Zhide Zhong +15

Robotera · Tsinghua University · HKUST (GZ) +2

Visuomotor Policy Learning World Action Models Robot Learning from Video Zero-Shot Robotic Manipulation Video Prediction

Ask PDF

Abstract

Robotic Manipulation

Explore similar work

InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization

AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps

Video Prediction Policy 2: Predict Better, Act Better