One from Infinity: Actualizing Futures from Pretrained World Models into Robot Actions

Bang Du; Yichen Xie; Shuqi Zhao; Yuxin Chen; Menglin Wu; Masayoshi Tomizuka

cs.ROSep 29, 2026

One from Infinity: Actualizing Futures from Pretrained World Models into Robot Actions

Authors: Bang Du, Yichen Xie, Shuqi Zhao, Yuxin Chen, Menglin Wu, Masayoshi Tomizuka

Organizations: University of California, Berkeley · Southern University of Science and Technology · Xi’an Jiaotong University

Abstract

A pretrained video world model admits many plausible futures for a scene, but a robot must realize the exact task-conditioned one. To turn world models into executable robot policies, existing methods fine-tune the heavy world model backbone using large-scale robot data and computational resources. Challenging this status quo, we argue that the expensive part has already been paid in the world model pretraining since the representation space of a video world model lays out the diverse potential futures. In this case, what remains is to select the future that accomplishes the task and to read out the actions that realize it. We formalize this task as actualization, which learns a task-conditioned selection and realization on top of a prior supplied by a frozen world model. This can be solved by a tiny actualizer model. We implement RoboActualizer with as few as 60M parameters on top of a frozen world model encoder. The actualizer is composed of two lightweight DiT experts that jointly predict future latents and actions by flow matching. The model can be trained entirely on a single GPU with 32 GB peak memory. With up to 100x fewer trainable parameters than existing WAMs and VLAs, RoboActualizer reaches great performance on simulation benchmarks including LIBERO, LIBERO-Plus, RoboTwin 2.0 and five tasks on two real-world platforms, with a low latency of 39 ms that allows real-time control.

Figures & tables

Panel of Figure 1 — Figure 1: RoboActualizer selects the task-conditioned future from the multiple plausible ones contained by the representation space (left). With as few as 60 M parameters, RoboActualizer outperforms baselines with $100\times$ more parameters on simulation benchmarks (right).

Figure 2: Overview of RoboActualizer framework. RoboActualizer is built upon a frozen world model. It actualizes the task-conditioned latent future with a lightweight model.

Figure 3: Cumulative explained variance of V-JEPA 2.1 latents. The shaded band is the range across benchmark suites. The dashed line marks the actualizer width $d_{\text{model}}=384$ , at which $93.8\%$ of the variance is retained.

Method	Trainable Params.	Emb. PT	Spatial	Object	Goal	Long	Overall	Latency (ms)
OpenVLA ( Kim et al., 2024 )	279M	Yes	84.7	88.4	79.2	53.7	76.5	145.5
$\pi_{0}$ ( Black et al., 2024 )	3.3B	Yes	96.8	98.8	95.8	85.2	94.1	120.4
$\pi_{0.5}$ ( Zhou et al., 2025 )	3.3B	Yes	98.8	98.2	98.0	92.4	96.9	128.5
UniVLA ( Bu et al., 2025 )	123M	Yes	96.5	96.8	95.6	92.0	95.2	157.3
Motus ( Bi et al., 2026 )	5.9B	Yes	96.8	99.8	96.6	97.6	97.7	2230.0
WorldVLA ( Cen et al., 2025 )	7.0B	No	87.6	96.2	83.4	60.0	81.8	397.5

Table 1: Success rates (%) on LIBERO benchmark. “Embodied PT” denotes additional embodied data pretraining before downstream task training.

Method	Trainable Params.	Camera	Robot	Lang.	Light	Bg.	Noise	Layout	Overall	Latency (ms)
OpenVLA ( Kim et al., 2024 )	279M	0.8	3.5	23.0	8.1	34.8	15.2	28.5	15.6	145.5
$\pi_{0}$ Black et al. (2024)	3.3B	13.8	6.0	58.8	85.0	81.4	79.0	68.9	53.6	120.4
$\pi_{0}$ -Fast	3.3B	65.1	21.6	61.0	73.2	73.2	74.4	68.8	61.6	68.5
UniVLA ( Bu et al., 2025 )	123M	1.8	46.2	69.6	69.0	81.0	21.2	31.9	42.9	157.3
WorldVLA ( Cen et al., 2025 )	7.04B	0.1	27.9	41.6	43.7	17.1	10.9	38.0	25.0	397.5
Fast-WAM ( Yuan et al., 2026 )	6.02B	16.4	44.5	68.9	78.2	53.7	37.7	60.7	51.5	111.7

Table 2: Success rates (%) on LIBERO-Plus benchmark. ∗ : no open-source code to measure latency.

Method	Trainable Params.	Success Rates (%)
$\pi_{0.5}$ ( Zhou et al., 2025 )	3.3B	31.4
LingBot-VA ( Li et al., 2026 )	5.3B	17.2
Fast-WAM ( Yuan et al., 2026 )	6.0B	41.9
RoboActualizer (ours)	60M	58.8

Table 3: Success rates (%) on RoboTwin 2.0 (50 demos per task).

Figure 4: Real-world tasks on humanoid and robot arm platforms.

Table 8

Figure 5: Predicted versus ground-truth future latent prediction at multiple time steps, visualized with PCA dimension reduction. Features predicted by latent expert tracks the teacher outputs greatly.

Representation	Params.	Input Range	Latent Predictivity	Success Rates (%)
DINOv3	300M	1 frame	✗	39.7
WAN VAE	127M	4 frames	✗	25.9
V-JEPA $2.1_{image}$	300M	1 frame	✗	46.4
V-JEPA $2.1_{image}$	300M	4 frames	✗	44.8
V-JEPA $2.1_{video}$	300M	4 frames	✓	60.0

Table 6: Ablation study on latent representation. The ability for future prediction inside the latent space is more important than its representation ability.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Table 8: Ablation study on model architecture design.

	LIBERO	LIBERO-Plus	RoboTwin 2.0
Action chunk / executed steps	16 / 10	16 / 10	32 / 24
Euler steps	4	4	10
CFG scale	1	1.5	1
Future-image generation	off	off	off
Warm-up no-op steps	30	30	–

Appendix

Table 9: Closed-loop inference settings.

Task	C	R	M	Task	C	R	M
adjust_bottle	100	100	100	place_can_basket	64	64	64
beat_block_hammer	76	64	70	place_cans_plasticbox	56	84	70
blocks_ranking_rgb	36	36	36	place_container_plate	100	96	98
blocks_ranking_size	24	32	28	place_dual_shoes	64	44	54
click_alarmclock	92	84	88	place_empty_cup	72	72	72
click_bell	100	92	96	place_fan	40	56	48

Appendix

Table 10: RoboTwin 2.0 per-task success rate (%),

25

episodes per cell. C = Clean, R = Random, M = mean.

Explore similar work

FutureWorlds: Learning Robotic World Models from Alternative Futures
Oct 1, 2026Hao Wu, Shengju Qian, Weiyan Wang +5World Models Scene Context
HKUST (GZ) · CUHK · Tencent +3
Latent Action as Intention Enables Efficient Future Imagination for World Action Models
Aug 25, 2026Xiang Li, Yupeng Zheng, Songen Gu +11Efficient World-Action Model Latent Actions
1CollegeAI, THU · 2AIR, THU · 4TARS Robotics +6
$τ_0$ -WM: A Unified Video-Action World Model for Robotic Manipulation
May 31, 2026Pengfei Zhou, Shengcong Chen, Di Chen +17Efficient World-Action Model Video World Models
1Shanghai Innovation Institute · 2AGIBOT Finch

Oct 1, 2026, cs.CVJ/K move · Enter open · S save

FutureWorlds: Learning Robotic World Models from Alternative Futures

Hao Wu, Shengju Qian, Weiyan Wang, Fan Xu+4

HKUST (GZ) · CUHK · Tencent+3

World Models Scene Context

Open paper AskPDF arXiv

Robotic world models predict action-conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate-specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory-Conditioned Search-Guided Policy Optimization), which converts video trajectory rewards into group-relative advantages to optimize the world model. On RT-1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32-frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical-flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander-wu/FutureWorlds.

Abstract

Explore similar work

FutureWorlds: Learning Robotic World Models from Alternative Futures

Latent Action as Intention Enables Efficient Future Imagination for World Action Models

τ0τ_0τ0​-WM: A Unified Video-Action World Model for Robotic Manipulation

$τ_0$ -WM: A Unified Video-Action World Model for Robotic Manipulation