Organizations: Tsinghua University · Johns Hopkins University · Northeastern University · Southeast University · Institute of Automation, Chinese Academy of Sciences · PAI X Lab, Yinwang Intelligent Technology Co. Ltd. · Lab, Yinwang Intelligent Technology Co. Ltd.
World Action Models (WAMs) can condition robot actions on generated visual futures, but this additional context is neither cost-free nor uniformly beneficial. We examine what a predicted future actually changes: the action delta Δa between the future-conditioned and future-free execution paths of the same model. Under a shared WAM checkpoint, this delta barely shifts average benchmark success (+0.8 points on LIBERO, −0.8 on RoboTwin2.0), while generating visual futures increases inference latency by 1.7--1.9×. These aggregate metrics conceal a concentrated failure mode that we term future-induced negative transfer: while predictive deltas benefit long-horizon tasks (+2.4 points on LIBERO-Long) and remain negligible on most routine manipulations, they induce severe degradation on precision tasks---with Press Stapler and Place Can Basket losing 54.0 and 34.0 points, recurring as top failure modes across independently trained WAM variants. We introduce Utility-WAM for in-place correction of future-conditioned control. Its primary learning component is a residual adapter that repairs future-induced velocity deltas under a frozen shared WAM by aligning them with ground-truth dynamic guidance and penalizing excess error beyond the future-free baseline. The adapter selectively eliminates the majority of severe task drops (reducing losses to 6.0 and 14.0 points) while preserving long-horizon foresight. To eliminate superfluous video computation, a lightweight pre-generation gate reuses counterfactual action outcomes to selectively bypass video generation on 75.5% of RoboTwin queries, cutting latency by 38% while improving average success to 93.3% on RoboTwin2.0 and 98.9% on LIBERO. On physical bimanual manipulation across three real-world tasks, Utility-WAM succeeds in 53 of 60 trials, compared to 43 for FAST-WAM.
Figures & tables
Figure 1: Utility-WAM repairs future-induced action changes before allocating future-generation compute. Stage 1 trains a shared WAM with matched future-free and future-visible action paths. Stage 2 freezes the WAM and trains a residual adapter on self-generated future features, using beneficial ground-truth-future velocity deltas and the future-free path as supervision. These two stages establish the core correction method. Stage 3 reuses the adapter’s paired final-action outcomes to train a gate. At inference, the gate runs before video generation: it either executes the future-free path and skips that compute, or invokes future generation and the corrected future-conditioned path. Solid arrows denote inference data flow; dashed orange arrows denote training-only supervision.
Figure 2: Accuracy–compute frontier of selective routing. Success rate versus effective video generation steps on LIBERO and RoboTwin 2.0; moving left saves video generation. Neff denotes the average number of video generation steps per action query. Red stars mark the validation-calibrated operating points; the remaining points form a test-time threshold sweep. The selective configuration is compared with two compute-matched baselines. The correction-only curve is a static schedule that updates the future video only during the first n of the ten denoising steps of every query, so that Neff=n . The dashed Random Gate curve routes each query to the corrected future-conditioned path with probability Neff/10 .
Figure 4: Real-world success counts (20 trials per task) on the tasks in Figure 4 b–d. Shared WAM and Utility-WAM (Correction) follow the configurations of Table 2 and are evaluated under the same protocol. Bold and underlined entries denote the best and second-best results in each column, respectively (including ties).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
FAST-WAM
Ours
Task
Joint
IDM
Shared WAM (w)
Correction
Selective
Press Stapler
−42.5
−40.0
−54.0
−6.0
−6.5
Place Can Basket
−33.5
−37.5
−34.0
−14.0
−17.0
Open Microwave
−45.0
0.0
−1.5
−1.0
−0.5
Blocks Ranking Size
−9.0
−11.5
−2.5
−2.0
−0.5
Open Laptop
−8.5
−7.0
−2.5
−2.0
−0.5
Appendix
Table 8: Per-task success changes on RoboTwin 2.0: the ten most harmed tasks. All entries are ΔJt in percentage points relative to the future-free reference, averaged over clean and randomized conditions; negative values indicate harm. Tasks are ranked by the mean ΔJt over the three future-conditioned comparisons (Joint, IDM, and Shared WAM (w)), most negative first, with ties broken by Shared WAM (w). FAST-WAM columns use the published per-task results of Yuan et al. (2026) with FAST-WAM as reference. Our columns use Shared WAM (wo) as the common reference and are paired over the same episodes. The bottom rows summarize all 50 tasks.
Figure 5: Representative successful real-world evaluation sequences. Keyframes for Place Empty Cup , Insert Can , and Stack Three Blocks are shown from top to bottom and progress from left to right, illustrating the main manipulation stages and the final task configurations after gripper withdrawal. These examples are qualitative and are not used for method attribution.
World action models (WAMs) improve robot control by modeling how observations evolve, but generating future observations at test time incurs substantial latency. Fast-WAM removes this process for efficiency; however, our matched implementations show lower generalization for Fast-WAM than for future-aware alternatives, especially with scarce robot demonstrations and in out-of-distribution scenarios. To bridge this gap, we introduce LAWA, a WAM architecture that uses compact latent actions as an operational representation of future intentions, enabling efficient test-time future imagination without generating future observations. Specifically, a discrete tokenizer enhanced by action-free pre-training produces manipulation-centric codebook targets. LAWA jointly denoises a continuous latent state anchored to these targets with executable action chunks while omitting the future-video branch at inference. On RoboCasa, LAWA achieves state-of-the-art average success rates of 65.6% and 80.8% in the few-shot and full data settings, improving over the matched Fast-WAM baseline by 9.6 and 4.5 points, respectively. It also preserves the performance level of the matched Joint-WAM variant while requiring 42.9% lower inference latency. LAWA also demonstrates competitive zero-shot robustness on LIBERO-Plus and superior performance on real-world tasks. These results show that future imagination need not be discarded: retaining it with compact latent actions yields an effective trade-off among performance, generalization, and latency. Code and models will be released.
World-Action Models (WAMs) have emerged as a promising paradigm for embodied control by coupling future visual prediction with action generation. However, most existing WAMs rely on photorealistic future prediction, which incurs high inference latency and makes real-time robot deployment difficult. This motivates a more efficient WAM design that preserves the control benefits of future visual prediction while reducing its inference cost. We introduce Efficient-WAM, a World-Action Model that reduces the cost of future imagination while preserving its control benefit. Efficient-WAM improves inference efficiency via a compact video expert transferred from WAN-2.2-5B, token-sparse video latents, and asymmetric video-action denoising that allocates fewer sampling steps to video than to actions. Instead of optimizing the future branch for visual fidelity, Efficient-WAM treats future video prediction as a compact guidance signal for action generation. Comprehensive experiments on RoboTwin 2.0 and real-world manipulation tasks show that Efficient-WAM maintains strong action performance despite visibly coarse future predictions. While maintaining competitive control capabilities, our 1B-parameter model can reduce per-chunk latency to around 100 ms during physical deployment, achieving a 30x speedup over existing WAMs.
Jiajun Li, Tiecheng Guo, Yifan Ye +9
1The University of Hong Kong · 2Peking University · 3Muka Robotics +2
World Action Models (WAMs) improve robot manipulation by learning how the environment evolves beyond the current observation. However, existing approaches face a fundamental dilemma: Joint-WAMs preserve future-aware representations during inference but incur prohibitive computation costs, while efficient alternatives remove future modeling at inference time and may lose the robustness benefits of temporal reasoning. In this work, we revisit the role of future representations in WAMs and show that inference-time future conditioning is critical for generalization under distribution shifts. This observation motivates Faster-WAM, an efficient future-conditioning WAM that preserves future representations while avoiding expensive video-action interaction. Faster-WAM introduces a sparse future-conditioning framework that computes future representations once and selectively reuses them throughout action denoising. Specifically, we propose SparseMoT to replace ubiquitous layer-wise fusion with selective video-action interaction at a compact subset of network stages, and Interval KV-Fusion to aggregate multi-depth future representations without increasing attention complexity. Experiments demonstrate that Faster-WAM achieves a substantially better performance-efficiency trade-off than existing WAMs. On the out-of-distribution LIBERO-Plus benchmark, Faster-WAM improves success rate from 49.14% to 73.57% compared with Fast-WAM, while running 2.21× faster than Joint-WAM. It further achieves state-of-the-art performance on LIBERO and RoboTwin 2.0, while demonstrating strong robustness in real-world manipulation.
Weiheng Zhao, Haoyi Jiang, Xin Shi +5
Huazhong University of Science and Technology · D-Robotics · Horizon Robotics +1