cs.CVAug 25, 2026

GlanceWAM: Sparse Test-Time Imagination for World-Action Models

Authors: Linhan Wang, Zijian An, Mingyuan Zhang, Chen Dai, Yi Xu, Can Cui, Jiayan Wang, Zichong Yang, +3 more

Organizations: Virginia Tech · Drexel University · Northeastern University · Purdue University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchronous video generation at control rate is latency-prohibitive, while abandoning test-time visual imagination sacrifices task success. We show that visual imagination achieves both real-time inference and superior success rates when generated asynchronously off the critical path and consumed directly in latent space. We introduce GlanceWAM, which decouples imagination from control on a single shared video DiT backbone: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate (48 ms) purely in latent space without blocking. Enabled by a non-interfering attention mask that isolates video representations and staleness-robust horizon training that accommodates asynchronous lookahead aging, GlanceWAM breaks the speed-success dilemma. Trained purely on demonstrations, it attains 72.2% on the 24-task RoboCasa kitchen benchmark (vs. 67.1% for synchronous Cosmos Policy) and 99.0% on LIBERO while cutting per-chunk control latency 24×24\times relative to synchronous world-action models (48 ms on one A100). In single-arm and bimanual real-robot manipulation, it achieves higher average success than π0.5π_{0.5} without any robot-data pretraining. Code is available at https://github.com/linhanwang/GlanceWAM.

Figures & tables

Figure 1: Synchronous imagination versus sparse lookahead foresight. (a) Existing WAMs either skip test-time foresight or imagine video synchronously with every action chunk, paying KK heavy video denoising steps per chunk. (b) GlanceWAM glances ahead asynchronously to a single latent lookahead frame Hf≈3H_{f}\approx 3 s away and reuses it across ≈4{\approx}4 action chunks, each decoded in 4848 ms. (c) Success rate on RoboCasa kitchen (24 tasks, demos only).
Figure 2: GlanceWAM architecture. (a) In co-training, the video DiT predicts the future target xt+Hf\mathbf{x}_{t+H_{f}} , while the action head learns inverse dynamics toward a lookahead frame xt+u\mathbf{x}_{t+u} at a random offset u∼U(0,Hf]u\sim\mathcal{U}(0,H_{f}] , conditioned on Δ=u\Delta=u , so it sees every lookahead age it will meet at deployment. Two separate VAE passes and the prefix-LM mask M\mathbf{M} keep the lookahead out of video prediction. (b) At inference, the lookahead latent z^la\hat{\mathbf{z}}_{\text{la}} is generated asynchronously once per HfH_{f} and never decoded; each 0.80.8 s action chunk reads the held z^la\hat{\mathbf{z}}_{\text{la}} at the decaying offset Δ\Delta in 4848 ms, within the offset range seen in training.
Figure 3: Non-interfering attention mask. 3-class prefix-LM mask M\mathbf{M} .
Figure 4: Imagined lookaheads. Observation ot\mathbf{o}_{t} , the GlanceWAM lookahead z^la\hat{\mathbf{z}}_{\text{la}} (decoded for display only), and the real frame xt+3 s\mathbf{x}_{t+3\,\text{s}} , in RoboCasa kitchen (top) and real IMETA Y1 rollouts (bottom). The lookahead gets the target layout right while coarsening texture. Wrist view inset; more in Figure 10 .
Table 5
Figure 5: Real-robot evaluation on IMETA Y1. Success of GlanceWAM, π0.5\pi_{0.5} , and Fast-WAM over 50 trials per task; thumbnails show one GlanceWAM rollout per task.
(a) SystemSR (%)
Cosmos Policy (external anchor)67.1
plain co-training (no lookahead)64.4
GlanceWAM, single-layer lookahead71.5
same checkpoint, lookahead zeroed61.6
same checkpoint, offset Δ\Delta pinned †61.8
retrained without mask (bidirectional)47.0
Table 3: Internal comparison on RoboCasa kitchen (24-task SR, held-out scenes, demos only; shared backbone, data, protocol). (a) Components. (b) Lookahead sampler steps, single-layer checkpoint, paired re-evaluation ( 1,2001{,}200 episodes/point). † Correct lookahead, Δ\Delta fixed to one frame; paired sweep, 1,200 episodes (baseline 71.371.3 , zeroed 62.762.7 ).
Figure 8
Figure 8: Latency vs. success on LIBERO. Per-chunk latency ( B=1B{=}1 ); marker area: model size.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

 RoboCasaLIBEROAuxiliary supervisionWorld model
 kitchen(avg)beyond RGB demosat test time
DeVA ( Zhang et al., 2026 )72.099.0affordance + depth decoders (contact labels, depth model)joint denoising, every chunk
w/o guidance (their ablation)66.0—none(same)
Flex- π\pi ( Yan et al., 2026 ) , action-only—98.73D pointmaps + DINOv3 semantic futuresdropped (fast path)
Flex- π\pi , full joint—99.2(same)joint denoising, every chunk (slow path)
GlanceWAM (ours)72.299.0noneasync lookahead frame, ∼\sim 3 s
Appendix
Table 4: Comparison with concurrent world-action models. Benchmark numbers as reported by each paper under its own evaluation protocol. The DeVA ablation row removes its affordance and depth decoders, leaving RGB demonstrations only, as in GlanceWAM. Flex- π\pi does not report RoboCasa kitchen; its two LIBERO rows are the reported FLEX- π∗\pi^{*} deployment modes.
Figure 9: Where the action head reads the lookahead (extends Figure 7 ). Two probed contexts, third-person view of the decoded lookahead z^la\hat{\mathbf{z}}_{\text{la}} . Cross-attention: value-weighted contribution of each lookahead token to the action queries’ cross-attention output, i.e. the norm of its attention-weighted value after the output projection. The value-weighted share in § 4.4 is ∥lookahead part∥/(∥obs part∥+∥lookahead part∥)\|\text{lookahead part}\|/(\|\text{obs part}\|+\|\text{lookahead part}\|) of the attention output. Action sensitivity: action shift when a 2×22{\times}2 -token lookahead patch is overwritten with the observation tokens at the same location. Each map is normalized independently.
Figure 10: Lookahead frames on eight kitchen tasks (extends Figure 4 ). For eight RoboCasa kitchen tasks (articulation, knobs, buttons, pick-and-place): the observation at tt , the GlanceWAM lookahead, and the environment frame at t+3t+3 s. Lookaheads get the target layout right (closed doors, objects on target surfaces) while coarsening texture; they are decoded to RGB only for display. One third-person view per panel, wrist view inset.
Figure 11: Generated lookaheads vs. sampler budget KK (RoboCasa kitchen demonstration contexts, matched seeds per row; last column: the demonstration frame at t+Hft{+}H_{f} ). Fidelity improves from K=1K{=}1 to K=10K{=}10 (the one-step lookahead smears the arm and the manipulated object, most visibly in the wrist insets), yet success barely changes over K=1K=1 – 3030 (Table 3 b). Layout as in Figure 10 .
Figure 12: Real-robot rollout keyframes. Initial, grasp, and final frames from successful GlanceWAM rollouts. Single-arm rows ( stack cups , pick and place , remove cup cap ) are side-view recordings cropped to the workspace; bimanual rows ( stack cups , bowl ) are frames from the policy’s top camera.

Explore similar work

CardsList

  1. ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?

    Jun 17, 2026Yuyang Zhang, Wenyao Zhang, Zekun Qi +7World ModelsVideo Generation

    Shanghai Jiao Tong University · Tsinghua University · Tencent Robotics X +1

  2. Latent Action as Intention Enables Efficient Future Imagination for World Action Models

    Aug 25, 2026Xiang Li, Yupeng Zheng, Songen Gu +11Efficient World-Action ModelLatent Actions

    1CollegeAI, THU · 2AIR, THU · 4TARS Robotics +6

  3. Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination

    Jun 8, 2026Jiajun Li, Tiecheng Guo, Yifan Ye +9Efficient World-Action ModelFaster-Wam

    1The University of Hong Kong · 2Peking University · 3Muka Robotics +2