AWAM: Active Trajectory-Conditioned View Imagination for Robust Robot Manipulation

Jiawei Geng; Zhenyang Liu; Yuzheng Wu; Yanwei Fu

cs.ROOct 16, 2026

AWAM: Active Trajectory-Conditioned View Imagination for Robust Robot Manipulation

Authors: Jiawei Geng, Zhenyang Liu, Yuzheng Wu, Yanwei Fu

Organizations: Fudan University · Shanghai Innovation Institute

Abstract

Rapid progress in embodied intelligence has enabled increasingly capable robots to follow language instructions and manipulate diverse objects. However, strong results in structured environments do not automatically transfer to open-world conditions, where clutter, occlusion, and changing viewpoints can obscure the spatial relationships required for action. To address this perceptual gap, a virtual active-perception paradigm is introduced. AWAM, an Active World-Action Model actively plans where to observe and imagines what will be seen along the planned trajectory before acting. Task-related virtual camera motion is predicted and used as a structured query into a geometry-conditioned world model, producing a reusable spatial prior for manipulation, which is later compressed into a fixed memory and fused with live observations through cross-attention for diffusion-based action prediction. AWAM achieves competitive success rates with 75.9% on LIBERO-Occ, 62.4% on LIBERO-Plus, 76.5% on RLBench-OG Occlusion1, and 94.4% in real-robot trials. These results support improved robustness to the evaluated occlusions and distribution shifts, addressing part of the gap between structured manipulation and less controlled environments.

Figures & tables

Figure 1: Overview of AWAM. From partially observed initial RGB-D inputs, AWAM imagines task-conditioned virtual views and compresses them into a predictive memory once per episode. The policy then reuses this memory together with live visual observations for closed-loop manipulation, without physically executing the predicted camera trajectory.

Figure 2: Overview of AWAM. A task-guided virtual camera trajectory guides a geometry-conditioned world model to construct predictive memory once per episode. During closed-loop control, the cached world memory is fused with live visual observations to predict action chunks.

Method	Spatial	Object	Goal	Long	Avg.
$\pi_{0}$	81.0	84.0	76.6	38.4	70.0
Motus	82.2	78.0	66.0	43.4	67.4
Fast-WAM	82.8	86.4	78.8	44.2	73.0
VIM	83.0	92.0	81.0	40.0	74.0
AWAM	84.6	93.2	84.8	41.0	75.9

Table 1: Success rate (%) on LIBERO-Occ. Each suite contains 500 evaluation rollouts.

Method	Avg. $\uparrow$	Avg. Rank $\downarrow$	basketball in hoop	scoop with spatula	take plate off dish rack	water plants
Diffusion Policy	$27.4{\pm}0.8$	$4.70$	$0.0{\pm}0.0$	$0.0{\pm}0.0$	$4.0{\pm}0.0$	$2.0{\pm}2.0$
ARP	$73.0{\pm}1.6$	$2.80$	$\mathbf{100.0{\pm}0.0}$	$88.0{\pm}4.0$	$\mathbf{100.0{\pm}0.0}$	$66.0{\pm}14.0$
RVT2	$72.8{\pm}2.0$	$3.05$	$\mathbf{100.0{\pm}0.0}$	$82.7{\pm}7.5$	$98.7{\pm}1.9$	$72.0{\pm}8.6$
TVVE	$75.0{\pm}1.2$	$2.40$	$\mathbf{100.0{\pm}0.0}$	$88.0{\pm}8.0$	$\mathbf{100.0{\pm}0.0}$	$88.0{\pm}4.0$
AWAM	$\mathbf{76.5{\pm}1.1}$	$\mathbf{2.05}$	$\mathbf{100.0{\pm}0.0}$	$\mathbf{92.0{\pm}3.3}$	$\mathbf{100.0{\pm}0.0}$	$\mathbf{94.7{\pm}1.9}$

Table 2: RLBench-OG Occlusion1 success (%, mean

\pm

std. over three runs). Avg. Rank is averaged across 10 tasks; lower is better.

Method	Camera	Robot	Language	Light	Background	Noise	Layout	Total
$\pi_{0}$	13.8	6.0	58.8	85.0	81.4	79.0	68.9	53.6
Motus	17.9	43.3	74.1	54.7	44.9	26.2	52.0	44.0
Fast-WAM	18.8	44.5	68.9	78.2	45.7	29.8	60.7	48.2
AWAM	53.4	48.9	61.0	84.3	61.1	60.7	73.0	62.4

Table 3: Success rate (%) on all 10,030 LIBERO-Plus instances; Total is instance-weighted.

Figure 3: Real-world rollouts of AWAM under partial occlusion and clutters

Method	apple in box	bottle in cluster	block behind box	Overall
$\pi_{0}$	22/30(73.3%)	24/30(80.0%)	20/30(66.7%)	66/90(73.3%)
Motus	24/30(80.0%)	26/30(86.7%)	23/30(76.7%)	73/90(81.1%)
Fast-WAM	24/30(80.0%)	25/30(83.3%)	23/30(76.7%)	72/90(80.0%)
AWAM	28/30 (93.3%)	30/30 (100.0%)	27/30 (90.0%)	85/90 (94.4%)

Table 4: Real-world manipulation, successes/trials (success rate, %).

Variant	Learned trajectory	Generation	Memory	Success (%) $\uparrow$	$\Delta$
(a)Live observations only	$\times$	$\times$	$\times$	64.4	-11.5
(b)Trajectory-conditioned live policy	✓	$\times$	$\times$	67.3	-8.6
(c)sampled spherical views	$\times$	✓	✓	70.2	-5.7
(d)Learned path, warp only	✓	$\times$	✓	69.0	-6.9
AWAM	✓	✓	✓	75.9	–

Table 5: Component analysis on LIBERO-Occ.

\Delta

denotes the change from full AWAM in percentage points.

Figure 4: Virtual-view predictions along the planned trajectory. Warped inputs (orange), predictions (blue), and ground truth (green) show progressively improved visibility; RGB decodes are diagnostic only.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Setting
Virtual-view queries	21 viewpoints
World feature resolution	$16\times 16$ tokens per view
World feature width	5120
Memory feature width	384
Memory encoder	2 layers, 8 attention heads
Compressed memory	294 tokens

Appendix

Table 6: Reference architectural settings used in the LIBERO implementation of AWAM. These values specify the implementation used in our experiments and are reported for reproducibility.

Figure 5: Example rollouts on LIBERO-Occ.

Figure 6: Example rollouts on RLBench-OG Occlusion1.

Figure 7: Third-person views (for record only) of real-robot inference on three real-world tasks, with one early and one late frame per task.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,615 papers

OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation
May 7, 2026Yushan Liu, Peibo Sun, Shoujie Li +7World Models for Robotics Action-Conditioned World Models
Tsinghua University · Shanghai Jiao Tong University · Nanyang Technological University
ActiveWAM: Evidence-Aware Active Vision for World-Action Models
Oct 1, 2026Renjun Wu, Luzhou Ge, Xuesong LiWorld Action Models Multi-Task Robotic Manipulation
Beijing Institute of Technology
AR-WAM: A Visual-Conditioned Agent-Ready World Action Model for Robotic Manipulation
Sep 20, 2026Yicheng Jiang, Zesen Gan, Xiaobo Wang +8Robotic Control Long-Horizon Robotic Manipulation
The Hong Kong University of Science and Technology · Shenzhen University of Advanced Technology · Sangfor Technologies Inc. +4

May 7, 2026J/K move · Enter open · S save

OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation

Yushan Liu, Peibo Sun, Shoujie Li, Yifan Xie+6

Tsinghua University · Shanghai Jiao Tong University · Nanyang Technological University

World Models for Robotics Action-Conditioned World Models Object-Centric World Models Object-Centric Representation Learning Robotic Manipulation Vision-Language-Action Models+1

Open paper AskPDF arXiv

World Action Models (WAMs) enhance Vision-Language-Action policies by jointly predicting scene evolution and robot actions, but existing methods usually represent the predicted world as holistic images, video tokens, or global latents. These representations are difficult for an action decoder to address when an instruction refers to a particular object, especially under scene shifts where object identity is entangled with context. We propose OA-WAM, an Object-Addressable World Action Model for robust robot manipulation. OA-WAM decomposes each frame into N+1 slot states, with one robot slot and N object slots. Each slot contains a persistent address vector and a time-varying content vector, and is fused with text, image, proprioception, and past-action tokens in a block-causal sequence. A world head predicts next-frame slot states, while a flow-matching action head decodes a 16-step continuous action chunk in the same forward pass. Addressability is enforced by routing cross-slot attention through address-only keys and resetting the address slice at every transformer layer, separating which object to act on from what that object currently is without adding extra tokens. OA-WAM matches strong VLA and WAM baselines on LIBERO (97.8%) and SimplerEnv (79.3%), reaches state-of-the-art performance on the most relevant LIBERO-Plus geometric axes, and remains competitive on the seven-axis aggregate. A causal slot-intervention test yields a swap-binding cosine of 0.87, versus at most 0.09 for holistic baselines. These results suggest that addressable object states provide an effective interface for robust world-action modeling under scene perturbations.

Abstract

Robotic Manipulation

Explore similar work

OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation

ActiveWAM: Evidence-Aware Active Vision for World-Action Models

AR-WAM: A Visual-Conditioned Agent-Ready World Action Model for Robotic Manipulation