UniWAM: Unified World-Action Model

Jiayi Chen; Wenxuan Song; Jingbo Wang; Shuai Zhou; Xicheng Gong; Zehua Fan; Ziyang Zhou; Junwu E

cs.ROOct 1, 2026

UniWAM: Unified World-Action Model

Authors: Jiayi Chen, Wenxuan Song, Jingbo Wang, Shuai Zhou, Xicheng Gong, Zehua Fan, Ziyang Zhou, Junwu E, +8 more

Abstract

Vision-language-action models benefit from the understanding and reasoning capabilities of pretrained vision-language models, but action-only supervision provides limited grounding in world dynamics. Conversely, world-action models inherit spatiotemporal priors from video generation models, yet remain limited in semantic understanding and reasoning under distribution shifts. We introduce UniWAM, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction. To ensure the quality of the training data, we developed a rigorous data cleaning and annotation pipeline for both human egocentric data and robot data. To adapt the vision-language component to embodied tasks while preserving its inherited language capabilities, we represent low-level actions in natural language and introduce a pre-training recipe that assigns complementary supervision from visual question answering (VQA) data, human egocentric data, and robot demonstrations to the appropriate model components. During post-training, future visual noise augmentation reduces reliance on precise future predictions, while history-conditioned flow matching uses encoded action history to initialize action generation. Together, these designs significantly reduce denoising steps while maintaining performance. UniWAM achieves state-of-the-art (SOTA) performance across multiple evaluations, including in-distribution performance, robustness, generalization, instruction following, and long-horizon task execution. Furthermore, we uncover a log-linear scaling law of unified human-robot co-training, demonstrating the effectiveness of large-scale pre-training on a mixture of human and robot data.

Figures & tables

Figure 1 : UniWAM unifies physical reasoning, visual generation, and action prediction in a unified architecture and pretrains on robot and human egocentric data as well as VQA data. The resulting model exhibits strong robustness, generalization, instruction-following, and reasoning abilities.

Data Type	Embodiment Type	Data Sources	Duration (hours)
Robot	Dual-arm/Mobile	AgiBot World 2026	$\sim$ 891
	Dual-arm/Mobile	AgiBot World Alpha	$\sim$ 595
	Single-arm	Bridge	$\sim$ 80
	Single-arm	Droid	$\sim$ 365
	Single-arm	Fractal	$\sim$ 340
	Dual-arm	Robocoin	$\sim$ 1088

Table 1 : Overview of the training datasets.

QA Type	Data Source	QA Pairs
Spatial Understanding	SAT	172K
	RefSpatial	1.430M
	VST-P	563K
	SenseNova-SI	800K
	GRiD-3D	358K
Grounding	RoboPoint	930K

Table 2 : Composition of the vision–language pretraining data.

Figure 2 : EgoANT annotation pipeline. An untrimmed egocentric video is represented as timestamped contact sheets and segmented through a full-episode coarse pass followed by local temporal refinement. With the refined boundaries fixed, the multi-candidate annotation route generates descriptions using raw-frame annotation, an alternative FFmpeg decoding and frame-sampling path, and relabeling conditioned on seed or prior descriptions. A text-only selector chooses the final description without re-examining the video. The output consists of temporal intervals paired with concise subtask descriptions, which we use to annotate EgoDex.

Stage	Input / Configuration	Model
Global segmentation	Full-episode timestamped contact sheets; $0.5$ s sampling, up to 20 frames per sheet, and completed-event rules	Qwen3.6-27B
Local refinement	Local timestamped contact sheets, coarse hypothesis, and episode-level context; no additional window padding	Qwen3.6-27B
Segment annotation	Fixed temporal segments; raw-frame, FFmpeg-based, and seed- or prior-conditioned annotation passes	Qwen3.5-397B
Candidate selection	Candidate description texts only; no video frames	Qwen3.5-397B

Table 3: EgoANT configuration for human-data annotation. Each row summarizes the inputs and model used at one stage. Candidate selection uses the listed model in text-only mode.

Figure 3 : Overview of our UniWAM. (a) UniWAM comprises a physical reasoner, a world generator, and an action predictor interconnected through joint multimodal attention. Robot data and human egocentric data are jointly used to train all three experts, with supervision for the physical reasoner provided in the form of physical language. General VQA and Embodied Question Answering (EQA) data are additionally used to train the physical reasoner. (b) Overview of the pretraining data: Human and robot data are measured in hours; vision–language data are measured in millions of samples. Percentages are computed within each data category.

Figure 4 : History-conditioned action denoising. Action generation starts from perturbed action history rather than Gaussian noise.

Method	Spatial	Object	Goal	Long	Average $\uparrow$
$\pi_{0}$ ( Black et al., 2025b )	98.0	96.8	94.4	88.4	94.4
PD-VLA ( Song et al., 2025 )	95.5	96.7	94.9	91.7	94.7
$\pi_{0.5}$ ( Black et al., 2025a )	98.8	98.2	98.0	92.4	96.9
GR00T-N1.7 ( NVIDIA, 2026 )	97.7	98.5	97.5	94.4	97.0
OpenVLA-OFT ( Kim et al., 2025a )	97.6	98.4	97.9	94.5	97.1
Fast-WAM ( Yuan et al., 2026 )	98.2	100.0	97.0	95.2	97.6

Table 4 : Success rate (%) on the standard LIBERO suites.

Method	C2C	C2R	Average
VLA
GR00T-N1.7 ( NVIDIA, 2026 )	43.6	20.7	32.2
StarVLA ( StarVLA Community, 2026 )	58.1	10.6	34.4
Xiaomi Robotics-0 ( Cai et al., 2026b )	62.90	18.20	40.55
Abot-M0 ( Yang et al., 2026c )	57.40	30.36	43.88
X-VLA ( Zheng et al., 2026a )	68.00	20.90	44.45

Table 5 : Mean task success rate (%) on the 50-task RoboTwin 2.0 evaluation. C2C and C2R denote Clean2Clean and Clean2Random, respectively; gray shading indicates C2R. Bold and underlined entries denote the highest and second-highest values in each column across both groups. Methods are grouped into vision-language-action (VLA) and world action model (WAM) families.

Method	Camera	Robot	Language	Light	Background	Noise	Layout	Total
$\pi_{0}$ ( Black et al., 2025b )	79.6	21.1	72.5	84.7	86.2	68.3	69.4	67.4
GR00T-N1.6 ( NVIDIA GEAR Team, 2025 )	92.6	33.5	80.1	93.6	95.4	93.6	75.0	79.4
OpenVLA-OFT ( Kim et al., 2025a )	92.8	30.3	85.8	94.9	93.9	89.3	77.6	79.5
Spatial Forcing ( Li et al., 2026a )	95.2	47.9	73.5	91.2	95.6	92.2	74.8	80.5
MemoryVLA ( Shi et al., 2026 )	91.4	48.6	79.4	95.2	95.3	94.0	75.7	81.9
$\pi_{0.5}$ ( Black et al., 2025a )	87.6	78.4	80.0	92.6	91.6	91.4	81.8	86.2

Table 6 : Success rate (%) under all seven LIBERO-Plus perturbation dimensions (OOD evaluation). Bold entries denote the highest value in each column among the listed methods. Underlined entries denote the second-highest value in each column.

Figure 5 : Overview of our real-world evaluation tasks. The top panel shows representative execution sequences for four instruction-following tasks: Pick-Anything, Diverse-Interaction, Place-Relative, and Drawer Storage. The bottom panel presents a long-horizon task, where the robot sequentially completes multiple manipulation stages.

Figure 6 : Real-world instruction-following evaluation across four manipulation tasks.

Figure 7 : Data scaling comparison between Robot and Robot + Human. Left: validation MSE. Right: RoboTwin clean2rand success rate. The x-axis reports the number of robot-data hours on a logarithmic scale. Robot + Human augments the robot data with a fixed proportion of human data. At the final data point, the training set comprises 4958 hours of robot data and 5072 hours of human data. All settings use mixed training with VQA data.

Figure 8 : Human–robot transfer emerges with physical-language pretraining. t-SNE visualization of physical reasoner representations from robot (purple) and human (gray) data without (left) and with (right) physical language supervision. The representations exhibit greater overlap with supervision.

Figure 9 : Ablation studies on RoboTwin 2.0.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10 : Real-world experimental setup. Our setup consists of two robot arms, three cameras for observation, and a diverse set of everyday objects used in the manipulation experiments.

Figure 11 : Action generation performance across denoising steps. N2A and A2A denote noise-to-action and action-to-action generation, respectively.. The endpoint of the solid portion corresponds to the N2A success rate, while the endpoint of the hatched portion corresponds to the A2A success rate. All success rates are reported in percent.

Task	C2R	C2C	Average	Task	C2R	C2C	Average
adjust_bottle	72	72	72.00	place_can_basket	35	36	35.50
beat_block_hammer	56	65	60.50	place_cans_plasticbox	86	97	91.50
blocks_ranking_rgb	67	64	65.50	place_container_plate	93	97	95.00
blocks_ranking_size	38	36	37.00	place_dual_shoes	77	87	82.00
click_alarmclock	92	100	96.00	place_empty_cup	94	98	96.00
click_bell	99	100	99.50	place_fan	69	76	72.50

Appendix

Table 7: Task-level UniWAM success rates (%) on RoboTwin 2.0. C2R denotes clean2rand and C2C denotes clean2clean. C2C and C2R values are reproduced from the supplied experiment record. Average is their arithmetic mean.

Explore similar work

May 12, 2026cs.RO

World Action Models: The Next Frontier in Embodied AI

Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning, yet they learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention. A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline. We term this emerging paradigm World Action Models (WAMs): embodied foundation models that unify predictive state modeling with action generation, targeting a joint distribution over future states and actions rather than actions alone. However, the literature remains fragmented across architectures, learning objectives, and application scenarios, lacking a unified conceptual framework. We formally define WAMs and disambiguate them from related concepts, and trace the foundations and early integration of VLA and world model research that gave rise to this paradigm. We organize existing methods into a structured taxonomy of Cascaded and Joint WAMs, with further subdivision by generation modality, conditioning mechanism, and action decoding strategy. We systematically analyze the data ecosystem fueling WAMs development, spanning robot teleoperation, portable human demonstrations, simulation, and internet-scale egocentric video, and synthesize emerging evaluation protocols organized around visual fidelity, physical commonsense, and action plausibility. Overall, this survey provides the first systematic account of the WAMs landscape, clarifies key architectural paradigms and their trade-offs, and identifies open challenges and future opportunities for this rapidly evolving field.

Siyin Wang, Junhao Shi, Zhaoyang Fu +11

1Fudan University · 2Shanghai Innovation Institute · 3National University of Singapore

Recent World-Action Models World Models Embodied Artificial Intelligence Action Generation Next-State Prediction Egocentric Video+1

Ask PDF

Jun 17, 2026cs.CV

ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?

World Action Models (WAMs) commonly rely on video generation to bridge visual world modeling and robot control. However, video-based WAMs face three coupled limitations: dense multi-frame future tokens make inference costly, full video prediction spends capacity on action-irrelevant temporal and appearance details, and long-horizon future imagination may introduce errors that mislead action prediction. These issues raise a simple question: Does world action model really need video generation? We propose ImageWAM, a simple WAM framework that repurposes pretrained image editing models for robot action prediction. In contrast to video generation, image editing provides a better-matched prior: it only needs to model a target-frame transformation, focuses on action-relevant current-to-target visual differences, and grounds task instructions to localized visual changes through edit pretraining. In practice, ImageWAM does not decode the target frame at inference time; instead, it conditions a flow-matching action expert on the KV caches produced by image-editing denoising, using them as a compact world-action context. ImageWAM outperforms standard VLA baselines and matching competitive WAMs without additional policy pretraining across different simulator and real-world experiments. It also reduces FLOPs to 1/6 and latency to 1/4 of video-based WAMs. Attention analysis further shows that editing caches focus on task-relevant change regions, supporting image editing as an effective alternative to video-based world-action modeling.

Yuyang Zhang, Wenyao Zhang, Zekun Qi +7

Shanghai Jiao Tong University · Tsinghua University · Tencent Robotics X +1

World Models Video Generation Image Editing Query Image

Ask PDF

Sep 22, 2026cs.RO

An Action Is Worth One Patch: Unified World-Action Modeling with PatchWAM

Generative visual models offer a foundation for learning representations of physical dynamics, yet their extension to continuous control raises a fundamental question: do visual prediction and action generation require separate computational pathways? Existing approaches usually introduce trainable action heads or separate action experts to bridge low-dimensional states and high-dimensional visual representations. In this work, we explore whether the visual backbone's existing capacity can also support control when actions are expressed in a compatible representation. Thus, we introduce PatchWAM (Patch World-Action Model), which treats continuous actions as another type of patch through a fixed mapping called Action-as-Patch. This allows a single model to predict both how the robot should move and what the scene may look like afterward. Visual prediction and action generation become parts of the same generative process, without a dedicated action head or separate action expert. Experiments with subsampled training windows show gains over a matched dual-expert control, while benchmark evaluations reach 91.8% success rate on LIBERO-Plus and 96.12% on RoboTwin 2.0 in a full-data setting with additional augmented demonstrations. More broadly, the result suggests that capability need not be added where it can be inherited: the constraint on extending a generative backbone is the interface a new signal is written in, not the capacity to model it.

Tianheng Wang, Zhou Xie, Heng Jia +3

Westlake University · Lanzhou University · Zhejiang University +2

Action Generation Visuomotor Control World Models Robot Systems

Ask PDF