ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignment

Yuzhi Chen; Ronghan Chen; Dongjie Huo; Yandan Yang; Dekang Qi; Haoyun Liu; Tong Lin; Shuang Zeng

cs.CVMar 24, 2026

ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignment

Authors: Yuzhi Chen, Ronghan Chen, Dongjie Huo, Yandan Yang, Dekang Qi, Haoyun Liu, Tong Lin, Shuang Zeng,

Organizations: AMAP CV Lab Alibaba Group

Abstract

Video-based world models offer a powerful paradigm for embodied simulation and planning, yet state-of-the-art models often generate physically implausible manipulations - such as object penetration and anti-gravity motion - due to training on generic visual data and likelihood-based objectives that ignore physical laws. We present ABot-PhysWorld, a 14B Diffusion Transformer model that generates visually realistic, physically plausible, and action-controllable videos. Built on a curated dataset of three million manipulation clips with physics-aware annotation, it uses a novel DPO-based post-training framework with decoupled discriminators to suppress unphysical behaviors while preserving visual quality. A parallel context block enables precise spatial action injection for cross-embodiment control. To better evaluate generalization, we introduce EZSbench, the first training-independent embodied zero-shot benchmark combining real and synthetic unseen robot-task-scene combinations. It employs a decoupled protocol to separately assess physical realism and action alignment. ABot-PhysWorld achieves new state-of-the-art performance on PBench and EZSbench, surpassing Veo 3.1 and Sora v2 Pro in physical plausibility and trajectory consistency. We will release EZSbench to promote standardized evaluation in embodied video generation.

Figures & tables

Figure 1: Overview of the data curation pipeline. (a) shows the multi-stage filtering and balancing flow from raw aggregation ( $\sim$ 3M clips) to training-ready splits (SFT, RL, and A2V data). (b) Task-aware quota allocation: head tasks are capped at 8–15%, body tasks are uniformly sampled at 40–50%, and long-tail tasks are fully preserved to maximize task diversity. (c) Dataset and robot type distribution: the left ring shows the original composition and the right ring shows the rebalanced result after hierarchical sampling. (d) Physics-aware video captioning pipeline: a perception module (Qwen3-VL 32B) extracts structured physical attributes, followed by a writing module (Qwen3 32B FP8) that generates four-phase captions covering scene setup, action detail, state transition, and camera summary.

Figure 2: Two-stage training pipeline. Stage 1 : SFT on the DiT to predict future frames from observations and instructions. Stage 2 : generate $N$ candidates, score via physics checklist, and apply DPO via LoRA on frozen DiT weights.

Figure 3: Construction pipeline of the EZSbench. Top : dual-source image augmentation—Branch 1 generates synthetic initial observations via text-to-image (Nano Banana) by varying robot morphology, scene, task, and viewpoint; Branch 2 applies VLM-guided background editing to real-world images while preserving foreground interactions. Down : three-stage dense description synthesis—visual anchoring grounds the scene layout and object coordinates, action simulation infers kinematically compliant trajectories with micro-physical interactions, and narrative synthesis produces a documentary-style caption integrating initial state, trajectory, and final state.

Model	AQ	BC	IQ	MS	OC	SC	I2VB	I2VS	Quality Score	Domain Score	Avg.
Wan 2.5	0.5477	0.8985	0.6458	0.9623	0.2190	0.8784	0.9438	0.9428	0.7548	0.8644	0.8096
GigaWorld-0	0.4757	0.9219	0.6506	0.9908	0.1944	0.9111	0.9673	0.9607	0.7591	0.8583	0.8087
Veo 3.1	0.5458	0.9216	0.7244	0.9712	0.2214	0.9146	0.9317	0.9614	0.7740	0.8350	0.8045
Wan2.1_14B	0.4723	0.9315	0.7118	0.9917	0.1921	0.9185	0.9745	0.9451	0.7672	0.8391	0.8032
WoW-wan 14B	0.4664	0.9295	0.7027	0.9858	0.1941	0.9149	0.9613	0.9292	0.7605	0.8301	0.7953
Cosmos-Predict 2.5	0.4897	0.9166	0.7405	0.9906	0.1911	0.8973	0.9304	0.9030	0.7574	0.8021	0.7797

Table 1: Quantitative comparison on the PAI-Bench robot domain subset.

Figure 4: Architecture of the action-conditioned video generation model. We selectively duplicate DiT blocks as parallel context blocks to process action maps, and fuse their outputs residually into the main DiT.

Model	AQ	BC	IQ	MS	OC	SC	I2VB	I2VS	Quality Score	Domain Score	Avg.
WoW-wan 14B	0.4764	0.9412	0.7514	0.9922	0.1236	0.9347	0.9495	0.9178	0.7609	0.7951	0.7780
GigaWorld-0	0.4466	0.8893	0.6565	0.9889	0.1222	0.8678	0.9325	0.9139	0.7272	0.7826	0.7549
Cosmos-Predict 2.5	0.4148	0.8835	0.6810	0.9878	0.0970	0.8366	0.9054	0.8653	0.7089	0.7698	0.7394
UnifoLM-WMA-0	0.4861	0.9575	0.7390	0.9925	0.1046	0.9452	0.8421	0.8170	0.7355	0.5232	0.6294
Our Model	0.4789	0.9494	0.7483	0.9910	0.1301	0.9442	0.9680	0.9453	0.7694	0.8366	0.8030

Table 2: Quantitative comparison on EZSbench.

Figure 5: Qualitative comparison on PAI-Bench.

Model	PSNR	SSIM	Traj. Consis.
Enerverse-AC	20.42	0.7542	0.8157
Gen-Sim	18.05	0.7413	0.6195
Ours	21.09	0.8126	0.8522

Table 3: Quantitative results on action-conditioned generation.

Figure 6: Vision-action alignment verification. Calibrated action signals are rendered as semi-transparent action maps and overlaid onto video frames. Samples with spatial deviation between projected trajectories and observed robot motion are identified and removed.

Figure 7: Stage 1 captioning example 1. Structured perception output showing extracted physical attributes, object identities, and spatial relations.

Figure 8: Stage 1 captioning example 2. The perception module identifies robot morphology, object properties, and contact events from the video sequence.

Figure 9: Stage 1 captioning example 3. Spatial layout parsing and state transition detection across the manipulation trajectory.

Figure 10: Stage 2 captioning example 1. The writing module synthesizes a four-phase narrative covering scene setup, action detail, state transition, and camera summary from the structured perception output.

Figure 11: Stage 2 captioning example 2. Physics-grounded narrative synthesis capturing fine-grained manipulation dynamics and causal state transitions.

Figure 12: Zero-shot qualitative comparison on EZSbench. Baseline models exhibit placement errors (Wan-2.5, Veo 3.1, WoW), contactless grasping (Sora v2), and geometric collapse during contact interactions (Giga R0, Veo 3.1). Our method (bottom row) correctly follows compositional instructions and maintains physical plausibility.

Figure 13: Case study on zero-shot test set. Our model handles diverse unseen manipulation tasks including multi-object attribute binding, deformable object manipulation with dual-arm coordination, articulated object interaction, rigid body placement, contact-intensive wiping, and object relocation—all with strict physical plausibility and spatiotemporal coherence.

Figure 14: Action-to-video qualitative comparison. Our method preserves object geometry and visual integrity during contact-intensive manipulation. Baselines (Genie-Envisioner, Enerverse-AC) produce object deformation, contactless grasping, and localization errors.

Embodied World Models

Embodied world models predict how environments change under an agent’s actions to support planning, learning, simulation, and control.

See all 159 papers

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation
Jun 26, 2026Peiwen Zhang, Yufan Deng, Shangkun Sun +11Physical Consistency in Video Generation Video Diffusion Models
Peking University, NVIDIA
RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation
Apr 21, 2026Feng Jiang, Yang Chen, Kyle Xu +8Robotic Manipulation Robot Manipulation Benchmarks
Peking University · Tsinghua University · Lightwheel
Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation
Jun 15, 2026Jie Zhang, Xiaoyue Chen, Anzhe Chen +36Action-Conditioned Video Generation Video Diffusion Models
Qwen Team

Jun 26, 2026J/K move · Enter open · S save

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation

Peiwen Zhang, Yufan Deng, Shangkun Sun, Juncheng Ma+10

Peking University, NVIDIA

Physical Consistency in Video Generation Video Diffusion Models Robotic Manipulation Embodied World Models

Open paper AskPDF arXiv

Video generation models have emerged as a promising paradigm for embodied world simulation. However, both general-domain video generators and robot-specific data fine-tuned models can still produce physically implausible manipulations, including discontinuous motion trajectories and inconsistent robot-object interactions, which limits their reliability as world simulators. Through extensive experiments, we find that such physical instability mainly arises from two factors: deformation of moving objects and implausible spatio-temporal correlations among interacting entities, particularly during contact. Building on this observation, we propose PhysisForcing, a scalable training framework that strengthens physical consistency by focusing supervision on physics-informative regions through joint optimization of pixel-level and semantic-level features. The framework consists of a pixel-level trajectory alignment loss, which supervises DiT features using reference point trajectories, and a semantic-level relational alignment loss, which aligns DiT features with inter-region relations extracted from a frozen video understanding encoder. Extensive experiments on R-Bench, PAI-Bench, and EZS-Bench show that PhysisForcing consistently improves embodied video generation over strong baselines, improving the Wan2.2-I2V-A14B and Cosmos3-Nano base models on R-Bench by 22.3% and 9.2% (7.1% and 3.7% over vanilla finetuning), with the Cosmos3-Nano variant attaining the best overall score. Beyond generation, as a world model under the WorldArena action-planner protocol it raises the closed-loop success rate from 16.0% to 24.0% and further improves downstream policy success, indicating that physically aligned video models yield stronger representations for robotic manipulation.

Abstract

Embodied World Models

Explore similar work

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation

RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation

Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation