AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation

Ge Yuan; Qiyuan Qiao; Jing Zhang; Dong Xu

cs.ROFeb 23, 2026

AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation

Authors: Ge Yuan, Qiyuan Qiao, Jing Zhang, Dong Xu

Organizations: The University of Hong Kong · Beihang University

Abstract

Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. In this work, we introduce a unified framework, World-Model-Driven Diffusion Policy with Online Adaptive Learning (AdaWorldPolicy) to enhance robotic manipulation under dynamic conditions with minimal human involvement. Our core insight is that world models provide strong supervision signals, enabling online adaptive learning in dynamic environments, which can be complemented by force-torque feedback to mitigate dynamic force shifts. Our AdaWorldPolicy integrates a world model, an action expert, and a force predictor-all implemented as interconnected Flow Matching Diffusion Transformers (DiT). They are interconnected via the multi-modal self-attention layers, enabling deep feature exchange for joint learning while preserving their distinct modularity characteristics. We further propose a novel Online Adaptive Learning (AdaOL) strategy that dynamically switches between an Action Generation mode and a Future Imagination mode to drive reactive updates across all three modules. This creates a powerful closed-loop mechanism that adapts to both visual and physical domain shifts with minimal overhead. Across a suite of simulated and real-robot benchmarks, our AdaWorldPolicy achieves state-of-the-art performance, with dynamical adaptive capacity to out-of-distribution scenarios.

Figures & tables

Figure 1: An overview of our AdaWorldPolicy with Adaptive Online Learning (AdaOL). At timestep $t$ , our AdaWorldPolicy Network operates in two modes. Mode I (Action Generation) : AdaWorldPolicy network acts as an action policy generator $P^{\text{policy}}(a|o)$ , which takes the current multi-modal observation $o$ (from static/gripper cameras and force sensor) to generate an action $a$ . This action is then executed by the robot. During offline training, this step is supervised by the imitation loss $\mathcal{L}_{1}$ (see Section 3.3 ). Mode II (Future Imagination) : Subsequently, our AdaWorldPolicy network turns into an action-conditioned world model $P^{\text{imagine}}(o^{\prime}|o,a)$ which takes the same observation $o$ and the executed action $a$ to predict an Imagined Observation at timestep $t+1$ . The core of our AdaOL strategy lies in the online updating loop (red arrows). The discrepancy between the Imagined Observation and the real Observation at timestep $t+1$ (e.g., under in-domain setup or under domain shifts like lighting or pose variations) is quantified by a prediction loss $\mathcal{L}_{2}$ . This loss drives an online update to a small subset of shared network parameters, creating a closed-loop system that continuously adapts to real-world dynamics.

Figure 2: Network architecture and workflow of AdaWorldPolicy. Our unified multi-modal framework builds upon a shared multi-modal transformer backbone. It synergistically integrates three modules: a World Model for visual prediction, a Force Predictor for physical dynamics modeling, and an Action Model for policy generation. All modules are implemented as Flow Matching Diffusion Transformers (DiT) and interact through a shared Multi-modal Self-attention layer. Input modalities (vision, action, force, text) are first encoded, conditioned with global features (text, state, noise level) via the adaLN module, and then processed by the shared multi-modal self-attention layer. In our framework, the operational mode is determined by a switch on the action input: in Mode I (Action Generation) , the action token is provided as noise for the model to generate an action; in Mode II (Future Imagination) , a known action is provided as a condition for future prediction. A LoRA-based mechanism enables efficient online updates of a small set of parameters.

Figure 3: Input and output details of our unified model in two different modes. Mode I (Action Generation): The model takes an observation history $o$ (e.g., context length $T_{c}=5$ ) and predicts a future action sequence $\{a_{t},a_{t+1},\cdots,a_{t+T_{\text{a}}}\}$ (e.g., action horizon $T_{a}=8$ ). At test time, the robot executes this predicted action sequence. Mode II (Future Imagination): The model is conditioned on both the observation history $o$ and a ground-truth action sequence $a$ , then predicts the corresponding future observation sequence $\hat{o}^{\prime}$ . The discrepancy between this prediction and real environmental feedback is used to update the network parameters during our Adaptive Online Learning (AdaOL) phase.

Figure 4: Visualizations of the simulated benchmarks used in our experiments: Variant PushT for out-of-distribution robustness, LIBERO for long-horizon skills, and the CALVIN benchmark for language-conditioned tasks across different domains.

Methods	Pub.	Static Camera	Gripper Camera	Joint States	Success
UVA [ 25 ]	RSS’25	✓			0.89
AWP	Ours	✓			0.91
OpenVLA [ 23 ]	CoRL’24	✓	✓	✓	0.54
SpatialVLA [ 36 ]	RSS’25	✓	✓	✓	0.56
Pi0-fast [ 35 ]	RSS’25	✓	✓	✓	0.60
FlowVLA [ 49 ]	arXiv’25	✓	✓	✓	0.73

Table 1: Comparison of success rates on the LIBERO-10 benchmark, under two settings: with only a static camera, and with full multi-modal inputs. Note: AWP is the short name of our method AdaWorldPolicy without online adaptive learning.

Figure 5: Overview of our real-robot evaluation. Our experimental setup (center) features an INOVO robotic arm with multi-modal sensing capabilities, including gripper/static cameras and a force sensor. We test our AdaWorldPolicy on four diverse manipulation tasks (left), such as sweeping beans and placing eggs. To specifically evaluate the effectiveness of our AdaOL strategy, we introduce a variety of challenging out-of-distribution (OOD) shifts during execution (right). These include visual perturbations like drastic lighting and texture changes, as well as physical perturbations like swapping objects and altering the workspace geometry (e.g., tilting the whiteboard).

Methods	Pub.	Original	Texture	Rand Light	Rand Color
DP [ 12 ]	IJRR’24	0.78	0.18	0.14	0.11
OpenVLA [ 23 ]	CoRL’24	0.35	0.22	0.20	0.14
UniPi [ 13 ]	NIPS’23	0.42	0.35	0.33	0.18
UVA [ 25 ]	RSS’25	0.94	0.11	0.54	0.13
AWP	Ours	0.97	0.47	0.71	0.61
AWP (ol)	Ours	0.98	0.51	0.77	0.66

Table 2: Performance on the Variant PushT benchmark under different distribution shifts. We report the mean Intersection-over-Union (IoU) results. Note: AWP/AWP (ol) is the short name of our method AdaWorldPolicy without/with adaptive online learning.

Methods	Pub.	Success Rate (%) for Task Length					Avg. Len.
Methods	Pub.	1	2	3	4	5	Avg. Len.
DP [ 12 ]	IJRR’24	62.2	30.9	13.2	5.0	1.6	1.13 $\pm$ 0.02
MDT [ 39 ]	RSS’24	61.7	40.6	23.8	14.7	8.7	1.54 $\pm$ 0.04
RoboFlamingo [ 26 ]	ICLR’24	82.4	61.9	46.6	33.1	23.5	2.47 $\pm$ 0.00
GR-1 [ 47 ]	ICLR’24	85.4	71.2	59.6	49.7	40.1	3.06 $\pm$ 0.00
OpenVLA [ 23 ]	CoRL’24	91.3	77.8	62.0	52.1	43.5	3.27 $\pm$ 0.00

Table 3: Performance on the CALVIN benchmark for the cross-domain setting (ABC for training, D for evaluation). We report the success rate (%) for different task lengths (the number of consecutive instructions) and the average length of successfully completed sub-sequences (Avg. Len.). We compare with the baseline methods without pretraining on any extra human demonstration data outside the benchmark. Note: AWP/AWP (ol) is the short name of our method AdaWorldPolicy without/with adaptive online learning.

Figure 6: Real-world evaluation results under domain shifts. Our base model, AdaWorldPolicy (or AWP in short), shows a strong performance in the original environment (left). When suffering from various visual and physical shifts at test time (right), our full model with online adaptive learning, AWP (ol) , consistently and significantly improves the success rate, showcasing robust online adaptation.

Configuration	Success Rate (%)
AdaWorldPolicy w/ AdaOL (Full)	76.3
AdaWorldPolicy w/o AdaOL	72.5
w/o Force Predictor	53.8
w/o World Model Supervision	46.3
MMSA $\rightarrow$ Concatenation	36.3
MMSA $\rightarrow$ Cross-Attention	50.0

Table 4: Ablation study on the main components of AdaWorldPolicy. We report the average success rate (%) across four real-world in-domain tasks. Our full method with online adaptive learning achieves the best performance, while removing or replacing any key component leads to a significant performance drop, which validates our design choices.

Configuration	#Sampling Steps $\mathbb{\downarrow}$	Success Rate (%) $\mathbb{\uparrow}$
AWP	20	96.33
	10	95.53
	5	94.67
	2	94.00
+ AdaOL	10	96.05
MMSA	10	95.53

Table 5: Ablation study on sampling steps, adaptation, and fusion mechanisms. We analyze the impact of inference sampling steps, the effectiveness of our AdaOL, and the choice of multi-modal fusion strategy. While reducing sampling steps slightly lowers performance, our method remains robust. Our AdaOL provides a consistent improvement. In addition, our MMSA module significantly outperforms the standard fusion baselines (Concatenation and Cross-Attention), validating its design. Note: AWP denotes our AdaWorldPolicy without using our online adaptive learning strategy (AdaOL). The symbol “

\underline{~~~}

” indicates that the result was reported in the main paper.

Figure 7: Visualization of imagined future frames in real-world scenarios generated by our AdaWorldPolicy. In each subfigures (a-d), we compare the imagined future frames (bottom row) with the actual real observations (top row) across various simulation and real-world tasks. In real-world scenarios (a-d), while the model generally captures the dynamics, some artifacts are observable due to the limitations of the base Cosmos-Predict2 model [ 3 ] , particularly in complex scenes with cluttered backgrounds or numerous objects, such as the egg manipulation task in (b).

Benchmark	Image Size	History Length	Action Horizon	#Imagined Frames
Libero10	$128\times 128$	5	20	20
PushT	$256\times 256$	5	20	20
CALVIN	$192\times 192$	1	12	12
Real-world	$112\times 160$	1	32	4

Table 6: Hyperparameter configurations for different benchmarks. We adjust image resolution and temporal parameters to balance the trade-off between visual precision, memory consumption, and inference speed. Notably, the Real-world setting uses a sparse prediction strategy to minimize latency.

Robot Policy Adaptation

Robot policy adaptation changes or steers learned robot behaviors to handle new tasks, environments, hardware, and user preferences.

See all 163 papers

Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback
Sep 28, 2026Yishu Li, Liyuan Geng, Xinyi Mao +2Robot Policy Learning Test-Time Adaptation
Robotics Institute, Carnegie Mellon University · Computer Science and Technology, Tsinghua University
GeniWorld: A Generalizable Interactive World Model for Robotic Manipulation via Visual Actions
Aug 6, 2026Chenghao Gu, Hanyang Yu, Jingbo Zhang +7Action-Conditioned World Models Robotic Policy Evaluation
Shenzhen International Graduate School, Tsinghua University · Tencent Robotics X · The Hong Kong University of Science and Technology +1
ORPA: Online Residual Policy Adaptation for Robot Manipulation Control with Human Feedback
Aug 18, 2026Muhammad A. Muttaqien, Tomohiro Motoda, Ryo Hanai +1Robotic Control Robot Policy Learning
Embodied AI Research Team · National Institute of AIST · Tokyo, Japan

Sep 28, 2026J/K move · Enter open · S save

Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback

Yishu Li, Liyuan Geng, Xinyi Mao, Amber Li+1

Robotics Institute, Carnegie Mellon University · Computer Science and Technology, Tsinghua University

Robot Policy Learning Test-Time Adaptation Robot Policy Adaptation Robot Dynamics Learning Robotic Manipulation Robot Learning

Open paper AskPDF arXiv

While pretrained robotic policies exhibit impressive capabilities in controlled environments, unobserved physical properties and dynamics require these policies to rapidly adapt during deployment. Existing test-time adaptation methods typically rely on sparse scalar rewards, failing to exploit the rich geometric and dynamic feedback from the environment during physical interaction. To address this challenge, we propose SCOUT, a dynamics-aware meta-learning framework that enables manipulation policies to rapidly adapt by continuously revising their internal beliefs about environment dynamics. Our approach couples an action-prediction policy with a forward dynamics model via a shared belief latent space. During meta-training, an inner loop updates this shared belief latent by minimizing the dynamics prediction error against the observed action outcome, while the outer loop optimizes the network for action selection. At deployment, this structure allows the agent to infer and adapt to unknown physical dynamics on the fly. By updating its latent belief based on action-outcome mismatches, the policy automatically adapts without risking catastrophic forgetting. We demonstrate that SCOUT significantly accelerates online adaptation across simulated manipulation benchmarks and achieves robust sim-to-real transfer in the real world. Project webiste can be found here: https://liy1shu.github.io/SCOUT/

Abstract

Robot Policy Adaptation

Explore similar work

Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback

GeniWorld: A Generalizable Interactive World Model for Robotic Manipulation via Visual Actions

ORPA: Online Residual Policy Adaptation for Robot Manipulation Control with Human Feedback