F3: What Makes Good Experience in World-Action Models?
Authors: Zihe Huang, Sirui Zhou, Haozhi Han, Ruge Zhang, Teng Ma, Tao Ma, Zheng Liu, Ce Zhang, Ting Cao, Yunxin Liu, Yunquan Zhang, Yifeng Chen, Kun Li
Organizations: Peking University · Institute for AI Industry Research (AIR), Tsinghua University · Independent Researcher · University of Chinese Academy of Sciences · Alibaba Group · Zhejiang University
Deployed world-action models (WAMs) can keep improving only if they make good use of their own experience. Existing methods supply retrieved rollouts or demonstrations to the policy as context, yet such experience often turns previously solved tasks into failures. What, then, makes good experience for a WAM? We show that experience is valuable only in use: its value depends not on whether the source rollout succeeded but on what is stored, when it is used, and where it enters the policy. We therefore propose F3, which makes these three decisions with Form, Fit, and Fusion. Form compresses successful and failed rollouts alike into short, verified operation records; Fit teaches the policy to read a record and to predict when it helps; Fusion injects the record's images, text and actions into the policy as a prefix and an action prior. Trained once on only 200 examples and deployed with a growing experience library, the model achieves state-of-the-art results on MolmoSpaces and RoboLab without any training on RoboLab: over native Cosmos 3 Nano, it raises success by 12.0 pp and 10.8 pp, reduces steps to success by 12.2% and 11.3%, and cuts regressions by 57%.
Figures & tables
Figure 2: F 3 succeeds more often and in fewer steps. (a) Success rate on MolmoSpaces and RoboLab; the dotted line marks native Cosmos 3 and hatching the gain of F 3 . (b) Mean steps on episodes that both policies complete; hatching marks saved steps. (c) Paired outcomes against native Cosmos 3 on the 100 MolmoSpaces episodes: regressions (S → F), rescues (F → S) and net change. Direct input breaks three times as many episodes as it rescues, whereas F 3 cuts regressions by 57%.
Figure 3: Overview of F 3 . Form (left) compresses successful and failed rollouts into short verified records; Fit (middle) teaches the WAM to read a record and learns when it helps; Fusion (right) injects the selected record’s images, text and actions as a prefix and an action prior. The WAM is trained once and stays fixed at deployment, while new rollouts grow the library.
MolmoSpaces (SR, %)
RoboLab (SR, %)
Policy
Type
MS-Pick
MB-Pick
MS-PnP
Overall
Simple
Moderate
Complex
Overall
Baseline policies
π0 -DROID [ 5 ]
VLA
3.7
7.0
10.0
7.0
7.2
3.6
0.0
5.0
π0 -FAST-DROID [ 24 ]
VLA
14.8
2.3
13.3
9.0
20.2
13.3
2.9
15.5
π0.5 -DROID [ 4 ]
VLA
18.5
25.6
6.7
18.0
29.7
31.5
13.5
28.0
DreamZero-DROID [ 32 ]
WAM
40.7
27.9
16.7
28.0
26.1
30.0
14.1
25.7
Table 1: Closed-loop manipulation performance. Success rate (SR, %) on 100 MolmoSpaces instances and 120 RoboLab tasks. Gray : RoboLab results from the official leaderboard [ 25 ] ; all other numbers are from our runs. The middle rows reimplement prior mechanisms on our fitted Cosmos 3 Nano and library (Sec. 4.1 ); our rows add one component at a time to it. (Mol)/(Rob): library source. ‡ Same run as the Zeva-style row. Bold : best.
Figure 4: Failed rollouts are as useful as successful ones. Left: growth of the MolmoSpaces library by source. Middle: the 55 records by operation and object. Right: injection rate when retrieved, share of the 490 injections, and the 35 of 67 successful episodes that use a failed-rollout record.
Direct input (native WAM)
Trained encoder and readers
Input
MS-Pick
MB-Pick
MS-PnP
All
MS-Pick
MB-Pick
MS-PnP
All
None
63.0
55.8
46.7
55.0
63.0
55.8
46.7
55.0
T
63.0
55.8
40.0
53.0
74.1
67.4
36.7
60.0
V
66.7
62.8
46.7
59.0
63.0
62.8
30.0
53.0
A
29.6
25.6
3.3
20.0
66.7
62.8
30.0
54.0 †
T+V
51.9
51.2
16.7
41.0
70.4
58.1
36.7
55.0
Table 2: The same experience hurts or helps depending on whether the WAM has learned to read it. Success on the 100 MolmoSpaces episodes when retrieved records are given to native Cosmos 3 Nano-DROID directly (left) or through the trained encoder and readers (right). T/V/A: text/vision/action; vision inserts three retrieved frames into a camera slot. Right: T+V+A is the epoch-5 full-input run, the other inputs replay its injection schedule, and the last row adds the Fit gate. † AED-style input (Sec. 4.1 ).
Figure 5: Fit training converges and improves success. (a) Training losses over five epochs. (b) Success of each saved checkpoint: F 3 stays at or above the similarity threshold and native Nano throughout.
Figure 6: Learning to read experience turns losses into gains. Each point is one input type on the 100 MolmoSpaces episodes, placed by the episodes it breaks (S → F) and rescues (F → S) relative to native Nano; above the diagonal is a net gain. Direct inputs fall in the loss region, the trained encoder brings them close to the diagonal, and F 3 lies furthest in the better direction.
Figure 7: How F 3 reuses experience within one episode. On the knife-to-bowl task of Fig. 1 b, injecting the fork, ladle and spatula records at the right calls lets the WAM grasp, carry and place the knife in 139 steps, while native Nano times out at 682. (a) Knife paths. (b) Gripper path and injected records. (c) Each record and the sub-goal it completes. (d) Placement error. (e) Steps per stage.
World action models (WAMs) jointly generate actions and imagined futures, but a plausible prediction need not be realized by its co-generated action. We introduce WISE (World Action Inference Scaling through Evaluation), a pre-execution selector that scores each sampled rollout on action validity and temporal regularity, inverse-dynamics consistency between the proposed action and the imagined motion, and instruction-conditioned task progress, and executes the highest-scoring candidate while keeping the WAM frozen. With four candidates per decision, WISE raises closed-loop success from 6.0% to 24.0% on RoboLab with Cosmos~3 Edge and from 75.1% to 78.3% mean success on LIBERO with DreamZero, and exceeds the best existing selector in aggregate by 4.0 and 0.9 points. Component ablations and agreement analyses show that the three scores rank candidates differently and that task progress carries the largest weight in the fused decision.
World-Action Models (WAMs) couple action generation with predictions of how physical interactions unfold. However, current post-deployment learning paradigms typically improve behavior without requiring better world predictions. Especially in dexterous manipulation, small execution errors can compound in high-dimensional action spaces, hindering policy improvement and pushing interactions beyond the world model's training distribution. Motivated by this, we propose Direct Experience World-Model Optimization (DEWO), a post-deployment learning paradigm for WAMs that, alongside action imitation, refines world representations through visual experience to better condition action generation. Specifically, it identifies interaction turning points and learns from successful and failed futures to support classifier-free guidance. An additional value head estimates task progress from video representations and activates guidance when progress stalls during inference. Across five DexJoCo tasks, DEWO improves average success across all three WAM formulations. Ablations show that visual supervision from successful and failed continuations improves both prediction and control beyond action supervision alone. On four real-world tasks across Wuji and Sharpa, 3 x 3 grid evaluations show that two rounds of deployment learning increase success from 51.0% to 71.7% in cells with at least one initial success, a gain of 20.7 percentage points. These findings support continued predictive learning for improving control through deployment experience, making world modeling an active part of WAM adaptation.
Xiangcheng Zhan, Zirui Chen, Yicheng Zhao +2
Harbin Institute of Technology · Dalian University of Technology · Southern University of Science and Technology
World Action Models (WAMs) emerge as a promising foundation for policy refinement by predicting the consequences of sampled actions. However, visually plausible predictions can mislead policy refinement if they fail to reflect the input actions. To address this mismatch, we introduce R2-WAM, a two-stage repair-and-reject post-training framework that first improves the consistency of predicted futures with input actions, then uses these futures to select inferior action samples for negative fine-tuning. The repair stage grounds imagination in observed robot behavior through a kinematic alignment score that measures agreement between predicted and demonstrated motion, enabling the predicted video to faithfully reflect its input actions. Using the repaired video model, the rejection stage compares imagined outcomes of sampled and demonstrated actions, selectively applying negative fine-tuning to samples whose predicted task progress falls below the demonstrated reference by a prescribed margin. Together, the two stages extend video prediction from representation learning to consequence-based policy refinement without additional environment interaction or changes to the inference procedure. R2-WAM achieves 93.8% average success on RoboTwin 2.0 across clean and randomized settings. On the long-horizon real-world Fold Shirt task, it achieves 87.5% average success, compared with 0% for Fast-WAM. Our project page is available at https://r2-wam.github.io/.
Ruiyan Xu, Haisheng Su, Sixu Lin +4
The Chinese University of Hong Kong, Shenzhen · Manifold AI · Southeast University +1