Organizations: Fudan University · Shanghai Innovation Institute · Shanghai Jiao Tong University · Southeast University · Southern University of Science and Technology · Noematrix Ltd.
Expert demonstrations show how to complete contact-rich tasks, but provide limited coverage of how interaction forces respond to different actions. Task-agnostic play provides broader experience of contact dynamics without specifying task goals. To fully leverage these two types of data, we introduce ForceWAM, a force world action model that decouples knowledge of task-compatible interaction outcomes from knowledge of how local actions change contact. A world model learns from successful demonstrations to jointly predict future video and six-axis force/torque trajectories. Together, these futures describe the expected evolution of motion and contact during the task. A separate inverse dynamics model (IDM) learns from both task and play, mapping current observations and recorded visual-wrench futures to their associated robot actions. Play data thus provide knowledge of force responses for goal-directed execution. At deployment, the IDM reactively translates generated futures into actions, reusing them with fresh wrench feedback between world-model updates. Across real-robot force-regulated sliding, box flipping with and without disturbances, and ring placement, ForceWAM achieves 85-95% mean success, surpassing world-modeling and reactive baselines. Our experiments further show that adding task-agnostic play data improves success by 15-45 percentage points, demonstrating its value for learning contact-rich execution. Project page and code are available at https://force-wam.github.io/.
Figures & tables
Figure 1: Motivation of ForceWAM. The illustrated end-to-end force-conditioned and force-envisioned policies use only task demonstration data: the former lack pretrained video-world-model priors, while the latter lack within-chunk force feedback. ForceWAM combines the pretrained prior with task demonstrations for task-compatible visual-wrench futures and action-labeled play for IDM execution learning, achieving the highest mean success across all four real-robot conditions.
Figure 2: ForceWAM training and deployment. Task demonstrations train the visual-wrench future generator, while task and play data train the IDM from recorded futures and associated actions. During deployment, ForceWM supplies future conditions and the causal IDM updates actions using measured wrench feedback.
Figure 3: Real-robot comparisons of ForceWAM, RDP, ForceVLA, and FA-WAM.
Method
Slide ↑
Flip ↑
Flip-D ↑
Hanoi ↑
RDP ( 2025 )
15%
40%
30%
45%
ImplicitRDP ( 2026 )
20%
45%
25%
55%
ImplicitRDP ( 2026 ) + Play
10%
25%
10%
30%
ForceVLA ( 2025 )
15%
55%
30%
60%
Cosmos Policy ( 2026 )
15%
40%
30%
60%
Cosmos Policy ( 2026 ) + Play
10%
20%
15%
40%
Table 1: Mean real-robot success rates (the best per column in bold). Slide requires full-path completion with 5 - 15 N contact force. “ + Play”: training with additional action-labeled play data.
Figure 4: Mean success with increasing play data. Task demos, the ForceWM model, and IDM optimizer steps are fixed.
Figure 5: ForceWM predictions during closed-loop box flipping. (a) Recorded wrist images (top) and decoded futures (bottom). The three central predictions share one conditioning observation. (b) Recorded (gray) and predicted (blue) Fy and Fz (all six wrench components in Appendix F ).
Future conditioning
Slide ↑
Flip ↑
Flip-D ↑
Hanoi ↑
Visual + wrench
85%
95%
85%
90%
Visual only
15%
40%
30%
70%
Wrench only
20%
30%
25%
30%
No future conditioning
15%
30%
35%
50%
Visual + wrench, no augmentation
65%
80%
70%
75%
Table 2: Online future-conditioning ablation: mean success rates. All IDMs use task + play data, with futures from one fixed ForceWM. Rows 1-4 vary future modalities during training and deployment. The last uses both without training-time future-conditioning augmentation.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Hardware setup and task objects.
Augmentation
Flip-box setting
Whole-condition dropout
0.10
Visual / wrench dropout
0.25 / 0.50
Scale jitter
[0.85,1.15]
Wrench noise std.
0.075 in normalized coordinates
Visual-latent noise std.
0.075 times latent standard deviation
Temporal offsets
{0,−1,−2,−3,−4,−5,−6,−9,−12}
Appendix
Table 3: Future-conditioning augmentation used for the flip-box IDM.
Figure 7: Full six-axis comparison for the rollout in Figure 5 . Recorded (gray) and predicted (blue) forces are shown on the left and torques on the right.
Contact-rich robotic manipulation studies how robots plan and control physical interactions such as grasping, insertion, pushing, and dexterous object handling.
Force signals provide critical interaction cues for contact-rich robotic manipulation. However, existing methods mostly use force as an additional observation modality, without fully exploiting its role in modeling future interaction dynamics or guiding execution-time feedback correction. In this paper, we propose FAWAM, a force-aware world action model that incorporates force information at three levels: perception, prediction, and closed-loop execution. FAWAM first encodes historical 6-axis force/torque signals to modulate action generation, then jointly predicts future actions and end-effector wrenches to explicitly model contact evolution. It further introduces a residual correction module that uses the predicted wrench trajectory as an execution-time reference to refine actions online based on real-time force feedback. Real-world experiments across multiple contact-rich tasks show that FAWAM improves the average success rate by 36.25% over vision-only baselines and 21.25% over existing force-aware baselines, demonstrating the effectiveness of our force-aware framework for robust contact-rich manipulation.
Haotian He, Zeyu Yan, Qipeng Liu +2
School of Mathematical Sciences, Peking University · School of Artificial Intelligence, Shanghai Jiao Tong University.
World Action Models (WAMs) combine future-state prediction with robot action generation, but existing approaches largely rely on visual futures. Visual prediction captures scene structure and object motion, yet provides limited supervision for force, deformation, shear, and slip during contact-rich manipulation. This creates two design requirements: tactile futures should carry meaningful physical information, and they should not become privileged cues for action generation. We present TacWAM, a mechanics-aware tactile WAM that addresses this challenge in three steps. First, a Spatially Aligned Fusion (SAF) Tactile Encoder maps tactile appearance, dense force fields, and deformation flow into a shared latent prediction space, with bilateral force and torque reconstruction preserving global contact information. Second, a tactile history encoder provides temporal context so future tactile prediction reflects how force and deformation change beyond the current tactile observation. Third, Anchor-Guided Tri-Modal (AGT) Attention separates current visual and tactile anchors, future prediction tokens, and action tokens, allowing future tactile states to supervise training without being directly read by the action branch. We evaluate TacWAM on four real-world contact-rich manipulation tasks covering fragile grasping, sustained surface contact, and dynamic in-hand manipulation. TacWAM achieves an average success rate of 75.0%, exceeding the strongest evaluated baseline by 37.5 percentage points. Staged ablations show consistent degradation when tactile history is removed and access to future prediction targets is relaxed. These results indicate that future tactile supervision can improve contact-aware action learning when combined with informative tactile representations and deployment-consistent information constraints.
Force-sensitive manipulation requires contact and load information that is not provided explicitly by visual observations. We present CW-WAM, a world action model that jointly predicts video, motion, and contact wrench. Inspired by hybrid motion-force control, it contains a unified interaction module that treats both motion and contact wrench as modeling targets and predicts time-aligned motion-wrench chunks as control references. A pair-causal attention mechanism structures the motion-wrench dependencies, while a long fixed-length wrench memory retains past interaction information. The predicted wrench chunk acts as a force reference for a 20-Hz outer-loop admittance controller within a slow-fast control system that maintains force feedback and smooth motion between model queries. CW-WAM improves contact prediction and increases the insertion success rate from 25% to 70% for square pegs with 1-mm clearance per side and from 40% to 95% for power adapters, compared with WAM without wrench input. It also improves lifting-height accuracy across different loads. Our framework with an admittance controller reduces force-reference RMSE from 4.8 to 2.7 N for peg insertion and from 2.0 to 1.3 N for wiping, respectively, with smaller force fluctuations during successful execution. Project website: https://cw-wam-project.github.io/