Robotic Multiphase Interaction: Manipulating Coupled Liquid and Solid Dynamics with a World Model
Authors: Yixuan Feng, Peng Wang
Organizations: College of Design and Engineering, National University of Singapore, 9 Engineering Dr 1, Singapore,117575. · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford GU2 7XH, United Kingdom.
This work presents Robotic Multiphase Interaction (RMI), a setting in which liquid enters a porous material and interacts mechanically with its deforming solid skeleton. Manipulation can therefore change pore volume, expel or redistribute retained liquid, and alter grasp stability at the same time. Spilled liquid can also create safety risks in domestic and manufacturing settings. This differs from most manipulation of solid objects and from tasks that involve both liquid and solid while keeping the phases spatially separate. We study a sponge filled with water as the first RMI example. We use implicit incompressible porous flow with smoothed particle hydrodynamics as the dynamics engine and enable robotic manipulation by adding Coulomb contact memory, hybrid velocity and force regulation, and a stability gate for lifting. The resulting environment connects robot commands to changes in the coupled liquid and solid state. A world model conditioned on actions predicts how this state evolves under candidate commands, while a temporal UNet generates actions using either Diffusion Policy or rectified flow matching. Our world model reduces retained water prediction error by more than 60% compared with the baseline. The best action sequence selected by the world model from policy proposals further reduces the predicted terminal water error by about half. These improvements show that modelling the coupled liquid and solid state helps the robot predict how its actions affect both the porous object and the liquid held inside.
Figures & tables
Fig. 1: Three manipulation settings. Left : conventional manipulation of a solid object, which may be rigid or deformable. Middle : both a solid and liquid are present but occupy different regions. Right : porous RMI, where liquid occupies the solid’s pore space and interacts with its deforming skeleton.
Fig. 2: RMI framework. Left : two grippers grasp a sponge soaked with liquid, with the relevant forces, contacts, and gripper width. MS and ML denote the solid and retained liquid masses, respectively. Middle : the state and action representations used in this work. Right : the training and inference pipeline for the world models and policies.
Fig. 3: Coulomb friction with contact memory. Each contacting solid particle stores an anchor in the gripper frame. The tangential displacement es and restoring demand Freq,s determine whether the contact sticks or slips within the Coulomb cone.
Interface
Model
Role
Causal
Baseline
The latest state through the rollout.
Causal
State-GRU-WM
Same GRU as AC-GRU-WM, no command.
Causal
AC-GRU-WM
Proposed world model, one command per step.
Lookahead
Linear-ARX
Locally linear residual on the full chunk.
Lookahead
AC-MLP-WM
One evaluation of (yt,vec(At)) .
TABLE I: World model predictors. Causal models receive one command at each step, while lookahead models see the complete action chunk at once.
Fig. 4: One completed run for the medium sponge at ϕ=0.7 , using 210N to hold the sponge before lifting and 300N to squeeze it afterwards. The left axis shows target and measured gripper force, while the right axis shows retained water. Most liquid leaves as the measured force approaches the holding target. Lifting changes the retained water only modestly, and the later squeeze removes a smaller additional amount. Water is shown as a 2s trailing average.
Fig. 5: Water removed during each controller phase over all runs that reach that phase. The initial force ramp and hold dominate transport, while lifting has the smallest median effect. The post-ramp and post-hold phases constitute the later squeeze stage, which removes part of the remaining liquid. The distributions show that clamping and holding, rather than lifting, account for most transport through the porous material.
Fig. 6: Lift success over porosity and the force used to hold the sponge before lifting. Each cell combines the six later squeeze settings. All conditions with ϕ=0.5 lift successfully. Higher porosity requires a stronger holding force and remains difficult for the small and large sponges.
8cm
10cm
12cm
Method
water ( g )
force ( N )
fric.
water ( g )
force ( N )
fric.
water ( g )
force ( N )
fric.
Baseline
12.23
7.93
0.57
16.71
10.01
0.61
23.31
10.94
0.58
State-GRU-WM
9.42
9.62
0.61
9.02
6.77
0.65
17.03
12.98
0.68
AC-GRU-WM
4.32
3.78
0.72
6.59
4.74
0.79
17.53
11.04
0.61
Action input ablation
AC-GRU-WM, shuffled A
15.28
15.41
0.55
15.68
14.67
0.60
18.49
11.27
0.61
TABLE II: Causal world model prediction on the collected test split at 0.64s . We report water MAE in grams, force MAE in newtons, and balanced accuracy over four friction classes. The first three rows form the main comparison, and bold marks the best of these models. The final row is the action input ablation discussed in Sec. VI-B .
Method
8cm ( g )
10cm ( g )
12cm ( g )
Original controller action
2.56
4.23
4.68
Diffusion policy, first sample
2.79
4.30
4.94
Diffusion policy +WT
1.24
2.57
2.82
Flow matching, first sample
2.61
4.28
4.86
Flow matching +WT
1.11
2.24
2.45
TABLE III: Predicted terminal water error in grams for the original controller action, the first policy sample, and the action selected from eight samples by Transformer WT .
State
8cm ( g )
10cm ( g )
12cm ( g )
LS
9.11
10.85
8.41
LS + G
8.68
11.70
14.63
LS + G + C
7.36
13.32
14.74
ALL
4.32
6.59
17.53
TABLE IV: Ablation of the state used by AC-GRU-WM at 0.64s , reported as water MAE in grams. LS contains liquid and solid phase variables, G adds gripper loading, C adds contact, and ALL adds residual slip.
Method
8cm ( g∣N )
10cm ( g∣N )
12cm ( g∣N )
water
force
water
force
water
force
Linear-ARX
4.29
1.77
9.53
13.48
9.67
25.77
AC-MLP-WM
3.74
2.72
5.75
3.96
7.44
5.87
AC-MLP-WM, shuffled A
17.47
13.41
26.15
19.60
42.87
26.02
TABLE V: Diagnostic in which the complete future action chunk is visible at every predicted step. Water and force MAE are reported at 0.64s in grams and newtons. Bold marks the best result within this diagnostic.
Method
8cm ( N )
10cm(N)
12cm(N)
MAE F
smooth F
MAE F
smooth F
MAE F
smooth F
Diffusion policy
5.23
0.31
5.87
0.27
4.79
0.34
Flow matching
5.11
0.35
6.26
0.31
5.37
0.37
TABLE VI: Ablation of the policy training objective on a shared temporal UNet. We report force MAE and the mean force change per step for one sample, with both quantities measured in newtons. Each sponge size contributes several thousand test windows.
Action source
8cm
10cm
12cm
Original controller action
3.61
5.22
6.00
DP-UNet +WT
2.61
3.69
4.37
FM-UNet +WT
2.51
3.43
4.06
TABLE S1: Combined predicted cost J for the original and selected actions.
World models offer a promising route toward robot planning by enabling agents to imagine and verify the consequences of actions before execution. However, current video-based world models often struggle to capture the physical constraints that govern manipulation, particularly contact. Further, their action conditioning is often constrained to specific embodiments such as parallel grippers. We propose Contact Flow, an embodiment-agnostic action representation that encodes manipulation through the trajectory of 3D contact points between an actor and a target object. By discarding actor-specific appearance and kinematics, Contact Flow provides a shared conditioning signal for both human demonstrations and robotic execution. Therefore, we can train a large-scale video generative model on both human and robotic object interaction videos conditioned on Contact Flow, yielding a world model that predicts physically plausible manipulation outcomes. We integrate this model into a propose-imagine-verify-act pipeline, where generated rollouts are assessed by a vision-language model before execution. Experiments on the DROID dataset and real-world tabletop manipulation tasks demonstrate that Contact Flow enables transfer between human demonstrations and different robotic embodiments.
Physical intelligence requires more than predicting how the world may evolve: predictions must remain actionable as the world continues to change. We introduce InternW0, the first instantiation of the InternW physical world model series from Shanghai AI Laboratory, built around omnimodal interfaces, asynchronous multi-frequency processing, and local physical modeling under partial observations and external influences. InternW0 jointly learns future visual dynamics and continuous robot control through an asymmetric video--action architecture with flow matching. A high-capacity video expert provides longer-horizon predictive context, while a lightweight action expert operates at a faster timescale. Instead of regenerating the future for every action update, InternW0 reuses layerwise K/V and adapts it to newly observed states through observation-conditioned context routing. Domain-specific interfaces and soft prompts support heterogeneous embodiments, while contact-aware post-training incorporates force and tactile signals for contact-rich manipulation. We train InternW0 on approximately 7,200 hours of heterogeneous robot and egocentric data, including EgoLab, a 275-hour real-laboratory egocentric dataset. Evaluation spans simulation benchmarks and real-world scientific tasks, including a 15-stage metal--organic framework synthesis workflow and 5-stage contact- and force-aware dexterous manipulation for general-purpose quantitative pipetting. These results advance scalable, asynchronous, and science-native physical world models for universal and efficient real-world interactions.
Jisong Cai, Yao Mu, Ganlin Yang +23
Physical Intelligence Team, Shanghai AI Laboratory
Visual world models have shown great potential in learning complex system dynamics. Recent advancements leverage these models as transition functions within Model Predictive Control (MPC) frameworks to solve various control tasks. When applied to robotics, however, they are limited to single-stage tasks such as reaching or grasping, and struggle with multi-stage ones that demand complex sequential planning. In this work, we introduce WorldDP, a world model framework designed for multi-stage robotic manipulation. Our hierarchical approach utilizes a high-level world model as a transition function to optimize for feasible subgoals during runtime, which are subsequently reached by a low-level Diffusion Policy. To further aid in learning dynamics and planning, we incorporate object-centric representations that decouple environmental entities and enable us to plan sequentially with respect to each. Evaluated across several robotics benchmarks, WorldDP consistently outperforms existing baselines, validating that coupling the world model's physically grounded planning with diffusion policy's efficient execution yields superior multi-stage performance. Project Page: https://raktimgg.github.io/worlddp-website/.