A robot that acts around people needs a world model of the human: a belief over the person's body state and a dynamics model that predicts where that state goes next. This abstract argues that two published components form exactly these two halves, and specifies how to couple them. KNOWN [1] reconstructs the 3D human body while estimating both aleatoric and epistemic uncertainty in real time (11.6 ms per frame); its aleatoric term identifies unreliable observations, its epistemic term identifies inputs from low-density regions of the training data, and refinement driven by the epistemic term reduces the error on such inputs from 81.3 to 70.3 mm. PhysMoP [2] predicts future body configuration by solving the Euler-Lagrange equations of the body with learned dynamics terms, reducing the 80 ms prediction error from 9.6 to 2.1 mm on Human3.6M and from 12.1 to 0.7 mm on 3DPW. We define the interface between the two, propose three mechanisms by which KNOWN's uncertainty enters PhysMoP, and specify an evaluation protocol on in-the-wild video. No new experimental results are reported here.
Figures & tables
Figure 1: The two halves of the human world model. (a) KNOWN [ 1 ] : a regression network outputs a distribution over SMPL pose θ , shape β and camera C ; forward kinematics and projection give the observation model p(p2D∣X) , from which aleatoric and epistemic uncertainty are read off; the uncertainties enter training through the NLL and uncertainty-guided refinement. (b) PhysMoP [ 2 ] : a physics branch predicts the Euler–Lagrange terms M , C , G and integrates them, a data branch predicts the whole future, and a fusion network combines the two. The arrow is the coupling proposed in Sec. 4 : KNOWN’s per-frame belief becomes the history q1:T that PhysMoP integrates forward.
Figure 2: Qualitative results of KNOWN on two Human3.6M images [ 1 ] . Left to right: input; per-joint aleatoric uncertainty (radius of the red circles); per-joint epistemic uncertainty (yellow); reconstruction colored by per-vertex epistemic uncertainty, before (left) and after (right) uncertainty-guided refinement. Dashed circles mark the occluded or ambiguous parts the uncertainties single out.
Method
All
Minority
MB
ms
HMR [ 14 ]
–
107.3
–
–
SPIN [ 15 ]
–
71.0
–
–
ProHMR [ 6 ]
–
–
223.0
16.4
Biggs et al. [ 7 ]
–
–
566.6
–
KNOWN, before refinement
58.1
81.3
103.9
11.6
KNOWN
55.9
70.3
103.9
11.6
Table 1: KNOWN on Human3.6M: mean per-joint position error after Procrustes alignment (P-MPE, mm) on all test images and on the minority subset, with model size and per-frame latency, from [ 1 ] . “Minority”: test images whose epistemic uncertainty exceeds a percentile threshold set on the training set. HMR and SPIN are evaluated on the same subset.
Figure 3: Qualitative results of PhysMoP on two Human3.6M test sequences [ 2 ] : ground truth in grey, prediction in blue, from the last history frame to 1 s ahead. Larger overlap means smaller prediction error.
Data
Method
80 ms
160
320
400
1000
H3.6M
siMLPe [ 10 ]
9.6
21.7
46.3
57.3
109.4
H3.6M
EqMotion [ 11 ]
9.1
20.1
43.7
55.0
106.9
H3.6M
PhysMoP, physics branch
2.1
7.6
28.4
42.2
148.2
H3.6M
PhysMoP, data branch
4.4
10.6
33.8
47.8
103.3
H3.6M
PhysMoP
2.1
7.6
28.4
43.8
103.9
3DPW
siMLPe [ 10 ]
12.1
22.1
38.1
44.5
72.2
Table 2: PhysMoP motion prediction: root-aligned MPJPE (mm) at each horizon, from [ 2 ] . Human3.6M follows protocol P1; the 3DPW rows use a model trained on AMASS and evaluated cross-dataset. Inputs are ground-truth motion-capture histories in all rows.
A general-purpose vision-language model can understand a task goal without knowing how a particular robot's motion and functional parts produce the intended effect. We introduce KnowBody, a harness that makes these action-relevant body relations explicit, queryable, and revisable while keeping the model weights frozen. Initialized from one off-task trajectory, a partial body model guides action selection and the interpretation of past interactions. New evidence refines the model, and knowledge dependent on revised body estimates is rechecked before reuse. Across 32 fixed-budget trials on four real-robot tasks, initialized KnowBody achieves 75% completion versus 25% for the native harness and requires fewer planner rounds on successful trials in tasks completed by both. With persistent updates enabled, planner rounds decrease by 29-53% from the first to the fifth recorded success.
Zeyu Lou, Yanhong Zeng, Yong Wang +1
Nanjing University · Ant Group · Zhejiang University
We reimplement the dynamics model of the Robotic World Model (RWM) and test the uncertainty estimate that its follow-up, RWM-U, uses to penalize reward in offline policy optimization. Our implementation matches the released model's gradients. On held-out data, training on the model's own rollouts beats one-step training at the preregistered long horizon for all ten episodes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss on sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.
A robot should be able to learn through experiments how unfamiliar objects behave and interact, then plan with that knowledge. It need not start from scratch: physics engines supply knowledge of motion and contact, but can omit entire mechanisms, such as glue curing, water heating, or wind. We present EMPIRIC, an agent that learns a residual world model: a physics engine extended with code for the missing mechanisms. The learned programs can introduce new forces, constraints, and hidden state, and Bayesian inference estimates their parameters and states from noisy observations. The resulting model lets the agent predict the outcomes of actions, choose informative experiments, and revise its hypotheses when predictions fail. Across five simulated domains, EMPIRIC learns interpretable, reusable models, and solves more tasks with fewer environment interactions than all three baselines. On a physical robot, it learns wind forces and domino masses to solve a manipulation task. Website and code: https://yichao-liang.github.io/empiric
Yichao Liang, Amber Li, Dat Nguyen +13
Basis Research Institute · University of Cambridge · Carnegie Mellon University +6