Probabilistic Physics Digital Twin for Human-Robot Interaction

Yushen Hu; Harshvardhan Chandirasekar; Apurv Kushwaha; Jonathan King; Chen Zhu-Tian

cs.ROOct 14, 2026

Probabilistic Physics Digital Twin for Human-Robot Interaction

Authors: Yushen Hu, Harshvardhan Chandirasekar, Apurv Kushwaha, Jonathan King, Chen Zhu-Tian

Organizations: University of Minnesota, USA · Howard University, USA

Abstract

Digital twins are increasingly used in robotics to simulate interactions, predict action outcomes, and support planning. However, constructing accurate digital twins requires precise estimates of physical properties such as mass, friction, and center of mass, which are often difficult to obtain in real-world environments. We propose probabilistic digital twins, which represent uncertain physical properties as probability distributions rather than fixed values. By simulating actions across these distributions, the twin predicts a range of possible outcomes and their likelihoods. These probabilistic simulations can guide both human and robot actions by identifying actions that are effective despite physical uncertainty. We demonstrate this framework through human-robot interaction scenarios, including interfaces that communicate simulated outcomes to users and robot planning strategies that explicitly account for uncertainty.

Figures & tables

Panel of Figure 1 — Fig. 1 : Visualization of scalar consequence-field construction. (a) and (b) show the stability field, (c) and (d) show the impact field. Each sampled point represents an expected consequence obtained by averaging multiple simulated episodes over sampled physical realizations.

Panel of Figure 2 — Fig. 2 : Visualization of location-valued consequence-field construction. The distribution of locations are sampled and constructed into a field.

Fig. 3 : User interface for probabilistic placement guidance. The live camera view (upper left) shows the physical scene and tracked objects, while the main view displays the corresponding Digital Twin with the reconstructed stability field over candidate support surfaces. The field is visualized from stable (blue) to unstable (red) and is used to provide placement guidance during interaction.

Fig. 4 : Four Isaac twin scenarios at representative sweep settings. Blue dots mark the fixed release candidates shared by all conditions; all cubes are $50\text{\,}\mathrm{mm}$ . (a,b) Seesaws: plate COM $10/6\,$\mathrm{mm}$$ inside the supporting cube’s pivot edge; cube/cylinder riders land $r=10/11/12$ or $4/5/6\,$\mathrm{mm}$$ beyond it, respectively (largest shown). Tipping depends on $r$ relative to the balance point. (c) Kinked ramp: $15^{\circ}$ upper face, $\theta_{2}\in\{26,29,32\}^{\circ}$ lower face ( $29^{\circ}$ shown), and release $30\text{\,}\mathrm{mm}$ downslope. Sliding depends on $\mu_{\text{pair}}$ versus $\tan\theta_{2}=0.488/0.554/0.625$ (nominal $\mu=0.60$ ). (d) Cube-on-cube: COM offset $20/25\,$\mathrm{mm}$$ ( $25$ shown, at the edge); geometry determines stability. — Fig. 4 : Four Isaac twin scenarios at representative sweep settings. Blue dots mark the fixed release candidates shared by all conditions; all cubes are $50\text{\,}\mathrm{mm}$ . (a,b) Seesaws: plate COM $10/6\,$ \mathrm{mm} $inside the supporting cube’s pivot edge; cube/cylinder riders land $r=10/11/12$ or $4/5/6\,$\mathrm{mm}$ beyond it, respectively (largest shown). Tipping depends on $r$ relative to the balance point. (c) Kinked ramp: $15^{\circ}$ upper face, $\theta_{2}\in\{26,29,32\}^{\circ}$ lower face ( $29^{\circ}$ shown), and release $30\text{\,}\mathrm{mm}$ downslope. Sliding depends on $\mu_{\text{pair}}$ versus $\tan\theta_{2}=0.488/0.554/0.625$ (nominal $\mu=0.60$ ). (d) Cube-on-cube: COM offset $20/25\,$ \mathrm{mm}$$ ( $25$ shown, at the edge); geometry determines stability.

condition (belief)	tipping	sliding
requested pose, uncorrected	$38.6$ $[33.3,43.9]$	$32.9$ $[27.2,38.4]$
None reactive, no field	$31.9$ $[24.7,39.4]$	$32.9$ —
Geo shape	$25.0$ $[18.3,31.9]$	$32.9$ —
Geo+Pose shape + pose	$0.3$	$32.9$ —
Det physics, point	$25.8$ $[19.7,32.5]$	$16.2$ $[10.0,23.0]$
Prob physics, sampled (ours)	$18.3$ $[12.2,25.0]$	$11.7$ $[6.1,17.8]$

TABLE I : Failure rate (%), robot in the loop,

K{=}4

rollouts. Tipping: 360 paired trials; sliding: 179. Brackets:

95\text{\,}\mathrm{\%}

nested-bootstrap intervals; a dash marks a condition identical to None on every trial.

Fig. 5 : User study stacking tasks. (a) Simple stack (T1). (b) Complex stack (T2). (c) Balanced stack (T3).

Fig. 6 : Post-task questionnaire results from eight participants. (a) Guidance ratings. (b) Interface preference.

Digital Twins

Digital twins are virtual models synchronized with real systems to support monitoring, prediction, simulation, and decision-making.

See all 253 papers

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction
Sep 18, 2026Tim Engelbracht, René Zurbrügg, Mayank Mittal +4Digital Twins Robotic Manipulation
ETH Zurich · NVIDIA · Microsoft +1
GaussTwin: Unified Simulation and Correction with Gaussian Splatting for Robotic Digital Twins
Mar 5, 2026Yichen Cai, Paul Jansonnie, Cristiana de Farias +2Digital Twins Robotics Simulation
Intelligent Autonomous Systems Lab, Technical University of Darmstadt, Germany. · NAVER LABS Europe. · Hessian.AI, Germany. +2
Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility
Oct 8, 2026Yuzhu Sun, Mien Van, Nguyen Minh Nhat +2Robotic RL Human-in-the-Loop AI
School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, Northern Ireland, UK

Sep 18, 2026J/K move · Enter open · S save

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

Tim Engelbracht, René Zurbrügg, Mayank Mittal, Marco Hutter+3

ETH Zurich · NVIDIA · Microsoft+1

Digital Twins Robotic Manipulation Robot Dynamics Identification

Open paper AskPDF arXiv

Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: https://timengelbracht.github.io/forcetwin-website/

Abstract

Digital Twins

Explore similar work

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

GaussTwin: Unified Simulation and Correction with Gaussian Splatting for Robotic Digital Twins

Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility