Physical Twins: Accelerating and Enabling Robot Learning with Phantom Platforms
Authors: Elizabeth Peiros, Derek Chen, Lucas Yager, Edward Lee, Kevin Wang, Zoe Samuels, Michael C. Yip
Organizations: Electrical and Computer Engineering Department, University of California, San Diego, La Jolla, CA 92093 USA · Mechanical and Aerospace Engineering Department, University of California, San Diego, La Jolla, CA 92093 USA
Improvements in human-robot physical interaction (pHRI) can have major implications for physical therapy, search and rescue, and telemedicine. However, a major challenge concerns human constraints and safety in human-robot physical experiments. Concerns about human studies also include repeatability, scalability, and participant diversity. To conduct such experiments, an IRB and willing human participants are required. In this work, we present an improved phantom device, a physical twin, that enables real-world RL-type testing for physically interactive algorithms. The new device not only replicates the ball-and-socket motion of the shoulder but also renders scapular and protraction/retraction motions. The experiments showcase the device's ability to render multiple 3D joint limits and demonstrate a Franka Panda arm sensing limits and interacting with the device as an arm for ADLs (activities of daily living) and pHRI tasks.
Figures & tables
Figure 2: Top image, Fig (a), highlights the rotation of the clavicle bone when x-rayed. The labeling shows both the angular change and the translational motion of the shoulder socket. Bottom image, Fig (b), shows the translation of joint motion into a mechanical mechanism. From top to bottom the acromioclavicular joint is transformed into a pivot joint, the shoulder socket is transformed into a spherical manipulator, and the scapular motion is re-imagined as a 4-bar mechanism.
Figure 3: (a) Model of the haptic shoulder with each components range of motion demonstrated in yellow, pink, and cyan (b) The Spherical Parallel Manipulator (SPM) Mechanism separated out (c) The concentric shaft coupling of the SPM with linked linkages demonstrated with the same color (d) The capstan drive mechanism for the SPM with steel string highlighted in yellow (e) The four-bar and pivot mechanisms separated out (f) Individual components of the four-bar mechanics with yellow being the driving motor, green and blue being the linkages, and red being the PTFE slides (g) Components of the pivot with the motor being highlighted in yellow
Joint Movement
Literature Limit
Our Design
GH elevation
90∘
94∘
GH internal/external rotation
160∘
400∘
Scapular protraction/retraction
40∘
140∘
Scapular elevation
60∘
114∘
Vertical scapular translation
6.24 cm
10.75 cm
Table I: Comparison of Maximum Shoulder Joint Range of Motion
Parameter
Symbol
Value
Units
Noise Floor
NF
0.035
N
Joint Count
Njoints
7
–
SNR Factor
SNRfactor
[5]
–
Total Length
H+U
[0.57]
m
Min Input Torque
taua,min
[0.698]
Nm
Table II: System parameters for the Franka Panda force detection
Figure 4: (a) Frames of the haptic shoulder, frame C is shown offset along the zb axis of B for visual clarity as the origins of B and C are coincident. (b) Diagram of the 4-bar mechanism with lengths, l1:4 , and joints at points p1:4 and the A and B frames. (c) Agile eye with u^ , v^ , and w^ vectors shown with arms 2 and 3 labeled. The table shows the mechanical constants (lengths) of SPM and 4-bar
Figure 5: The input to output torque ratio for the SPM across the configuration space is shown for a tendon resistance torque of 1 Nm pulling the joint back to center. The top image shows the maximum output torque given a specific configuration. The configuration limits an individual motor’s torque output in the desired direction as described in the Jacobian mapping.
Cone Half-Angle
Stiffness k (Nm/rad)
RMS ∣τxy∣ Error (Nm)
30∘
0.50
0.065
1.00
0.086
1.50
0.078
20∘
0.50
0.086
1.00
0.066
1.50
0.019
Table III: Error of rendered torque for 9 haptic cones
Figure 6: The results of experiments with the Franka Panda arm and ATI Axia80 M8 transducer force-torque sensor. Sub-figure (a) shows the wrench torque perceived by the Panda robot with 4 different shoulder ranges of motion. At the lowest range of motion, the robot perceives the highest resistance forces, emulating the stretching of tendons visualized with a bright pink. The largest range in which the robot does not hit the end ranges during its motion is nearly all shown in blue, indicating very low perceived tendon forces. Sub-figure (b) validates the Jacobian transform between the motor torque and output torque of the device. The ATI Axia80 M8 was used to measure the Haptic wrench (resistance torque), which was compared to the desired input torque direction and magnitude. Sub-figure (c) shows the attachment of the robot to the shoulder with the load cell.
Figure 7: Still captures from experiments involving the use of the Franka Panda platform performing tasks. Series (a) depicts the robot grasping the upper arm and lifting the arm from a relaxed position to a position within an acceptable range for [ 7 ] . The second series (b) shows the arm using WAM to lift the device from the armpit, which can be used to understand motions for patient transfer tasks.
Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: https://timengelbracht.github.io/forcetwin-website/
Physical human-robot interaction requires yielding transiently to contact yet recovering the commanded reference under sustained load. Finite-stiffness impedance control retains a static deflection there, while predictive alternatives typically optimize a nonlinear robot or impedance model online. Operational-space cancellation instead exposes a translational error double integrator with a fixed transition matrix and a configuration-scheduled input map, making interaction a predictive quantity rather than a property re-derived per configuration. We build on it a compact offset-free interaction-error MPC for torque-controlled manipulators: a force-domain random-walk state estimates persistent interaction and model error, and a 30-variable convex QP maps the correction through the current task inertia while constraining the applied joint torque. Conditional results establish impedance equivalence of the unconstrained passive feedback, offset-free regulation at feasible frozen configurations, and quadratic stabilizability of the scheduled backbone. In a 1kHz MuJoCo simulation of a 7-DOF Franka FR3, the estimator cuts steady-state error under a repeated 15N step from 2.77mm to 0.042mm when added to the otherwise identical 100Hz MPC. A stiffness-and-damping-calibrated impedance baseline attains 2.59mm but briefly saturates and needs 3.3x the peak positive joint power. Adding ideal measured-force cancellation to that baseline gives 1.39mm, so constant-load rejection is not unique to MPC; the sensorless controller still reaches 0.042mm in the moving task, a 65x reduction without force sensing and without the baseline's saturation or power cost. Demonstrated in simulation under a shared actuator budget, the contribution is an efficient operational-space realization complementing rather than replacing broader interaction-control architectures.
Yongyan Cao, Jinshan Tang
Voryx Robotics LLC, San Jose, CA 95136, USA · Department of Health Administration and Policy, George Mason University, Fairfax, VA, USA
Real-world physical human-robot interaction (pHRI) development is costly and slow, while simulation requires substantial task-specific scene and motion authoring. We present GenPHRI, a generative simulation framework for producing deployment-ready pHRI scenarios entirely from natural-language task descriptions using vision-language model (VLM) agents. Generated scenarios contain a room with furniture, a task-appropriate human pose and placement, and a robot executing the requested interaction. Generator and critic agents iterate on each component, and an orchestrator agent controls cross-stage decisions. With GenPHRI, we generate 50 different assistive scenarios in simulation with no manual intervention and deploy two tasks in a 12-participant user study. Vision policies trained on these scenarios achieve zero-shot task completion rates of ~80% in real-world trials against ~90% in simulation, while producing behaviors that participants rate as consistent with textual task prompts. Further, we demonstrate direct execution of generated motion as a complementary, training-free deployment pathway that enables rapid real-world inspection of generated trajectories at the cost of contact coverage. We release GenPHRI as a platform for generating, training, and deploying pHRI tasks from textual prompts. Additional information may be found on our project website: https://rchi-lab.github.io/gen_phri/