Tendon Force Modeling for Sim2Real Transfer of Reinforcement Learning Policies for Tendon-Driven Robots

Valentin Yuryev; Josie Hughes

cs.ROMar 4, 2026

Tendon Force Modeling for Sim2Real Transfer of Reinforcement Learning Policies for Tendon-Driven Robots

Authors: Valentin Yuryev, Josie Hughes

Organizations: These authors are with with CREATE Lab, Swiss Federal Institute of Technology Lausanne (EPFL), Lausanne, Switzerland.

Abstract

Robots which make use of soft or compliant inter- actions often leverage tendon-driven actuation which enables actuators to be placed more flexibly, and compliance to be maintained. However, controlling complex tendon systems is challenging. Simulation paired with reinforcement learning (RL) could be enable more complex behaviors to be generated. Such methods rely on torque and force-based simulation roll- outs which are limited by the sim-to-real gap, stemming from the actuator and system dynamics, resulting in poor transfer of RL policies onto real robots. To address this, we propose a method to model the tendon forces produced by typical servo motors, focusing specifically on the transfer of RL policies for a tendon driven finger. Our approach extends existing data- driven techniques by leveraging contextual history and a novel data collection test-bench. This test-bench allows us to capture tendon forces undergo contact-rich interactions typical of real- world manipulation. We then utilize our force estimation model in a GPU-accelerated tendon force-driven rigid body simulation to train RL-based controllers. Our transformer-based model is capable of predicting tendon forces within 3% of the maximum motor force and is robot-agnostic. By integrating our learned model into simulation, we reduce the sim-to-real gap for test trajectories by 41%. RL-based controller trained with our model achieves a 50% improvement in fingertip pose tracking tasks on real tendon-driven robotic fingers. This approach is generalizable to different actuators and robot systems, and can enable RL policies to be used widely across tendon systems, advancing capabilities of dexterous manipulators and soft robots.

Figures & tables

Fig. 1: Overview of our pipeline. We start by collecting tendon force data from real spring mass systems and training an estimator. We then use the estimated forces to simulate accurate tendon forces in highly massively parallelization simulator that supports tendon force driven robots. This allows us to train a contact and tendon force aware reinforcement learning controller that is deployable with minimized sim2real gap.

Fig. 2: Tendon-driven system setup for data collection using various springs and finger setup. The system can be represented as a mass spring system. The force sensor detects tendon tension forces applied by the servo motor to the system. For the finger setup, the tendon is routed directly through the load cell, allowing for data collection directly on a component of a dexterous hand.

Fig. 3: Motor motion that extends the spring and comes back to default position. Due to phenomena such as motor stick friction, the forces applied by the motor are non linear and result in high sim to real gap when ideal torque assumption is made in RL setting.

Fig. 4: Finger test bench setup. The two coupled joints are made out of flexible TPU material. The finger is driven by a one tendon. Antagonistic springs are attached to the back of the finger to allow for release mechanism with tendons are not engaged. Example of simulated tendon driven finger. Spring forces are applied at joints. $\alpha$ is an example desired angle used for the RL controller. The force estimated by the model is calculated at control frequency and is applied to multiple consecutive simulation steps.

Fig. 5: Prediction of tendon forces for a weak spring, strong spring and finger setup of a segment of the full testing trajectory corresponding to the 2.0 [s] steps as shown via the minimap. It can be seen that the transformer model manages to generalize to different measurements the best.

Fig. 6: RMSE of tendon force estimation for various springs and models over the full trajectory of steps. The error bars represent standard deviation of absolute error.

Fig. 7: Transformer tendon model estimation of forces given unblocked, blocked half way via semi-rigid item and blocked at the start with a compliant human hand. The command is a ramp of desired motor position commands. The arrows indicate approximate locations where the finger was blocked, resulting in measured position to no longer change signficantly. We can see that in all situations, the transformer models predicts the correct tendon forces and motor nonlinearities.

Fig. 8: Transformer tendon force estimation during a sinusoidal motion of a the finger test bench. The finger was randomly perturbed by hand, simulating finger contacts and slipping. Real motor positions are visualized on the right axis, showing at which positions the finger was blocked. The transformer model captures the latency of the system, whereas the ideal force directly tracking the position error, resulting in temporal offset from the real tendon force.

Fig. 9: Finger tip position when simulating the desired position command through an ideal tendon force source, the transformer model, and the real motion recorded on the real finger.

Fig. 10: Plot of end effector tracking controller trained on ideal tendon force model and transformer-based model deployed on a real finger test bench. Approximate finger motion is shown above. Region A shows how the ideal tendon force model fails to track finger opening due to lack of knowledge of the motor nonlinearities, causing it to overshoot and fully open the finger.

Sim-to-Real Transfer

Simulation-to-reality transfer adapts models and policies trained in simulation for reliable performance in physical or otherwise real-world settings.

See all 300 papers

Apr 10, 2026

Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks

Tendon drives paired with soft muscle actuation enable faster and safer robots while potentially accelerating skill acquisition. Still, these systems are rarely used in practice due to inherent nonlinearities, friction, and hysteresis, which complicate modeling and control. So far, these challenges have hindered policy transfer from simulation to real systems. To bridge this gap, we propose a sim-to-real pipeline that learns a neural network model of this complex actuation and leverages established rigid body simulation for the arm dynamics and interactions with the environment. Our method, called Generalized Actuator Network (GenAN), enables actuation model identification across a wide range of robots by learning directly from joint position trajectories rather than requiring torque sensors. Using GenAN on PAMY2, a tendon-driven robot powered by pneumatic artificial muscles, we successfully deploy dynamic but precise goal-reaching, ball-in-a-cup, and table tennis policies, trained entirely in simulation. To the best of our knowledge, this result constitutes the first successful sim-to-real transfer for a four-degrees-of-freedom muscle-actuated robot arm.

Jan Schneider, Mridul Mahajan, Le Chen +4

Max Planck Institute for Intelligent Systems, Tübingen · Boston University · ELLIS Institute Tübingen +4

Sim-to-Real Transfer Robotics Robotic Control Robot Dynamics Identification Tendon-Driven Actuation

Ask PDF

Aug 23, 2026

Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning

Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge, particularly when dealing with low-level torque control, where even small modelling inaccuracies can lead to unstable or unsafe behaviours. In this work, we propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms by combining trajectory matching, parameter optimization via genetic algorithms, and domain randomization. Using the 7-DOF Franka Emika Panda robot, we first identify friction, inertia, and gravity compensation parameters by minimizing the error between real and simulated joint trajectories. These calibrated dynamics are then used to train a TQC-based reinforcement learning agent in simulation. The trained policy is evaluated in both Gazebo and MuJoCo environments, and finally deployed on the real robot. Our results demonstrate a significant improvement in tracking accuracy and policy robustness after parameter tuning, with smooth policy transfer from simulation to the Real-World across multiple target-reaching tasks. This work highlights the effectiveness of accurate physical modelling in enabling stable and generalizable torque-based reinforcement learning policies.

Davide Bargellini, Alex Pasquali, Andrea Govoni +2

Department of Electrical, Electronic, and Information Engineering (DEI), University of Bologna, 40136 Bologna, Italy · Robotics and Mechatronics (RaM) group, University of Twente, Enschede, The Netherlands

Sim-to-Real Transfer Robotic RL RL Control

Ask PDF

Aug 28, 2026

Aero Hand Open: A Simulation-Ready Tendon-Driven Hand for Dexterous Manipulation Learning

Tendon-driven hands are anthropomorphic, and moving the actuators off the joints is what makes a hand of this capability affordable to build. Two effects produce that saving. Routing force through a cable removes the requirement that a motor fit inside the joint it drives, so smaller and cheaper motors suffice, and one motor can drive several joints through a single cable, so fewer motors are needed. They are also harder to learn on than a direct-drive hand. The underactuated transmission that produces the saving is itself difficult to represent in a simulator, and the joints one cable drives are not independently commandable. We present Aero Hand Open, a tendon-driven anthropomorphic hand that is released simulation-ready. Three things ship with it. A simulation model reproduces the cable transmission itself. An identified actuation map connects that model to the motor commands in both directions, including the three-way coupling of the thumb. A reinforcement learning package trains policies for the hand. Together they let a policy be trained entirely in simulation and run on the hand with no fine-tuning and no state estimation. We release the mechanical design, the simulation model, the identified mapping, the training environment and the deployment stack.

Nan Wang, Mohit Yadav, Jonathan Wulff +5

Chestnut Robotics

Robotic Hand Design Robotics Simulation Sim-to-Real Transfer Robotic Manipulation Tendon-Driven Actuation

Ask PDF

Abstract

Sim-to-Real Transfer

Explore similar work

Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks

Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning

Aero Hand Open: A Simulation-Ready Tendon-Driven Hand for Dexterous Manipulation Learning