cs.RONov 14, 2026

Residual Reinforcement Learning for Visual Servo Control with Zero-Shot Sim-to-Real Transfer

Authors: Mahdi Nobar, Alisa Rupenyan, John Lygeros, Jürg Keller

Organizations: Automatic Control Laboratory, ETH Zürich, Zurich, Switzerland · ZHAW Center for AI, Zurich University of Applied Sciences, Winterthur, Switzerland · Automation Institute, FHNW, Windisch, Switzerland

Abstract

We address tracking a moving object on a conveyor with a Franka Emika Panda robot arm, despite an imprecise digital twin, measurement errors, and noise. Our novel approach augments a loop-shaped inverse-Jacobian PI controller with a residual policy learned via Soft Actor-Critic in the digital twin. A delay-compensated Kalman filter fuses out-of-sequence camera measurements to provide latency-aware moving target position estimates. We compare the achieved performance to a performance bound for a class of model-based controllers derived from a frequency-domain sensitivity analysis, quantifying the minimum achievable error imposed by uncertain kinematic model mismatch. Simulation and hardware experiments indicate that our residual policy drives tracking error below the bound. Across conveyor speeds (27.9-41.8 mm/s), our method reduces mean tracking errors from 1.1-1.4 mm (baseline inverse-Jacobian PI) to 0.3-0.7 mm. These results show that learning residual corrections in a limited-fidelity simulator, combined with delay compensation, can substantially improve model-based control under modeling and measurement uncertainties while maintaining reliable sim-to-real transfer.

Figures & tables

Explore similar work

CardsList
  1. Stability-aware Residual Reinforcement Learning Framework for Robotic Manipulator Disturbance Compensation

    Sep 18, 2026Jihong Kim, Joonhyuk Kwon, Hwa Soo Kim +2Input-to-State StabilityRobotic Control

  2. Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning

    Aug 23, 2026Davide Bargellini, Alex Pasquali, Andrea Govoni +2Sim-to-Real TransferRobotic RL

  3. Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning

    Jul 2, 2026Satoshi Yamamori, Koji Ishihara, Kenjiro Minamikawa +4Robotic ControlSim-to-Real Transfer