Sim-to-Real Transfer

Ask about this topic

Overview

The field studies how to make models and control policies trained in simulation work reliably when they encounter real sensors, physics, objects, and environments. Common approaches include domain randomization, system identification and simulator calibration, real-to-sim-to-real data loops, and shared representations that align simulated and real visual, tactile, or proprioceptive inputs. Researchers apply these methods to tasks such as robotic manipulation, locomotion, navigation, and vehicle control, often testing zero-shot deployment alongside adaptation with limited real-world data. A central tension is that more realistic simulation can improve transfer but is costly to build, while broad randomization may not capture important mismatches; high simulation scores therefore do not always predict real-world performance.

Latest papers

CardsList

319 papers70 papers in the last four weeks, up 84% on the four weeks before. 0.4% of all new papers.

  1. Simulator Adaptation for Sim-to-Real Learning of Legged Locomotion via Proprioceptive Distribution Matching

    Apr 13, 2026Jeremy Dao, Alan FernSim-to-Real TransferQuadruped Locomotion

    Collaborative Robotics and Intelligent Systems Institute, Oregon State University, Corvallis, Oregon, 97331, USA.

  2. DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks

    Apr 13, 2026Yueci Deng, Guiliang Liu, Kui JiaWorld Models for RoboticsLatent World Models

    DexForce AI

  3. Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks

    Apr 10, 2026Jan Schneider, Mridul Mahajan, Le Chen +4Sim-to-Real TransferRobotics

    Max Planck Institute for Intelligent Systems, Tübingen · Boston University · ELLIS Institute Tübingen +4

  4. SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds

    Apr 9, 2026Yunsong Zhou, Hangxu Liu, Xuekun Jiang +12Sim-to-Real TransferSynthetic Data Generation for Robotics

    Shanghai AI Lab · Fudan University · Shanghai Jiao Tong University +1

  5. AgiPIX: Bridging Simulation and Reality in Indoor Aerial Inspection

    Apr 9, 2026Sasanka Kuruppu Arachchige, Juan Jose Garcia, Changda Tian +4Aerial RoboticsRobotic Inspection

    Computing Sciences department at Tampere University, Finland · U2IS at ENSTA, Paris · Institute of Computer Science, Foundation for Research and Technology - Hellas (FORTH), Heraklion, Greece.

  6. GaussFly: Contrastive Reinforcement Learning for Visuomotor Policies in 3D Gaussian Fields

    Apr 6, 2026Yuhang Zhang, Mingsheng Li, Yujing Shang +4Visuomotor Policy LearningVisual RL

    School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, Singapore · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore · College of Design and Engineering, National University of Singapore, Singapore 119077, Singapore +1

  7. Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Driving

    Apr 3, 2026Zilin Huang, Zhengyang Wan, Zihao Sheng +3Sim-to-Real TransferRL for Autonomous Driving

    Department of Civil and Environmental Engineering, University of Wisconsin-Madison, Madison, WI, 53706, USA

  8. Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robots

    Apr 3, 2026Junyang Liang, Yuxuan Liu, Yabin Chang +5Sim-to-Real TransferRobot Policy Adaptation

    Shenzhen University, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China

  9. Tune to Learn: How Controller Gains Shape Robot Policy Learning

    Apr 2, 2026Antonia Bronars, Younghyo Park, Pulkit AgrawalRobot Policy LearningRobotic Control

    MIT · MIT Improbable AI Lab

  10. Sim-to-Real Fruit Detection Using Synthetic Data: Quantitative Evaluation and Embedded Deployment with Isaac Sim

    Mar 30, 2026Martina Hutter-MironovovaSim-to-Real TransferSynthetic-to-Real Domain Adaptation

    YASKAWA Europe GmbH

  11. Tac2Real: Reliable and GPU Visuotactile Simulation for Online Reinforcement Learning and Zero-Shot Real-World Deployment

    Mar 30, 2026Ningyu Yan, Shuai Wang, Xing Shen +4Robotics SimulationSim-to-Real Transfer

    Shanghai Artificial Intelligence Laboratory. · The Hong Kong University of Science and Technology. · Shanghai Jiao Tong University +1

  12. Maximum Entropy Behavior Exploration for Sim2Real Zero-Shot Reinforcement Learning

    Mar 26, 2026Jiajun Hu, Nuria Armengol Urpi, Jin Cheng +1Zero-Shot RLReinforcement Learning

    Department of Mechanical Engineering, EPFL · Department of Computer Science, ETH Zurich · Max Planck Institute for Intelligent Systems

  13. RealMaster: Lifting Rendered Scenes into Photorealistic Video

    Mar 24, 2026Dana Cohen-Bar, Ido Sobol, Raphael Bensadoun +5Video-to-Video TranslationSim-to-Real Transfer

    TEL AVIV UNIVERSITY · REALITY LABS, META · TECHNION

  14. Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Models

    Mar 24, 2026Ruixing Jin, Zicheng Zhu, Ruixiang Ouyang +4Domain RandomizationRobotics Simulation

    School of Data Science, The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute

  15. PAM: A Pose-Appearance-Motion Engine for Sim-to-Real HOI Video Generation

    Mar 23, 2026Mingju Gao, Kaisen Yang, Huan-ang Gao +13HOI GenerationControllable Video Generation

    Peking University · Tsinghua University · SJTU +3

  16. Sim-to-Real of Humanoid Locomotion Policies via Joint Torque Space Perturbation Injection

    Mar 23, 2026Junhyeok Rui Cha, Woohyun Cha, Jaeyong Shin +2Sim-to-Real TransferRobotics Simulation

    Department of Intelligence and Information, Graduate School of Convergence Science and Technology, Seoul National University, Republic of Korea · 1X Technologies. This work was conducted while the author was at Seoul National University · Department of Intelligence and Information, Graduate School of Convergence Science and Technology, ASRI, AIIS, Seoul National University, Republic of Korea, and Advanced Institute of Convergence Technology (AICT), Suwon, Republic of Korea. He is the corresponding author of this paper.

  17. Swim2Real: VLM-Guided System Identification for Sim-to-Real Transfer

    Mar 21, 2026Kevin Qiu, Kyle Walker, Mike Y. Michelis +2Robot System IdentificationSim-to-Real Transfer

    University of Warsaw · IDEAS NCBR · EPFL +2

  18. AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learning

    Mar 20, 2026Huihua Zhao, Rafael Cathomen, Lionel Gulich +6RL for RoboticsHumanoid Robot Control

  19. SOFTMAP: Sim2Real Soft Robot Forward Modeling via Topological Mesh Alignment and Physics Prior

    Mar 19, 2026Ziyong Ma, Uksang Yoo, Jonathan Francis +3Soft RoboticsSim-to-Real Transfer

    Robotics Institute, Carnegie Mellon University, USA · Bosch Center for Artificial Intelligence, USA · School of Computer Science, The University of Sydney, Australia +2

  20. Ontology-Guided Diffusion for Zero-Shot Visual Sim2Real Transfer

    Mar 19, 2026Mohamed Youssef, Mayar Elfares, Anna-Maria Meer +2Sim-to-Real TransferImage-to-Image Translation

    University of Stuttgart, Germany · Fraunhofer IPA, Germany

  21. Scaling Sim-to-Real VLA Reinforcement Learning with Generative 3D Worlds

    Mar 19, 2026Andrew Choi, Xinjie Wang, Zhizhong Su +1Sim-to-Real TransferVision-Language-Action Models

    Horizon Robotics

  22. SLowRL: Safe Low-Rank Adaptation Reinforcement Learning for Locomotion

    Mar 17, 2026Elham Daneshmand, Shafeef Omar, Glen Berseth +2RL Fine-TuningRL for Legged Locomotion

    McGill University · Mila - Quebec AI Institute · Technical University of Munich +1

  23. Efficient and Reliable Teleoperation through Real-to-Sim-to-Real Shared Autonomy

    Mar 17, 2026Shuo Sha, Yixuan Wang, Binghao Huang +2Robot TeleoperationShared Autonomy

    Columbia University · University of Pennsylvania

  24. MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation

    Mar 17, 2026Abhay Deshpande, Maya Guru, Rose Hendrix +23Sim-to-Real TransferZero-Shot Robotic Manipulation

    Allen Institute for AI · Technische Universität Darmstadt · University of Washington +2

  25. Bridging the Simulation-to-Reality Gap in Electron Microscope Calibration via VAE-EM Estimation

    Mar 17, 2026Jilles S. van Hulst, W. P. M. H. Heemels, Duarte J. AntunesElectron MicroscopyScanning Transmission Electron Microscopy

    The authors are with the Control Systems Technology Section, Department of Mechanical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands.

  26. ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priors

    Mar 16, 2026Zifan Xu, Ran Gong, Maria Vittoria Minniti +10Long-Horizon Robotic ManipulationRobot Skill Learning

    Robotics and AI Institute · University of Texas at Austin · Sony AI

  27. Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

    Mar 16, 2026Jacob Levy, Tyler Westenbroek, Kevin Huang +6World Models for RoboticsRobot Policy Adaptation

    University of Texas at Austin · University of Washington · FieldAI

  28. Benchmarking Machine Learning Approaches for Polarization Mapping in Ferroelectrics Using 4D-STEM

    Mar 16, 2026Matej Martinc, Goran Dražič, Anton Kokalj +5Sim-to-Real TransferMaterials Science

    Jožef Stefan Institute, Jamova 39, Ljubljana, Slovenia

  29. HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation

    Mar 16, 2026Xingyi Wang, Chenyun Zhang, Weiji Xie +4Sim-to-Real TransferRL for Robotics

    Zhejiang University · Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Jiao Tong University +1

  30. CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control

    Mar 16, 2026Gelu Liu, Teng Wang, Zhijie Wu +3Robotic ControlRL for Robotics

    School of Electronics and Communication Engineering and Shenzhen Key Laboratory of Navigation and Communication Integration, Sun Yat-sen University, Shenzhen 518107, China