Real-to-Sim Transfer

Follow this topic Ask about this topic

Latest papers

14 papers4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

GRASE: Geometry-native Real-to-sim Agentic Scene Modeling from Video
Oct 15, 2026Zhaorui Lin, Haohan Liu, Zhen Chen +1Real-to-Sim Transfer 3D Scene Representation
Xi’an Jiaotong University (xjtu.edu.cn) · Fuzhou University (fzu.edu.cn) · Adobe Systems (adobe.com) +1
RoboSimStudio: An Interactive Toolkit for Human and Agent Rapid Real2Sim of Robot Manipulation Tasks
Oct 13, 2026Kaiyuan Chen, Shuangyu Xie, Uksang Yoo +9Real-to-Sim Transfer Robotics Simulation
University of California, Berkeley · NVIDIA
YUBI-Real2Sim: 3DGS Scene Verification for Reliable Policy Evaluation
Oct 13, 2026Hiroki Nishizawa, Takehiko Ohkawa, Nicola Scianca +3Real-to-Sim Transfer Robotic Policy Evaluation
AI Robot Association (AIRoA) · Waseda University
AIM: Adaptive Interaction Modeling Networks for Real-to-Sim Soft-Body Simulation
Oct 5, 2026Tiancheng Yang, Dingshuo Chen, Tianle Chen +2Robotics Simulation Robot Dynamics Learning
NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences +1
Getting Out and Getting Back: World and Behavior Grounding in Real2Sim2Real Co-Training
Sep 30, 2026Samuel Liu, Youngsun Kim, Martin Matak +1Sim-to-Real Transfer Synthetic Data Generation for Robotics
University of Cambridge · Industrial Next
ZeroBot: Learning from Scratch in Minutes with Generative Real2Sim
Sep 27, 2026Ivan Kapelyukh, Xiaohan Zhang, Stephen James +2Robotic RL Robotic Manipulation
Imperial College London · Robotics and AI Institute
Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim
Sep 27, 2026Ivan Kapelyukh, Yafei Hu, Ran Gong +6Real-to-Sim Transfer Robotics Simulation
Robotics and AI Institute · Imperial College London
RealSimLoop: Online Real-to-Sim Adaptation via Differentiable Reduced-Order Simulation with Vision Feedback
Sep 9, 2026Zhihao Cen, Chuhua Xian, Hailin Sun +6Real-to-Sim Transfer Physics-Based Simulation
South China University of Technology, China · The Chinese University of Hong Kong, China · The Chinese University of Hong Kong / Multi-scale Medical Robotics Centre, Hong Kong SAR, China +1
R2S-EGO: Dual-Proxy Refinement for Sparse-Capture Real-to-Sim
Aug 7, 2026Shuai Fang, Xin Deng, Yuchen Kang +2Robotics Simulation Synthetic Data Generation for Robotics
XPENG Robotics · The Hong Kong Polytechnic University
Real2Sim2Real for Vision-Language-Action Manipulation: An AMD ROCm-Based Pipeline
Jul 25, 2026Qing Yang, Xun Wang, Ziguan Wang +3Language-Conditioned Robot Manipulation Robotics Simulation
AMD AIG Team · Beijing Institute of Technology
Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
Jul 21, 2026Guanxiong Chen, Qianjun Xia, Jiawei Peng +24World Models for Robotics Robotics Simulation
University of British Columbia. · National University of Singapore. · Johns Hopkins University. +4
MoSA: Motion-constrained Stress Adaptation for Mitigating Real-to-Sim Gap in Continuum Dynamics via Learning Residual Anisotropy
May 21, 2026Jiaxu Wang, Junhao He, Jingkai Sun +5Residual Learning Sim-to-Real Transfer
Hong Kong University of Science and Technology, Guangzhou, China · MMLab, Chinese University of Hong Kong, Hong Kong SAR · The University of Hong Kong, Hong Kong SAR
UniPR: Unified Object-level Real-to-Sim Perception and Reconstruction from a Single Stereo Pair
Mar 20, 2026Chuanrui Zhang, Yingshuang Zou, ZhengXian Wu +3Real-to-Sim Transfer 3D Reconstruction
NTU · Tencent Robotics X · HKUST +2
SynthRender and I-AsSET: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perception
Feb 24, 2026Jose Moises Araya-Martinez, Thushar Tom, Adrián Sanchis Reig +3Sim-to-Real Transfer Synthetic-to-Real Domain Adaptation
Technical University Berlin, Industrial Automation Technology, Berlin, Germany · Mercedes-Benz AG, Future Manufacturing Technologies, Sindelfingen, Germany · Technical University Braunschweig, Institute for Cognitive Robotics, Braunschweig, Germany

Oct 15, 2026J/K move · Enter open · S save

GRASE: Geometry-native Real-to-sim Agentic Scene Modeling from Video

Zhaorui Lin, Haohan Liu, Zhen Chen, Xiangru Huang

Xi’an Jiaotong University (xjtu.edu.cn) · Fuzhou University (fzu.edu.cn) · Adobe Systems (adobe.com)+1

Real-to-Sim Transfer 3D Scene Representation 3D Reconstruction Multi-View 3D Reconstruction

Open paper AskPDF arXiv

Real-to-sim conversion aims to turn a captured video of a real environment into an editable 3D scene while preserving the geometry and layout of what was observed. Existing methods often follow a one-pass, loosely coupled paradigm: each stage receives only limited feedback from previous stages, allowing errors to propagate without reliable correction and leaving the final scene inconsistent. We present GRASE, an agentic framework that closes this loop by treating the captured multi-view video observations as a standing reference throughout scene modeling. Instead of committing to intermediate estimates, GRASE revises its current hypothesis whenever rendered evidence disagrees with the capture, overcoming the fixed intermediate assumptions of conventional pipelines. Specifically, GRASE first fits the architectural shell and its openings from depth and multi-view cues. An agent then places generated assets in a shared coordinate frame while checking their spatial relationships. Finally, each asset's geometry and placement are refined iteratively to improve agreement across the observed views. By coupling these steps through feedback, GRASE localizes structural discrepancies and revises only the responsible components, while preserving cross-view consistency. The resulting 3D scenes are faithful to the input, consistent across views, and suitable for simulations. In our experiments, GRASE substantially outperforms other state-of-the-art methods, including a direct real-to-sim conversion using GPT6-Astra.