Robotics Simulation

Ask about this topic

Overview

Robotics simulation studies how to represent robot bodies, sensors, environments, and interactions so researchers can test control and learning safely and repeatably before hardware deployment. Approaches range from rigid-body and contact physics engines such as MuJoCo and Isaac Sim, to finite-element models for soft materials, differentiable or learned dynamics, and generated scenes for navigation and manipulation. GPU-parallel reinforcement learning, synthetic demonstrations, digital twins, and simulation-in-the-loop planning help scale training, while benchmarks and hardware-in-the-loop tests assess transfer. The central tension is balancing physical and sensor fidelity with speed and reproducibility: visual realism alone does not ensure correct interactions, and simulated success does not always predict real-world performance without calibration and validation.

Latest papers

CardsList

322 papers59 papers in the last four weeks, up 64% on the four weeks before. 0.3% of all new papers.

Mar 1, 2026

D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping

Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real-world dynamics remains a significant challenge, especially in physical parameter identification. In this work, we introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine, enabling object mass identification from real-world visual observations and robot control signals, while enabling grasping policy learning simultaneously. Through optimizing the mass of the manipulated object, our method automatically builds high-fidelity and physically plausible digital twins. Additionally, we propose a novel approach to train force-aware grasping policies from limited data by transferring feasible human demonstrations into simulated robot demonstrations. Through comprehensive experiments, we demonstrate that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values. Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping, effectively reducing the sim-to-real gap.

Haozhe Lou, Mingtong Zhang, Haoran Geng +9

Physical Superintelligence (PSI) Lab, University of Southern California · Viterbi School of Engineering, University of Southern California · Department of EECS, University of California, Berkeley

Robotic GraspingRobotics SimulationSim-to-Real TransferRobot Dynamics IdentificationForce-Controlled Robotic Manipulation
AskPDF
Feb 27, 2026

MicroPush: A Lightweight Simulator for Sim-to-Real Microrobotic Pushing and Assembly

Developing learning-based autonomy for microrobotic manipulation is difficult because physical trials are slow, variable, and poorly suited to large-scale training. We present MicroPush, a lightweight simulation and benchmarking environment for pushing and assembly with magnetic rolling microrobots. The simulator combines overdamped planar dynamics with calibrated rolling actuation, reduced-order contact handling, short-range interaction corrections, and optional background flow, while providing common interfaces for planning, model-based control, reinforcement learning (RL), interactive visualization, and headless evaluation. In no-render tests, MicroPush remains substantially faster than real time, reaching 37.0x real time with 100 passive particles. We further evaluate the simulator through zero-shot sim-to-real policy transfer. A Residual RL-MPC policy is trained entirely in MicroPush, frozen after simulation training, and deployed on the physical system without real-world policy fine-tuning. A 20 μm magnetic microrobot pushes a similarly sized silica particle along a figure-eight trajectory with 1.91 μm median and 4.65 μm 95th-percentile path error, and the same autonomy stack performs sequential line, triangle, and six-particle hexagonal assembly. These results show that a deliberately reduced-order simulator can provide an efficient training and development substrate for sim-to-real microrobotic autonomy.

Yanda Yang, Zion Lee, Sambeeta Das

Department of Mechanical Engineering, University of Delaware, Newark, DE 19716, USA

Robot Policy LearningRobotics SimulationSim-to-Real TransferRobotic ManipulationRobot Manipulation BenchmarksRobotic Assembly
AskPDF
Feb 26, 2026

Simple Models, Real Swimming: Digital Twins for Tendon-Driven Underwater Robots

Mimicking the graceful motion of swimming animals remains a core challenge in soft robotics due to the complexity of fluid-structure interaction and the difficulty of controlling soft, biomimetic bodies. Existing modeling approaches are often computationally expensive and impractical for complex control or reinforcement learning needed for realistic motions to emerge in robotic systems. In this work, we present a tendon-driven fish robot modeled in an efficient underwater swimmer environment using a simplified, stateless hydrodynamics formulation implemented in the widespread robotics framework MuJoCo. With just two real-world swimming trajectories, we identify five fluid parameters that allow a matching to experimental behavior and generalize across a range of actuation frequencies. We show that this stateless fluid model can generalize to unseen actuation and outperform classical analytical models such as the elongated body theory. This simulation environment runs faster than real-time and can easily enable downstream learning algorithms such as reinforcement learning for target tracking, reaching a 93% success rate. Due to the simplicity and ease of use of the model and our open-source simulation environment, our results show that even simple, stateless models -- when carefully matched to physical data -- can serve as effective digital twins for soft underwater robots, opening up new directions for scalable learning and control in aquatic environments.

Mike Y. Michelis, Nana Obayashi, Josie Hughes +1

Soft Robotics Lab and ETH AI Center, ETH Zurich, Switzerland · Prema Lab, New York University, USA · CREATE Lab, EPFL, Switzerland

Underwater RoboticsRobotics SimulationDigital TwinsSoft RoboticsRobot Dynamics Identification
AskPDF
Feb 24, 2026

CableRobotGraphSim: A Graph Neural Network for Modeling Partially Observable Cable-Driven Robot Dynamics

General-purpose simulators have accelerated the development of robots. Traditional simulators based on first-principles, however, typically require full-state observability or depend on parameter search for system identification. This work presents CableRobotGraphSim, a novel Graph Neural Network (GNN) model for cable-driven robots that aims to address shortcomings of prior simulation solutions. By representing cable-driven robots as graphs, with the rigid-bodies as nodes and the cables and contacts as edges, this model can quickly and accurately match the properties of other simulation models and real robots, while ingesting only partially observable inputs. Accompanying the GNN model is a sim-and-real co-training procedure that promotes generalization and robustness to noisy real data. This model is further integrated with a Model Predictive Path Integral (MPPI) controller for closed-loop navigation, which showcases the model's speed and accuracy.

Nelson Chen, William R. Johnson, Rebecca Kramer-Bottiglio +2

1Rutgers University · 2Yale University

Robotics SimulationRobot Dynamics Learning
AskPDF
Feb 23, 2026

Smoothly Differentiable and Efficiently Vectorizable Contact Manifold Generation

Simulating rigid-body dynamics with contact in a fast, massively vectorizable, and smoothly differentiable manner is highly desirable in robotics. An important bottleneck faced by existing differentiable simulation frameworks is contact manifold generation: representing the volume of intersection between two colliding geometries via a discrete set of properly distributed contact points. A major factor contributing to this bottleneck is that the related routines of commonly used robotics simulators were not designed with vectorization and differentiability as a primary concern, and thus rely on logic and control flow that hinder these goals. We instead propose a framework designed from the ground up with these goals in mind, by trying to strike a middle ground between: i) convex primitive based approaches used by common robotics simulators (efficient but not differentiable), and ii) mollified vertex-face and edge-edge unsigned distance-based approaches used by barrier methods (differentiable but inefficient). Concretely, we propose: i) a representative set of smooth analytical signed distance primitives to implement vertex-face collisions, and ii) a novel differentiable edge-edge collision routine that can provide signed distances and signed contact normals. The proposed framework is evaluated via a set of didactic experiments and benchmarked against the collision detection routine of the well-established Mujoco XLA framework, where we observe a significant speedup. Supplementary videos can be found at https://github.com/bekeronur/contax, where a reference implementation in JAX will also be made available at the conclusion of the review process.

Onur Beker, Andreas René Geist, Anselm Paulus +4

University of Tübingen · Idiap Research Institute · EPFL

Robotics SimulationDifferentiable PhysicsMultibody Dynamics
AskPDF
Feb 21, 2026

RotorSuite: A MATLAB/Simulink Toolbox for Tilt Multi-Rotor UAV Modeling

In recent years, aerial platforms have evolved from passive flying sensors into versatile, contact-aware robotic systems, leading to rapid advances in platform design. Standard coplanar and collinear quadrotors have been complemented by modern tilted and tilting multi-rotor platforms with enhanced maneuverability. To properly analyze, control, and validate the performance of these emerging platforms, an accurate modeling step is required; however, this can be time-consuming, user-dependent and error-prone. To address this issue, we propose a MATLAB/Simulink toolbox for modeling and simulating the dynamics of a broad class of multi-rotor platforms through both an analytical and physics-based approaches. The toolbox, named RotorSuite, is provided with comprehensive documentation and example use cases, representing a valuable tool for didactic, research, and industrial development purposes.

Nicola Cigarini, Giulia Michieletto, Angelo Cenedese

Department of Information Engineering University of Padova, Vicenza, Italy · Department of Land, Environment, Agriculture and Forestry, University of Padova, Legnaro, Italy · Department of Management and Engineering, University of Padova, Padova, Italy +1

Aerial RoboticsRobotics Simulation
AskPDF
Feb 21, 2026

CLASH: Collision Learning via Augmented Sim-to-real Hybridization to Bridge the Reality Gap

The sim-to-real gap, particularly in the inaccurate modeling of contact-rich dynamics like collisions, remains a primary obstacle to deploying robot policies trained in simulation. Conventional physics engines often trade accuracy for computational speed, leading to discrepancies that prevent direct policy transfer. To address this, we introduce Collision Learning via Augmented Sim-to-real Hybridization (CLASH), a data-efficient framework that learns a parameter-conditioned impulsive collision surrogate model and integrates it as a plug-in module within a standard simulator. CLASH first distills a base model from an imperfect simulator (MuJoCo) using large-scale simulated collisions to capture reusable physical priors. Given only a handful of real collisions (e.g., 10 samples), it then (i) performs gradient-based identification of key contact parameters and (ii) applies small-step, early-stopped fine-tuning to correct residual sim-to-real mismatches while avoiding overfitting. The resulting hybrid simulator not only achieves higher post-impact prediction accuracy but also reduces the wall-clock time of collision-heavy CMA-ES search by 42-48% compared to MuJoCo. We demonstrate that policies obtained with our hybrid simulator transfer more robustly to the real world, doubling the success rate in sequential pushing tasks with reinforcement learning and significantly increase the task performance with model-based control.

Haotian He, Ning Guo, Siqi Shi +2

School of Mathematical Sciences, Peking University. · School of Artificial Intelligence, Shanghai Jiao Tong University. · School of Mathematical Sciences, Beijing Normal University.

Sim-to-Real TransferRobotics SimulationPhysics-Based SimulationRobot Dynamics LearningData-Efficient Robot Learning
AskPDF
Feb 16, 2026

A ROS2 Benchmarking Framework for Hierarchical Control Strategies in Mobile Robots for Mediterranean Greenhouses

Mobile robots operating in agroindustrial environments, such as Mediterranean greenhouses, are subject to challenging conditions, including uneven terrain, variable friction, payload changes, and terrain slopes, all of which significantly affect control performance and stability. Despite the increasing adoption of robotic platforms in agriculture, the lack of standardized, reproducible benchmarks impedes fair comparisons and systematic evaluations of control strategies under realistic operating conditions. This paper presents a comprehensive benchmarking framework for evaluating mobile robot controllers in greenhouse environments. The proposed framework integrates an accurate three dimensional model of the environment, a physics based simulator, and a hierarchical control architecture comprising low, mid, and high level control layers. Three benchmark categories are defined to enable modular assessment, ranging from actuator level control to full autonomous navigation. Additionally, three disturbance scenarios payload variation, terrain type, and slope are explicitly modeled to replicate real world agricultural conditions. To ensure objective and reproducible evaluation, standardized performance metrics are introduced, including the Squared Absolute Error (SAE), the Squared Control Input (SCI), and composite performance indices. Statistical analysis based on repeated trials is employed to mitigate the influence of sensor noise and environmental variability. The framework is further enhanced by a plugin based architecture that facilitates seamless integration of user defined controllers and planners. The proposed benchmark provides a robust and extensible tool for the quantitative comparison of classical, predictive, and planning based control strategies in realistic conditions, bridging the gap between simulation based analysis and real world agroindustrial applications.

Fernando Cañadas-Aránega, Francisco J. Mañas-Álvarez, José L- Guzmán +2

Department of Informatics, CIESOL, ceiA3, Universidad de Almería, Ctra. Sacramento s/n, Almería, 04120, Spain · Department of Computer Science and Automation, National University of Distance Education (UNED), C/ Juan del Rosal 16, Madrid, 28040, Spain · Department of Engineering, CIESOL, ceiA3, Universidad de Almería, Ctra. Sacramento s/n, Almería, 04120, Spain

Robotic ControlMobile RoboticsAgricultural RoboticsRobotics Simulation
AskPDF
Feb 16, 2026

Neurosim: A Fast Simulator for Neuromorphic Robot Perception

Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .

Richeek Das, Pratik Chaudhari

GRASP Laboratory, University of Pennsylvania

Robotics SimulationNeuromorphic Computing
AskPDF
Feb 16, 2026

RoboSolver: A Multi-Agent Large Language Model Framework for Solving Robotic Arm Problems

This study proposes an intelligent multi-agent framework built on LLMs and VLMs and specifically tailored to robotics. The goal is to integrate the strengths of LLMs and VLMs with computational tools to automatically analyze and solve problems related to robotic manipulators. Our developed framework accepts both textual and visual inputs and can automatically perform forward and inverse kinematics, compute velocities and accelerations of key points, generate 3D simulations of the robot, and ultimately execute motion control within the simulated environment, all according to the user's query. To evaluate the framework, three benchmark tests were designed, each consisting of ten questions. In the first benchmark test, the framework was evaluated while connected to GPT-4o, DeepSeek-V3.2, and Claude-Sonnet-4.5, as well as their corresponding raw models. The objective was to extract the forward kinematics of robots directly from textual descriptions. The results showed that the framework integrated with GPT-4o achieved the highest accuracy, reaching 0.97 in computing the final solution, whereas the raw model alone attained an accuracy of only 0.30 for the same task. Similarly, for the other two models, the framework consistently outperformed the corresponding raw models in terms of accuracy. The second benchmark test was identical to the first, except that the input was provided in visual form. In this test, the GPT-4o LLM was used alongside the Gemini 2.5 Pro VLM. The results showed that the framework achieved an accuracy of 0.93 in obtaining the final answer, which is approximately 20% higher than that of the corresponding raw model. The third benchmark test encompassed a range of robotic tasks, including simulation, control, velocity and acceleration computation, as well as inverse kinematics and Jacobian calculation, for which the framework achieved an accuracy of 0.97.

Hamid Khabazi, Ali F. Meghdari, Alireza Taheri

Social & Cognitive Robotics Laboratory, Center of Excellence in Design, Robotics, and Automation (CEDRA), Sharif University of Technology, Tehran, Iran · Chancellor, Fereshtegaan International Branch, Islamic Azad University, Tehran, Iran · Artificial Intelligence in Design and Complex Systems (AIDACS) Group, Sharif University of Technology, Tehran, Iran

Robot KinematicsRoboticsMulti-Agent LLM SystemsRobotic ControlVLMs for RoboticsRobotics Simulation
AskPDF
Feb 12, 2026

ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation with Moving Obstacles

Visual navigation models often struggle in real-world dynamic environments due to limited robustness to the sim-to-real gap and the difficulty of training policies tailored to target deployment environments (e.g., households, restaurants, and factories). Although real-to-sim navigation simulation using 3D Gaussian Splatting (GS) can mitigate these challenges, prior GS-based works have considered only static scenes or non-photorealistic human obstacles built from simulator assets, despite the importance of safe navigation in dynamic environments. To address these issues, we propose ReaDy-Go, a novel real-to-sim simulation pipeline that synthesizes photorealistic dynamic scenarios in target environments by augmenting a reconstructed static GS scene with dynamic human GS obstacles, and trains navigation policies using the generated datasets. The pipeline provides three key contributions: (1) a dynamic GS simulator that integrates static scene GS with a human animation module, enabling the insertion of animatable human GS avatars and the synthesis of plausible human motions from 2D trajectories, (2) a navigation dataset generation framework that leverages the simulator along with a robot expert planner designed for dynamic GS representations and a human planner, and (3) robust navigation policies to both the sim-to-real gap and moving obstacles. The proposed simulator generates thousands of photorealistic navigation scenarios with animatable human GS avatars from arbitrary viewpoints. ReaDy-Go outperforms baselines across target environments in both simulation and real-world experiments, demonstrating improved navigation performance even after sim-to-real transfer and in the presence of moving obstacles. Moreover, zero-shot sim-to-real deployment in an unseen environment indicates its generalization potential. Project page: https://syeon-yoo.github.io/ready-go-site/.

Seungyeon Yoo, Youngseok Jang, Dabin Kim +3

Department of Aerospace Engineering, Seoul National University, Seoul, Republic of Korea · InnoCORE AI-Transformed Aerospace Research Center, KAIST, Daejeon, Republic of Korea

Robotics SimulationRobot NavigationSynthetic Data Generation for RoboticsRobot Policy GeneralizationDynamic Obstacle Avoidance
AskPDF
Feb 5, 2026

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Vision-language-action models have advanced robotic manipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose a simulation-first framework to verify VLA architectures before real-world deployment and introduce MobileManiBench, a large-scale benchmark for mobile-based robotic manipulation. Built on NVIDIA Isaac Sim and powered by reinforcement learning, our pipeline autonomously generates diverse manipulation trajectories with rich annotations (language instructions, multi-view RGB-depth-segmentation images, synchronized object/robot states and actions). MobileManiBench features 2 mobile platforms (parallel-gripper and dexterous-hand robots), 2 synchronized cameras (head and right wrist), 630 objects in 20 categories, 5 skills (open, close, pull, push, pick) with over 100 tasks performed in 100 realistic scenes, yielding 300K trajectories. This design enables controlled, scalable studies of robot embodiments, sensing modalities, and policy architectures, accelerating research on data efficiency and generalization. We benchmark representative VLA models and report insights into perception, reasoning, and control in complex simulated environments, with all code, datasets, and models publicly released.

Wenbo Wang, Fangyun Wei, QiXiu Li +5

University of Sydney · 1Microsoft Research Asia · 3Tsinghua University

Mobile ManipulationRobotics SimulationSynthetic Data Generation for RoboticsVision-Language-Action ModelsRobot Manipulation BenchmarksRobot Manipulation Datasets
AskPDF
Feb 2, 2026

Bridging the Sim-to-Real Gap with multipanda_ros2: A Real-Time ROS2 Framework for Multimanual Systems

We present multipanda_ros2multipanda\_ros2, a novel open-source ROS2 architecture for multi-robot control of Franka Robotics robots. Leveraging ros2 control, this framework provides native ROS2 interfaces for controlling any number of robots from a single process. Our core contributions address key challenges in real-time torque control, including interaction control and robot-environment modeling. A central focus of this work is sustaining a 1kHz control frequency, a necessity for real-time control and a minimum frequency required by safety standards. Moreover, we introduce a controllet-feature design pattern that enables controller-switching delays of ≤2\le 2 ms, facilitating reproducible benchmarking and complex multi-robot interaction scenarios. To bridge the simulation-to-reality (sim2real) gap, we integrate a high-fidelity MuJoCo simulation with quantitative metrics for both kinematic accuracy and dynamic consistency (torques, forces, and control errors). Furthermore, we demonstrate that real-world inertial parameter identification can significantly improve force and torque accuracy, providing a methodology for iterative physics refinement. Our work extends approaches from soft robotics to rigid dual-arm, contact-rich tasks, showcasing a promising method to reduce the sim2real gap and providing a robust, reproducible platform for advanced robotics research.

Jon Škerlj, Seongjin Bien, Abdeldjallil Naceri +1

Munich Institute of Robotics and Machine Intelligence (MIRMI), Technische Universität München (TUM), Germany · Technische Universität Nürnberg (UTN), Germany · Mohamed Bin Zayed University of Artificial Intelligence, UAE

Robotic ControlRobot Dynamics IdentificationMulti-Robot SystemsRobotics SimulationSim-to-Real TransferRobotics
AskPDF
Jan 29, 2026

From Instruction to Event: Sound-Triggered Mobile Manipulation

Current mobile manipulation research predominantly follows an instruction-driven paradigm, where robots rely on predefined textual commands to execute tasks. However, this setting confines robots to a passive role, limiting robotic autonomy and the ability to react to dynamic environmental events. To address these limitations, we introduce Sound-Triggered Mobile Manipulation (STMM), where robots must actively perceive and interact with sound-emitting objects without explicit action instructions. To support STMM, we develop Habitat-Echo, a simulation platform that integrates sound rendering with physical interaction. We further propose a hierarchical baseline that translates high-level planning into low-level executions, where a task planner predicts a skill chain for policy models to execute sequentially. Experiments indicate the feasibility of perceiving auditory events and executing corresponding physical interactions without explicit instructions. Notably, in challenging multi-event scenarios, the robot successfully isolates the primary sources from overlapping acoustic interference to execute the first interactions, and subsequently proceeds to manipulate the secondary objects. These new challenges of planning-to-execution position STMM as a measurable research direction.The code and datasets will be released upon acceptance.

Hao Ju, Shaofei Huang, Hongyu Li +4

University of Macau · Beihang University · Hefei University of Technology

Robotics SimulationMobile RoboticsRobotic ManipulationRobot Task PlanningRobot Manipulation Benchmarks
AskPDF
Jan 20, 2026

LogicEnvGen: Task-Logic Driven Generation of Diverse Simulated Environments for Embodied AI

Simulated environments play an essential role in embodied AI, functionally analogous to test cases in software engineering. However, existing environment generation methods often emphasize visual realism (e.g., object diversity and layout coherence), overlooking a crucial aspect: logical diversity from the testing perspective. This limits the comprehensive evaluation of embodied agent adaptability and planning robustness across distinct simulated environments. To bridge this gap, we propose LogicEnvGen, a novel method driven by Large Language Models (LLMs) that adopts a top-down paradigm to generate logically diverse simulated environments as test cases for agents. Given an agent task, LogicEnvGen first analyzes its execution logic to construct decision-tree-structured behavior plans and then synthesizes a set of logical trajectories. Subsequently, it adopts a heuristic algorithm to refine the trajectory set, reducing redundant simulation. For each logical trajectory, which represents a potential task situation, LogicEnvGen correspondingly instantiates a concrete simulated environment. Furthermore, we introduce LogicEnvEval, a novel benchmark for simulated environment generation, with four quantitative metrics. Experimental results verify the lack of logical diversity in baselines and demonstrate that LogicEnvGen achieves 1.08-2.67x greater diversity, significantly improving the performance in revealing agent faults by 3.34%-72.00%.

Jianan Wang, Siyang Zhang, Bin Li +4

College of Computer Science and Technology, National University of Defense Technology · Intelligent Game and Decision Lab (IGDL), Beijing · School of Artifical Intelligence, Shanghai Jiao Tong University

Robotics SimulationAI Agent EvaluationSynthetic Benchmark GenerationEmbodied AI
AskPDF
Jan 5, 2026

Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot

The development of robust and generalizable robot learning models is critically contingent upon the availability of large-scale, diverse training data and reliable evaluation benchmarks. Collecting data in the physical world poses prohibitive costs and scalability challenges, and prevailing simulation benchmarks frequently suffer from fragmentation, narrow scope, or insufficient fidelity to enable effective sim-to-real transfer. To address these challenges, we introduce Genie Sim 3.0, a unified simulation platform for robotic manipulation. We present Genie Sim Generator, a large language model (LLM)-powered tool that constructs high-fidelity scenes from natural language instructions. Its principal strength resides in rapid and multi-dimensional generalization, facilitating the synthesis of diverse environments to support scalable data collection and robust policy evaluation. We introduce the first benchmark that pioneers the application of LLM for automated evaluation. It leverages LLM to mass-generate evaluation scenarios and employs Vision-Language Model (VLM) to establish an automated assessment pipeline. We also release an open-source dataset comprising more than 10,000 hours of synthetic data across over 200 tasks. Through systematic experimentation, we validate the robust zero-shot sim-to-real transfer capability of our open-source dataset, demonstrating that synthetic data can server as an effective substitute for real-world data under controlled conditions for scalable policy training. For code and dataset details, please refer to: https://github.com/AgibotTech/genie_sim.

Chenghao Yin, Da Huang, Di Yang +16

Robotic Data CollectionRobotics SimulationSynthetic Data Generation for RoboticsSynthetic Benchmark GenerationRobot Manipulation Benchmarks
AskPDF
Dec 1, 2025

TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation

Simulation provides a low-cost, scalable pathway to large-scale robotic manipulation data collection. However, existing 3D scene generation methods can rarely be applied directly to manipulation data synthesis, as their generated scenes often lack instance-level interactivity and physical plausibility. Focusing on tabletop manipulation, we propose TabletopGen, a training-free and automated tabletop scene generation and interactive simulation engine. Starting from text or a single image, we first obtain independent 3D object models via generative instance extraction. Second, we introduce a novel pose and scale alignment approach that recovers a collision-free scene layout using a Differentiable Rotation Optimizer and a Top-View Spatial Alignment mechanism. Finally, we assemble the generated scene in a physics simulator with collision geometry, yielding a stable, interactable environment for synthesizing multimodal manipulation data. Extensive experiments and user studies demonstrate that TabletopGen achieves state-of-the-art performance in visual fidelity, layout accuracy, and physical plausibility. Furthermore, we validate the executability of the collected trajectories on a real robotic arm via zero-shot real-to-sim-to-real policy transfer, indicating that TabletopGen can serve as a reliable data engine for robotic manipulation data synthesis.

Ziqian Wang, Yonghao He, Licheng Yang +6

School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences, Beijing, China · D-Robotics, Beijing, China +2

Robotic Data CollectionRobotics SimulationSynthetic Data Generation for Robotics3D Scene GenerationRobotic Manipulation3D Scene Layout Generation
AskPDF
Nov 11, 2025

CENIC: Convex Error-controlled Numerical Integration for Contact

State-of-the-art robotics simulators operate in discrete time. This requires users to choose a time step, which is both critical and challenging: large steps can produce non-physical artifacts, while small steps force the simulation to run slowly. Continuous-time error-controlled integration avoids such issues by automatically adjusting the time step to achieve a desired accuracy. But existing error-controlled integrators struggle with the stiff dynamics of contact, and cannot meet the speed and scalability requirements of modern robotics workflows. We introduce CENIC, a new variable-step integrator that brings together recent advances in convex time-stepping and error-controlled integration, inheriting benefits from both continuous integration and discrete time-stepping. CENIC runs at fast real-time rates comparable to discrete-time robotics simulators like MuJoCo, Drake and Isaac Sim, while also providing guarantees on accuracy and convergence.

Vince Kurtz, Alejandro Castro

Robotics SimulationMultibody Dynamics
AskPDF
Nov 10, 2025

Rapidly Learning Soft Robot Control via Implicit Time-Stepping

With the explosive growth of rigid-body simulators, policy learning in simulation has become the de facto standard for most rigid morphologies. In contrast, soft robotic simulation frameworks remain scarce and are seldom adopted by the soft robotics community. This gap stems partly from the lack of easy-to-use, general-purpose frameworks and partly from the high computational cost of accurately simulating continuum mechanics, which often renders policy learning infeasible. In this work, we demonstrate that rapid soft robot policy learning is indeed achievable via implicit time-stepping. Our simulator of choice, DisMech, is a general-purpose, fully implicit soft-body simulator capable of handling both soft dynamics and frictional contact. We further introduce delta natural curvature control, a method analogous to delta joint position control in rigid manipulators, providing an intuitive and effective means of enacting control for soft robot learning. To highlight the benefits of implicit time-stepping and delta curvature control, we conduct extensive comparisons across four diverse soft manipulator tasks against one of the most widely used soft-body frameworks, Elastica. With implicit time-stepping, parallel stepping of 500 environments achieves up to 6x faster speeds for non-contact cases and up to 40x faster for contact-rich scenarios. Finally, a comprehensive sim-to-sim gap evaluation--training policies in one simulator and evaluating them in another--demonstrates that implicit time-stepping provides a rare free lunch: dramatic speedups achieved without sacrificing accuracy.

Andrew Choi, Dezhong Tong, Xiaonan Huang

Horizon Robotics, Cupertino, CA, USA · Robotics Department at the University of Michigan, Ann Arbor, MI, USA

Robot Policy LearningRobotics SimulationContinuum Robot ControlSoft RoboticsDeformable Object Simulation
AskPDF
Jul 15, 2025

Robot Drummer: Learning Rhythmic Skills for Humanoid Drumming

Humanoid robots have seen remarkable advances in dexterity, balance, and locomotion, yet their role in expressive domains such as music performance remains largely unexplored. Musical tasks, like drumming, present unique challenges such as split-second timing, rapid contacts, and multi-limb coordination over performances lasting minutes. In this paper, we introduce Robot Drummer, a simulation framework for humanoid drumming across a diverse repertoire of songs. We formulate humanoid drumming as the realization of timed contact events encoded as a Rhythmic Contact Chain. To handle the long-horizon nature of musical performance, we decompose each track into fixed-length segments and train a single policy across all segments in parallel using reinforcement learning. Through extensive experiments on over thirty popular tracks, our results demonstrate that Robot Drummer consistently achieves high F1 scores and enables efficient learning of long-horizon musical performances. The learned behaviors exhibit emergent human-like drumming strategies, such as cross-arm strikes, and adaptive stick assignments, demonstrating the potential of reinforcement learning to bring humanoid robots into the domain of creative musical performance. Project page: robotdrummer.github.io

Asad Ali Shahid, Francesco Braghin, Loris Roveda

IDSIA, SUPSI, Lugano, Switzerland · Department of Mechanical Engineering, Politecnico di Milano, Italy

Long-Horizon Robotic ManipulationRobot Skill LearningRobotics SimulationHumanoid Robot ControlRobotic RL
AskPDF
Sep 11, 2024

Highly-Efficient Differentiable Simulation for Robotics

Robotics simulators have improved significantly in computational speed and scalability, enabling them to generate years of simulated data for complex systems in minutes or hours. Despite these advances, efficiently and accurately computing simulation derivatives remains an open challenge. Addressing this would accelerate the convergence of reinforcement learning and trajectory optimization algorithms, particularly for contact-rich problems. This paper introduces a unifying framework for robotic simulation that accounts for all factors, including dynamics, collisions, and friction. The resulting algorithm computes analytical derivatives of the simulation by implicit differentiation, explicitly handling the intrinsic non-smoothness of the collision and frictional stages while exploiting the sparsity induced by the multi-body structure. Benchmark results demonstrate state-of-the-art performance, with timings ranging from 5 μ5\,μs for a 7-dof manipulator to 95 μ95\,μs for a 36-dof humanoid, an improvement of at least two orders of magnitude over alternative methods. Implemented in C++, the code will be open-sourced after the review process to support applications such as simulation-driven learning and real-time control.

Quentin Le Lidec, Louis Montaut, Yann de Mont-Marin +3

Inria, Ecole normale sup´erieure CNRS, PSL Research University 75005 Paris, France

Robotics SimulationMultibody DynamicsDifferentiable Physics
AskPDF
Aug 31, 2018

Gibson Env: Real-World Perception for Embodied Agents

Developing visual perception models for active agents and sensorimotor control are cumbersome to be done in the physical world, as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly. This has given rise to learning-in-simulation which consequently casts a question on whether the results transfer to real-world. In this paper, we are concerned with the problem of developing real-world perception for active agents, propose Gibson Virtual Environment for this purpose, and showcase sample perceptual tasks learned therein. Gibson is based on virtualizing real spaces, rather than using artificially designed ones, and currently includes over 1400 floor spaces from 572 full buildings. The main characteristics of Gibson are: I. being from the real-world and reflecting its semantic complexity, II. having an internal synthesis mechanism, "Goggles", enabling deploying the trained models in real-world without needing further domain adaptation, III. embodiment of agents and making them subject to constraints of physics and space.

Fei Xia, Amir Zamir, Zhi-Yang He +3

Stanford University · University of California, Berkeley

Embodied AIRobotics SimulationRobotic PerceptionSim-to-Real TransferEmbodied Agent Benchmarks
AskPDF