Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training

Liqian Ma; Vaibhav Saxena; Utkarsh Aashu Mishra; Shuo Cheng; Yongxin Chen; Danfei Xu

cs.ROOct 14, 2026

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training

Authors: Liqian Ma, Vaibhav Saxena, Utkarsh Aashu Mishra, Shuo Cheng, Yongxin Chen, Danfei Xu

Organizations: Georgia Institute of Technology

Abstract

Sim-and-real co-training is an effective way to scale robot learning by augmenting limited real-world demonstrations with simulation data. However, existing approaches typically treat simulators as fixed data sources, relying on uniform generation, offline retrieval, or manual parameter alignment by domain experts. We formulate Active Generative Simulation: optimizing a simulator's data-generation distribution to maximize downstream real-world performance. To solve this problem, we propose MetaGenSim, a meta-learning framework that trains an auxiliary rater network via meta-gradients to automatically discover high-utility data-generation distributions within a user-specified simulation parameter space, without manual parameter alignment. Across simulation and real-world manipulation tasks, MetaGenSim outperforms uniform generation and retrieval baselines, achieves a 150% relative improvement in real-world full-task success over uniform generation, and matches expert manual alignment. We further show that distributions curated with a lightweight proxy policy transfer to VLA fine-tuning, and that MetaGenSim can exploit partial alignment through atomic skill stitching and object-receptacle composition when exact sim-and-real task alignment is unavailable. Project website: https://meta-gensim.github.io

Figures & tables

Figure 2: MetaGenSim . We propose a meta-learning framework for Active Generative Simulation with three stages: (a) Rater network learning: We meta-learn a rater network that weights simulation samples according to their contribution to policy performance on a held-out validation set of real demonstrations. (b) Parameter utility landscape: We map these scores back to their corresponding simulation parameters and use kernel density estimation (KDE) to construct a probability landscape that identifies high-utility regions. (c) Targeted synthesis and co-training: We sample from the constructed distribution to generate a targeted simulation corpus, then co-train the final policy on this synthetic data and few-shot real demonstrations to maximize real-world success.

Figure 3: Sim-and-Sim Co-training performance using BC-RNN. Success bars are mean over 3 seeds, showing Standard Error of the Mean (SEM). Diffusion Policy exhibits a similar trend; due to space constraints, we include those results in Appendix G .

Method	stackCube		mugBin		bowlTray		mugHang		Total
	Pick	Pick+Place	Pick	Pick+Place	Pick	Pick+Place	Pick	Pick+Place	Pick	Pick+Place
RealOnly	9 /20	4 /20	4 /20	4 /20	16 /20	13 /20	6 /20	5 /20	35 /80	26 /80
UniformGen	7 /20	2 /20	4 /20	3 /20	17 /20	15 /20	5 /20	3 /20	33 /80	23 /80
STRAP	6 /20	5 /20	6 /20	5 /20	13 /20	12 /20	11 /20	5 /20	36 /80	27 /80
IWR	10 /20	4 /20	5 /20	4 /20	19 /20	17 /20	14 /20	8 /20	48 /80	33 /80
MetaGenSim (ours)	17 /20	14 /20	14 /20	14 /20	19 /20	19 /20	17 /20	11 /20	67 /80	58 /80

Table 1: Sim-and-Real Co-training results for Diffusion Policy on 4 real-world tasks. Success counts are for Pick and Pick+Place stages. The best performing method (excluding Oracle ) is highlighted in bold.

Figure 4: Rater score distribution of the probing and the real dataset ( green for high, red for low). The probing dataset is decomposed into four tasks with representative scored samples. The rater assigns high utility scores to the relevant stackCube task while suppressing scores for misaligned tasks. The vertical lines represent mean scores.

Figure 5: MetaGenSim for real-world tasks. (Top-left) We meta-learn the rater network using 10 real demonstrations and an 800-demo simulated probing dataset spanning 4 tasks. (Top-right) The trained rater assigns utility scores to probing data. We visualize their 12-D task configurations (table color, camera yaw, object spatial arrangements, and one-hot vector indicating tasks) using t-SNE, colored by utility scores ( green : high; red : low). High-scoring trajectories match the target task in visual and spatial arrangements, while low-scoring ones are misaligned. (Bottom) We sample from the induced parameter distribution to generate a large-scale, target-aware simulation dataset.

Method	stackCube	mugBin	bowlTray	mugHang	Total
RealOnly	1 /20	6 /20	12 /20	0 /20	19 /80
UniformGen	2 /20	12 /20	13 /20	3 /20	30 /80
MetaGenSim	11 /20	18 /20	18 /20	5 /20	52 /80
Oracle	13 /20	19 /20	17 /20	7 /20	56 /80

Table 2: Full-task success counts for fine-tuning the pretrained

\boldsymbol{\pi_{0.5}}

VLA.

Figure 6: Generalization to compositionally novel target tasks. (Left) The real-world task mugTray lacks a direct simulation counterpart. MetaGenSim constructs a co-training dataset by identifying and prioritizing aligned sub-skills, specifically mugBin for grasping the mug and bowlTray for placing an object on the tray. (Right) Real-robot success counts.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: MetaGenSim for Real-World Tasks and Parameter Space Probing. (Top) The four real-world target tasks considered in our main experiments. (Middle) We use the trained rater network to compute utility scores for each trajectory in the probing dataset. We show t-SNE plots where the 12-D task configurations for each trajectory (3-D table color, 1-D camera yaw, 2-D spatial arrangements for two objects, and a 4-D one-hot vector representing task) are fit to 2-D, and colored as per their utility score ( green for high, red for low). The top-scored trajectories can be observed to match the visual and spatial arrangements of the real task, while the lowest-rated trajectories differ significantly in either aspect. (Bottom) Samples from the final MetaGenSim dataset, generated by sampling from the induced data generation parameter distribution.

Figure 8: Visualization of initial frames from the probing dataset for the sim-and-real co-training experiments. The dataset provides broad coverage across all tasks and exhibits a diverse range of camera poses, table textures, and object placements, ensuring comprehensive exploration of the parameter space.

Figure 9: Target tasks for sim-and-sim co-training experiments.

batch size	64
sequence length	10
optimizer	Adam
learning rate	1e-4
image encoder	ResNet18 w/ FiLM
image resolution	$(84,84)$

Appendix

Table 3: Policy hyperparameters for BC-RNN and Diffusion Policy.

Target Task		1600 demos	8000 demos
stackCube	Pick	7 /20	7 /20
stackCube	Pick+Place	2 /20	3 /20
mugBin	Pick	4 /20	2 /20
mugBin	Pick+Place	3 /20	1 /20

Appendix

Table 4: Scaling up UniformGen . We generate 8000 uniform demos (

5\times

more than UniformGen used in the main experiments) for stackCube and mugBin , and report the resulting co-training performance. Increasing the amount of uniformly generated data can degrade co-training performance.

Panel of Figure 10 — Figure 10: Rater score distributions for additional real-world target tasks. The aligned simulation task generally receives higher scores and a heavier upper tail than misaligned tasks.

Figure 11: MetaGenSim -generated data for different real-world target-task variants. Top row shows the standard setting used in our main experiments, including image samples and the distributions of camera yaw and object placements; the dashed red line denotes the ground-truth camera yaw obtained via calibration. The second row shows generated data for stackCube with a blue table, where MetaGenSim predominantly selects blue table textures. The third row considers stackCube under a different camera pose, and the camera-yaw distribution shifts accordingly. The fourth row shows stackCube with up-and-down object placements, where the generated placement distribution correspondingly shifts to up-and-down configurations.

Figure 12: Sim-and-Sim Co-training performance using Diffusion Policy. Success bars are mean over 3 seeds, showing Standard Error of the Mean (SEM). The overall trend is similar to the BC-RNN results in Figure 3 : MetaGenSim consistently improves over UniformGen and remains close to Oracle .

Figure 13: MetaGenSim for dynamic tasks and physics parameters. Given target demonstrations for a specific rope or cloth configuration, MetaGenSim identifies the aligned density and size parameters.

Task	TargetOnly	MetaGenSim
mugBin -Pick	5 /20	14 /20
mugBin -Pick+Place	5 /20	14 /20
stackCube -Pick	7 /20	17 /20
stackCube -Pick+Place	3 /20	17 /20

Appendix

Table 5: Multi-task real-world co-training results. MetaGenSim is trained to target mugBin and stackCube simultaneously and substantially improves over target-only training on both tasks.

Figure 14: MetaGenSim with 3D camera poses for the stackCube task. Beyond the 1D yaw parameter used in the main real-world experiments, MetaGenSim can optimize over yaw, pitch, and distance.

Figure 15: Sensitivity analysis for sim/real data budgets and the co-training ratio $\alpha$ . Performance is best around $\alpha=0.9$ ; increasing the simulation budget helps only when sufficient real demonstrations are available.

Target Task		LowBW	HighBW	HighTemp
stackCube	Pick	12 /20	17 /20	2 /20
stackCube	Pick + Place	7 /20	14 /20	0 /20

Appendix

Table 6: Ablations on hyperparameters of the data generation process ( 4.2 ) for the stackCube task. LowBW denotes a bandwidth of

\frac{1}{4}

, HighBW denotes a bandwidth of

2\times

, and HighTemp denotes a temperature of

4\times

. Moderate temperature and sufficient bandwidth yield the best co-training performance, while overly high temperature hurts performance.

Synthetic Data Generation for Robotics

Synthetic data generation for robotics creates simulated scenes, sensor readings, and action trajectories to train and evaluate robots.

See all 303 papers

A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies
Apr 15, 2026Yu Lei, Minghuan Liu, Abhiram Maddukuri +2Robot Policy Learning Synthetic-to-Real Domain Adaptation
Department of Computer Science, The University of Texas at Austin · Amazon FAR · NVIDIA
HyperSim: A Holistic Sim-To-Real Framework For Robust Robotic Manipulation
May 26, 2026Junyi Dong, Haotian Luo, Ziwei Xu +11Synthetic-to-Real Domain Adaptation Sim-to-Real Transfer
CloudRobo Lab, Huawei Cloud Computing Technologies Co.,Ltd. · Shanghai Jiao Tong University · The University of Hong Kong
ARSTAG: An Agentic Real2Sim2Real System for Task-Specific Robot Data Generation
Sep 21, 2026Bowei Li, Yuner Zhang, Changliu LiuRobotics Simulation Sim-to-Real Transfer
Carnegie Mellon University, Pittsburgh, PA 15213, USA

Apr 15, 2026J/K move · Enter open · S save

A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

Yu Lei, Minghuan Liu, Abhiram Maddukuri, Zhenyu Jiang+1

Department of Computer Science, The University of Texas at Austin · Amazon FAR · NVIDIA

Robot Policy Learning Synthetic-to-Real Domain Adaptation Cross-Domain Transfer Learning Importance Weighting

Open paper AskPDF arXiv

Co-training, which combines limited in-domain real-world data with abundant surrogate data such as simulation or cross-embodiment robot data, is widely used for training generative robot policies. Despite its empirical success, the mechanisms that determine when and why co-training is effective remain poorly understood. We investigate the mechanism of sim-and-real co-training through theoretical analysis and empirical study, and identify two intrinsic effects governing performance. The first, structured representation alignment"**, reflects a balance between cross-domain representation alignment and domain discernibility, and plays a primary role in downstream performance. The second, the **importance reweighting effect", arises from domain-dependent modulation of action weighting and operates at a secondary level. We validate these effects with controlled experiments on a toy model and extensive sim-and-sim and sim-and-real robot manipulation experiments. Our analysis offers a unified interpretation of recent co-training techniques and motivates a simple method that consistently improves upon prior approaches. More broadly, our aim is to examine the inner workings of co-training and to facilitate research in this direction.

Abstract

Synthetic Data Generation for Robotics

Explore similar work

A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

HyperSim: A Holistic Sim-To-Real Framework For Robust Robotic Manipulation

ARSTAG: An Agentic Real2Sim2Real System for Task-Specific Robot Data Generation