A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

Yu Lei; Minghuan Liu; Abhiram Maddukuri; Zhenyu Jiang; Yuke Zhu

cs.ROApr 15, 2026

A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

Authors: Yu Lei, Minghuan Liu, Abhiram Maddukuri, Zhenyu Jiang, Yuke Zhu

Organizations: Department of Computer Science, The University of Texas at Austin · Amazon FAR · NVIDIA

Abstract

Co-training, which combines limited in-domain real-world data with abundant surrogate data such as simulation or cross-embodiment robot data, is widely used for training generative robot policies. Despite its empirical success, the mechanisms that determine when and why co-training is effective remain poorly understood. We investigate the mechanism of sim-and-real co-training through theoretical analysis and empirical study, and identify two intrinsic effects governing performance. The first, structured representation alignment"**, reflects a balance between cross-domain representation alignment and domain discernibility, and plays a primary role in downstream performance. The second, the **importance reweighting effect", arises from domain-dependent modulation of action weighting and operates at a secondary level. We validate these effects with controlled experiments on a toy model and extensive sim-and-sim and sim-and-real robot manipulation experiments. Our analysis offers a unified interpretation of recent co-training techniques and motivates a simple method that consistently improves upon prior approaches. More broadly, our aim is to examine the inner workings of co-training and to facilitate research in this direction.

Figures & tables

Figure 2: Importance reweighting reshapes the learned action distribution via reweighting score functions during training time.

Figure 3: Visualization of controlled toy example. We co-train $\sim$ 30 and $\sim$ 3000 samples from target and source domain. Vertically in each column, the differences of prediction samples showcase the impact of representation alignment; horizontally in each row, the differences showcase the shift of importance reweighting effect controlled by mixing ratio $w$ .

Figure 4: L2 loss with sweeping mixing ratio and delta

z

distance, and ANOVA variance decomposition. When representations overlap (red line) or are disjoint (blue line), co-trained models are insensitive to mixing ratios. The importance reweighting effect (blue bar) can only explain

20\%

of the performance variance.

Figure 5: Visualizations of the designed sim-and-sim and sim-and-real tasks. In the physics-only setting, we vary the physical parameters of objects, including mass, friction, and size, while keeping the appearance similar. Across all target environments, we tune the robot controller configurations to match those in the real world.

Figure 6: UMAP visualization of latent features. We visualize the features after the vision stem and after the encoder trunk $f_{\phi}$ . Red and blue dots represent the real and simulation features, respectively. We show the results for a specific mixing ratio in this figure. More details and results are provided in the Appendix D.2 .

Figure 7: Correlation between representation alignment and success rate. We group the results of different tasks in each setting and compute the Spearman and Pearson correlation coefficients after normalization. Blue, yellow, and green-series dots represent the same task in Fig. 8 . Gray dash line indicates the fitted curve.

Figure 8: Success rate of sim-and-sim co-training with different domain gaps. Each data point is computed over three policy checkpoints, and each checkpoint is evaluated for 200 trials. The best performance is consistently achieved in the range of

(0.016,0.3)

	NutAssembly	MugCleanup	MugHang	Avg
co-training	0.85	0.44	0.27	0.52
+discrimination	0.79	0.415	0.215	0.47

Table 1: Performance degrades after adding penalty for representation alignment. The experiments are run with

w=0.1

Figure 9: Average performance on sim-and-sim experiments. We can find that alignment-based methods (+OT/+ADDA) get improvement in the balanced mixing ratio group, while discernibility-based methods (+CFG) get improvement mainly in the unbalanced mixing ratio group. More detailed results are available in the Appendix. D.1

Method	Success Rate									Avg
Method	NutAssembly			MugCleanup			MugHang			Avg
Real-only	11/30			8/30			6/30			8.6/30
Mixing Ratio $w$	0.016	0.1	0.3	0.016	0.1	0.3	0.016	0.1	0.3	-
Co-Training	17/30	11/30	16/30	16/30	9/30	7/30	8/30	13/30	7/30	15.3/30
+ OT	15/30	17/30	11/30	8/30	15/30	15/30	11/30	9/30	4/30	14.3/30
+ ADDA	13/30	13/30	15/30	6/30	14/30	11/30	10/30	14/30	7/30	14.3/30

Table 2: Real-world policy performance under different co-training strategies.

Figure 10: Performance of different co-training methods on MugCleanup with

w=0.1

in sim-and-sim settings. The plots are similar for all tasks and balanced mixing ratios.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: N/M scaling

Figure 12: Empirical measurement about training sample overlapping using Bhattacharyya coefficient. We compute it in the same way as in Song et al. (2025) , but include the distances between observation features as we are modeling the conditional distribution.

Panel of Figure 13 — Figure 13: Detailed results of sim-and-sim evaluations.

Figure 14: Balanced mixing ratios are robust to different policy architectures. Each data point is computed over three policy checkpoints, and each checkpoint is evaluated for 200 trials. The best performance is consistently achieved in the range of

(0.016,0.3)

Task	Pretrain+FT	Co-training
NutAssembly	0.77	0.925
MugCleanup	0.215	0.495
MugHang	0.225	0.26

Appendix

Table 3: Performance comparison with Pre-train+FT.

Figure 15: Experiments on different dataset sizes. Each data point is computed over three policy checkpoints, and each checkpoint is evaluated for 200 trials. The best performance is consistently achieved in the range of

(w_{n},w_{q})

, which supports our guideline below.

w_{n}=\frac{N}{N+M}.

Appendix

Algorithm 2 Guideline for Co-Training Mixing Ratio Selection

Panel of Figure 16 — Figure 16: Existence of three regimes in co-training.

Panel of Figure 17 — Figure 17: More UMAP visualizations of the observation representations.

Panel of Figure 18 — Figure 18: More UMAP visualizations of the deeper layer representations.

Robot Policy Learning

Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.

See all 372 papers

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training
Oct 14, 2026Liqian Ma, Vaibhav Saxena, Utkarsh Aashu Mishra +3Synthetic Data Generation for Robotics Sim-to-Real Transfer
Georgia Institute of Technology
BEACON: Cross-Domain Co-Training of Generative Robot Policies via Best-Effort Adaptation
May 9, 2026Antong Zhang, Han Qi, Heng YangSynthetic-to-Real Domain Adaptation Robot Policy Learning
Department of Computer Science, Brown University · School of Engineering and Applied Sciences, Harvard University
TUCO: Curating Simulation Demonstrations for Sim-to-Real Robot Policy Co-Training
Oct 4, 2026Ning Zhu, Mengfei Zhao, Yikai Tang +5Robot Policy Learning Sim-to-Real Transfer
Stanford University · Nanyang Technological University · AXIS Robotics +4

Oct 14, 2026J/K move · Enter open · S save

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training

Liqian Ma, Vaibhav Saxena, Utkarsh Aashu Mishra, Shuo Cheng+2

Georgia Institute of Technology

Synthetic Data Generation for Robotics Sim-to-Real Transfer Meta-Learning Robotic Manipulation

Open paper AskPDF arXiv

Sim-and-real co-training is an effective way to scale robot learning by augmenting limited real-world demonstrations with simulation data. However, existing approaches typically treat simulators as fixed data sources, relying on uniform generation, offline retrieval, or manual parameter alignment by domain experts. We formulate Active Generative Simulation: optimizing a simulator's data-generation distribution to maximize downstream real-world performance. To solve this problem, we propose MetaGenSim, a meta-learning framework that trains an auxiliary rater network via meta-gradients to automatically discover high-utility data-generation distributions within a user-specified simulation parameter space, without manual parameter alignment. Across simulation and real-world manipulation tasks, MetaGenSim outperforms uniform generation and retrieval baselines, achieves a 150% relative improvement in real-world full-task success over uniform generation, and matches expert manual alignment. We further show that distributions curated with a lightweight proxy policy transfer to VLA fine-tuning, and that MetaGenSim can exploit partial alignment through atomic skill stitching and object-receptacle composition when exact sim-and-real task alignment is unavailable. Project website: https://meta-gensim.github.io

Abstract

Robot Policy Learning

Explore similar work

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training

BEACON: Cross-Domain Co-Training of Generative Robot Policies via Best-Effort Adaptation

TUCO: Curating Simulation Demonstrations for Sim-to-Real Robot Policy Co-Training