CoDA: Co-Evolution of Data Engines and Action Policies

Tianjiao Yu; Zhilin Ou; Yifan Shen; Xinzhuo Li; Yuanzhe Liu; Onkar Kishor Susladkar; Fan Wang; Ismini Lourentzou

cs.ROOct 15, 2026

CoDA: Co-Evolution of Data Engines and Action Policies

Authors: Tianjiao Yu, Zhilin Ou, Yifan Shen, Xinzhuo Li, Yuanzhe Liu, Onkar Kishor Susladkar, Fan Wang, Ismini Lourentzou

Organizations: University of Illinois Urbana-Champaign · The Chinese University of Hong Kong, Shenzhen · Amazon

Abstract

Generative simulation can automate robot environment construction, but physically valid environments are not necessarily useful for training. The challenge is to identify environments that address a policy's failures and supervision that helps it overcome them. We introduce CoDA, a framework in which a simulation data engine and an action policy co-evolve through a shared heterogeneous memory of environment construction, policy failures, and learning progress. The memory compiles validated diagnoses from a vision-language critic into generation programs that recreate difficulties and training programs that help the policy overcome them. We further propose Text2potential to convert these diagnoses into staged potentials for action-chunk value learning, providing dense supervision for policy updates while preserving the task's optimal policy under standard shaping assumptions. Measured learning progress guides further generation and program refinement. Across four manipulation task families, CoDA improves out-of-distribution success from 5.3% after supervised fine-tuning to 27.6%, exceeding the strongest closed-loop baseline by 9.5 percentage points with the same generation budget. On LIBERO-Pro, 15 rounds of generation and training raise success from 25% to 84% under position perturbations and from 1% to 76% under instruction perturbations.

Figures & tables

Figure 1: The 0.64314 0.68627 0.84706C0.34902 0.7451 0.88235o \__color_backend_reset: \__color_backend_reset: 0.44706 0.78039 0.59608D0.5451 0.81176 0.3098A \__color_backend_reset: \__color_backend_reset: framework. The data engine $\mathcal{G}$ generates executable scenes with independently editable task, spatial, and object-attribute factors $(\tau,\sigma,\alpha)$ and a sparse success verifier $\nu$ . Construction traces and rollout-grounded failure diagnoses enter a shared memory, which organizes them into reusable difficulty modes. Validated modes are compiled into generation programs that reproduce the diagnosed conditions and training programs that specify subgoals, potentials, and learning tools. Text2potential supplies dense feedback through chunk-level value targets that guide policy updates, and measured learning progress guides the next generation round. — Figure 1: The 0.64314 0.68627 0.84706C0.34902 0.7451 0.88235o __color_backend_reset: __color_backend_reset: 0.44706 0.78039 0.59608D0.5451 0.81176 0.3098A __color_backend_reset: __color_backend_reset: framework. The data engine $\mathcal{G}$ generates executable scenes with independently editable task, spatial, and object-attribute factors $(\tau,\sigma,\alpha)$ and a sparse success verifier $\nu$ . Construction traces and rollout-grounded failure diagnoses enter a shared memory, which organizes them into reusable difficulty modes. Validated modes are compiled into generation programs that reproduce the diagnosed conditions and training programs that specify subgoals, potentials, and learning tools. Text2potential supplies dense feedback through chunk-level value targets that guide policy updates, and measured learning progress guides the next generation round.

	OOD by family				Average
Data generation	Pick-place	Articulated	Put-inside	Stack/insert	In	OOD
pretrained, zero-shot	3.1 $\pm$ 0.6	1.6 $\pm$ 0.5	2.7 $\pm$ 0.6	1.8 $\pm$ 0.6	3.4 $\pm$ 0.2	2.3 $\pm$ 0.3
+ seed demonstrations (SFT)	6.4 $\pm$ 0.6	4.7 $\pm$ 0.4	5.9 $\pm$ 0.5	4.2 $\pm$ 0.4	11.7 $\pm$ 0.4	5.3 $\pm$ 0.3
domain randomization	8.8 $\pm$ 0.4	8.3 $\pm$ 0.6	7.6 $\pm$ 0.4	6.9 $\pm$ 0.5	17.2 $\pm$ 0.3	7.9 $\pm$ 0.4
uniform variants [ 29 ]	10.6 $\pm$ 0.4	6.9 $\pm$ 0.6	9.7 $\pm$ 0.4	6.8 $\pm$ 0.3	21.6 $\pm$ 0.3	8.5 $\pm$ 0.2
scalar-feedback co-evolution [ 10 ]	20.9 $\pm$ 0.7	15.3 $\pm$ 0.7	19.6 $\pm$ 1.2	16.6 $\pm$ 0.7	33.8 $\pm$ 0.4	18.1 $\pm$ 0.5

Table 1: Policy generalization across four manipulation task families. OOD success rates (%) under different generation and training strategies. In denotes success on each generator’s own distribution. Values are mean

\pm

standard error over three seeds. Best and second best highlighted.

Panel of Figure 2 — Figure 2: Co-evolution over rounds. (a) Average OOD success as number of accepted environments per task family increases. Dashed line marks the comparison reported in Table 1 . Curves show mean over three seeds, with standard error shaded for CoDA. (b) Number of difficulty modes newly selected for training, retained for further training, and removed from the memory in each round. Labels above bars separately report the number of generation programs reused from other task families. (c) OOD success gain (percentage points) per round by CoDA and generation guided by task success rates.

		Object		Goal		Spatial		Overall
Method	StdLBR	Pos	Task	Pos	Task	Pos	Task	Pos	Task
OpenVLA [ 12 ]	0.765	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
$\pi_{0}$ [ 1 ]	0.942	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
$\pi_{0.5}$ [ 8 ]	0.971	0.17	0.01	0.38	0.00	0.20	0.01	0.25	0.01
Human-written programs	—	0.14	0.00	0.00	0.00	0.23	0.05	0.12	0.02
ASPIRE (code-as-policy) [ 18 ]	—	0.98	0.95	0.81	0.45	0.51	0.60	0.77	0.67

Table 2: LIBERO-Pro generalization to position and instruction perturbations. Success rates under position (Pos) and instruction (Task) perturbations for the Object, Goal, and Spatial suites. StdLBR reports success on standard LIBERO. Best and second best highlighted.

Critic	OOD success (%) $\uparrow$	Refuted (%) $\downarrow$	Human agr. (%) $\uparrow$
Qwen3-VL-235B (open weights)	24.9 $\pm$ 0.8	27.5	73.8
InternVL3-78B (open weights)	25.7 $\pm$ 0.7	24.1	76.3
Claude Opus 5	26.1 $\pm$ 0.6	20.3	81.5
GPT-5.5 (default)	27.6 $\pm$ 0.5	18.2	82.4

Table 3: Critic ablations. OOD success at a fixed training budget, diagnoses refuted by control evaluations, and agreement with two human annotators. Best and second best results are highlighted.

Training signal	OOD success	Steps to 25% OOD (k)	Agreement
sparse verifier only ( $\Phi\equiv 0$ )	22.3 $\pm$ 0.8	31	—
potential from task description [ 42 ]	24.6 $\pm$ 0.7	29	95.3
free-form dense reward, not a potential	23.5 $\pm$ 1.0	33	72.4
frozen actor: $Q_{\phi}$ -weighted selection only [ 7 ]	21.4 $\pm$ 0.9	—	96.8
Text2potential without refinement	26.1 $\pm$ 0.6	27	95.9
Text2potential (ours)	27.6 $\pm$ 0.5	21	97.2

Table 4: Text2potential ablations. We report OOD success (%) at a fixed training budget, policy updates needed to reach 25% OOD success (in thousands), and agreement (%) with the sparse-only critic’s action-chunk selections after convergence. Best and second best highlighted.

Panel of Figure 3 — Figure 3: Successful rollouts. Left: Pre-defined LIBERO-Pro tasks. Right: Evolved scenes, one row per variant. Columns are the initial state, grasp, transport, place, and success.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Memory configuration	OOD success	Rounds to target
No memory	19.2 $\pm$ 0.7	14
$\Omega$ only (engine side)	22.8 $\pm$ 0.7	9
$\Phi$ only (policy side)	22.1 $\pm$ 0.8	11
Single memory: raw trajectory retrieval	20.1 $\pm$ 0.9	13
Single memory: distilled insights	21.6 $\pm$ 0.7	12
Single memory: skill library [ 34 ]	21.0 $\pm$ 0.8	12

Appendix

Table 5: Effect of memory design on policy learning. OOD success (%), mean

\pm

standard error over three seeds at the same fixed generation budget, and # rounds required to reach 25% OOD success.

\Omega

and

\Phi

denote scene generation and policy training programs, respectively. The full method retains both program types. Best and second highlighted.

Learned on $\downarrow$ / lifted to $\rightarrow$	Pick-place	Articulated	Put-inside	Stack/insert
Pick-place	72 / 100	38 / 46	61 / 79	55 / 68
Articulated	41 / 52	69 / 100	44 / 55	37 / 48
Put-inside	63 / 81	40 / 50	70 / 100	52 / 64
Stack/insert	57 / 70	36 / 45	54 / 66	66 / 100
Re-learned from scratch on target	31 / 38	28 / 35	30 / 37	27 / 33

Appendix

Table 6: Transfer of generation programs across task families. Rows indicate the family in which programs were learned, and columns indicate the family in which they are evaluated. Each entry reports two percentages: variants within the success band / learning progress relative to programs learned within the target family. Diagonal entries provide the within-family reference, with learning progress normalized to 100%. The final row reports programs learned from scratch in the target family using the same LLM token budget as transfer.

Loop run around $\downarrow$ / policy trained $\rightarrow$	$\pi_{0.5}$	OpenVLA-OFT
$\pi_{0.5}$	27.6	21.1
OpenVLA-OFT	25.8	22.7
no memory	5.3	3.8

Appendix

Table 7: Transfer of shared memory across policy backbones. OOD success (%), with rows identifying the backbone used to collect memory and columns identifying the backbone trained and evaluated. Off-diagonal entries use frozen memory from the other backbone, with no additional environment generation. Diagonal entries report the full CoDA framework run with the corresponding backbone. The final row gives each backbone’s SFT baseline without memory. Best and second best results in each column are highlighted.

	Verification (%) $\uparrow$	Coverage (bits) $\uparrow$	Targeting (%) $\uparrow$	Refuted (%)	Human agr. (%) $\uparrow$
uniform variants	71.3	3.42	24.6	—	—
CoDA, no controls	88.2	3.21	58.4	—	73.6
0.64314 0.68627 0.84706C0.34902 0.7451 0.88235o __color_backend_reset: __color_backend_reset: 0.44706 0.78039 0.59608D0.5451 0.81176 0.3098A __color_backend_reset: __color_backend_reset: (ours)	88.9	3.18	66.7	18.2	82.4

Appendix

Table 8: Generation quality and validation of failure diagnoses. Verification (%) measures the percentage of scene proposals passing physical validity checks. Coverage is the entropy over task, spatial, and object-attribute factors

(\tau,\sigma,\alpha)

. Targeting measures the percentage of generated variants within the success band. Refuted reports the percentage of failure diagnoses rejected after comparing policy performance on targeted and control scene variants. Best and second best highlighted.

	LLM tokens (M) $\downarrow$	Sim-hours $\downarrow$	GPU-hours $\downarrow$	OOD success $\uparrow$
uniform variants	3.1	14.2	38	8.5
scalar-feedback	4.6	22.5	51	18.1
0.64314 0.68627 0.84706C0.34902 0.7451 0.88235o __color_backend_reset: __color_backend_reset: 0.44706 0.78039 0.59608D0.5451 0.81176 0.3098A __color_backend_reset: __color_backend_reset: , fixed schedule	8.3	27.8	74	26.1
0.64314 0.68627 0.84706C0.34902 0.7451 0.88235o __color_backend_reset: __color_backend_reset: 0.44706 0.78039 0.59608D0.5451 0.81176 0.3098A __color_backend_reset: __color_backend_reset: , adaptive schedule	7.4	25.1	66	27.6

Appendix

Table 9: Computational cost and effect of round scheduling. LLM token usage, simulation time, and GPU time are reported per 1,000 accepted environments. Token counts include scene generation, failure diagnosis, and potential compilation. The fixed schedule starts a new generation round every

K_{\pi}

policy-update steps, while the adaptive schedule uses the remaining difficulty modes and recent learning progress. Best and second best highlighted.

Panel of Figure 4 — Figure 4: Learning dynamics and memory accumulation. (a) In-distribution success on pick-and-place tasks during policy training, using the same generated data with potential shaping or sparse rewards alone. Shaping accelerates learning, with both conditions reaching similar final success rates. (b) Agreement between the action chunks selected by the two value critics, reaching approximately 97% at the end of training. (c) OOD success at successive memory snapshots, indexed by the number of validated difficulty modes. Using both program stores yields higher final success than retaining generation or training programs alone.

Panel of Figure 5 — Figure 5: Scene variations generated by CoDA. Four examples show changes in object layout, size, and identity for tool sorting, serving-board arrangement, craft-station organization, and noodle preparation tasks. Each panel contains a seed scene, three spatial perturbations of increasing magnitude, a target-size change, and a target-object replacement. All displayed variants pass validity checks.

Panel of Figure 6 — Figure 6: Successful policy rollouts on benchmark and generated environments. Left: five LIBERO-Pro tasks involving object placement into a basket or cabinet. Right: a fruit-preparation seed scene and five generated variants, covering three spatial perturbations, target resizing, and target replacement. Each row shows one rollout, with columns depicting the initial scene, grasp, transport, placement, and successful task completion.

Robot Policy Learning

Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.

See all 372 papers

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training
Oct 14, 2026Liqian Ma, Vaibhav Saxena, Utkarsh Aashu Mishra +3Synthetic Data Generation for Robotics Sim-to-Real Transfer
Georgia Institute of Technology
Scene2Demo: Self-Evolving Embodied Data Generation via Object-Action Graph
Feb 12, 2026Xiang Liu, Sen Cui, Guocai Yao +4Robot Demonstration Generation Synthetic Data Generation for Robotics
Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Oct 1, 2026Awomo-PhysicalRSI Team, Danjiao Ma, Enhui Ma +21Robot Demonstration Generation Synthetic Data Generation for Robotics
Awomo-PhysicalRSI Team

Oct 14, 2026J/K move · Enter open · S save

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training

Liqian Ma, Vaibhav Saxena, Utkarsh Aashu Mishra, Shuo Cheng+2

Georgia Institute of Technology

Synthetic Data Generation for Robotics Sim-to-Real Transfer Meta-Learning Robotic Manipulation

Open paper AskPDF arXiv

Sim-and-real co-training is an effective way to scale robot learning by augmenting limited real-world demonstrations with simulation data. However, existing approaches typically treat simulators as fixed data sources, relying on uniform generation, offline retrieval, or manual parameter alignment by domain experts. We formulate Active Generative Simulation: optimizing a simulator's data-generation distribution to maximize downstream real-world performance. To solve this problem, we propose MetaGenSim, a meta-learning framework that trains an auxiliary rater network via meta-gradients to automatically discover high-utility data-generation distributions within a user-specified simulation parameter space, without manual parameter alignment. Across simulation and real-world manipulation tasks, MetaGenSim outperforms uniform generation and retrieval baselines, achieves a 150% relative improvement in real-world full-task success over uniform generation, and matches expert manual alignment. We further show that distributions curated with a lightweight proxy policy transfer to VLA fine-tuning, and that MetaGenSim can exploit partial alignment through atomic skill stitching and object-receptacle composition when exact sim-and-real task alignment is unavailable. Project website: https://meta-gensim.github.io

Abstract

Robot Policy Learning

Explore similar work

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training

Scene2Demo: Self-Evolving Embodied Data Generation via Object-Action Graph

Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation