Self-improving embodied agents are increasingly being developed and compared on robot simulation benchmarks. Several recent works expose the benchmark's success criteria to the learning agent, a practice we call Criteria Exposure (CE). These criteria, originally written to verify policies trained by imitation learning, often demand more than the corresponding language instructions, so a score from such a verifier may not faithfully reflect whether the instructed task was completed, especially for robot-control programs written by an agent. As embodied coding agents report ever higher benchmark scores, a natural question arises: when a coding agent is exposed to the benchmark's criteria, does it generate code that better performs the instructed task, or code that simply overfits to the verification criteria? We conduct controlled experiments with a self-improving agent with and without CE, and find that with CE the agent quickly produces code with a higher success rate under the given criteria, but this advantage disappears when the criteria are changed slightly without departing from the original instructions. More importantly, the skills the agent develops under CE transfer poorly across tasks. We further propose instruction-faithful criteria and show that they better reflect the performance of agent-generated programs.
Figures & tables
Figure 1: Learning the task or the verifier? (a) : in press_by_number, the criteria Corig verify a button protocol that the instruction does not state (highlighted). Without Criteria Exposure (w/o CE) the learner sees only whether each episode succeeded; with it (w/ CE) it also reads the criteria code, score and stages. Right of each setting: the test successes of its library on 20 layouts, judged by Corig and by the instruction-faithful criteria Cfaith . (b) : libraries learned under Corig on the 16 criteria-limited tasks, 320 test episodes each, with every episode judged by both criteria ( section 4.2.1 ). The lead of w/ CE under Corig , 51.6% against 27.8%, is gone under Cfaith (45.0% against 47.2%).
Figure 2: Overview of the self-improvement loop. Left, learning: the actor (GPT-6 Astra, high reasoning effort) writes and executes Python cells that call the robot’s primitives on five development layouts, and sees the result of each cell. Between batches the learner (Claude Code with Opus 5.5, high reasoning effort) turns the evidence into one revision of the library Lk , whose skills are callable code that reports what it did and whose memory entries say when to apply what, with the evidence for it; a revision is usually tried first in verification rollouts, and the loop rolls back to the best revision so far when a batch falls clearly below it. Whether the learner also sees the success criteria is the one switch between the two settings, w/ CE and w/o CE (the switch under the learner). Right, evaluation: the final library LK is frozen and read-only, and the same actor runs alone on layouts the loop never saw.
Tasks
Empty library L∅
w/o CE
w/ CE
All (25)
24.6
47.8 (+23.2)
62.0 (+37.4)
Criteria-limited (16)
8.4
27.8 (+19.4)
51.6 (+43.2)
Faithful (9)
53.3
83.3 (+30.0)
80.6 (+27.3)
Table 1: Test success under the original criteria (SR, %, layouts 0–19 of seed 0). Parentheses: difference from the empty library in points. The criteria-limited tasks count the episodes of section 4.2.1 , judged by Corig .
Figure 3: Development success by round , on the 16 criteria-limited and the 9 faithful tasks. Dashed: w/ CE; solid: w/o CE; round 0 is the empty library. On criteria-limited tasks both settings start at 6% and part in round 1 (40% against 28%); after that the success-only learner barely moves and ends at 31%, against 55% with exposure. On faithful tasks the success-only learner is ahead through round 4 and both end above 90% (91% and 95%); its lead, at most 16 points, is close to the 14-point gap of round 0 (66% against 52%), where both run the same empty library.
Figure 5
Target
Library source
L∅
w/o CE
w/ CE
RoboDojo, 10 target tasks × 20 layouts
RoboDojo
65/200
80/200 (+15)
71/200 (+6)
LIBERO-PRO, 6 tasks × 10 seeds
RoboDojo
45/60
48/60 (+3)
46/60 (+1)
RoboCasa365, 10 target tasks × 10 seeds
RoboDojo
22/100
24/100 (+2)
24/100 (+2)
RoboCasa365, 10 target tasks × 10 seeds
RoboCasa365
22/100
33/100 (+11)
29/100 (+7)
Table 2: Reusing libraries on new tasks and benchmarks (successes, each benchmark’s own criteria). The RoboDojo libraries are distilled from ten learned tasks and carried over to LIBERO-PRO and RoboCasa365; the last row uses libraries distilled from other RoboCasa365 tasks. Every row is paired with the empty library on the same layouts; parentheses give the change in successes against it.
Table 5: Tasks by type. ⚫: high severity; ❍: medium; blank: low or none. The last column marks the 33 tasks whose criteria require both arms to return home (not counted as a type). Total: tasks at medium or high severity (high in parentheses). r : reviewed together with its _random variant.
Coding agents can construct executable programs and revise them through feedback, offering a foundation for embodied control without target-task fine-tuning. Through a programmatic robot interface, the agent observes the scene, executes control programs, and adapts them to the resulting physical state, serving as the closed-loop policy. However, this within-episode adaptation does not automatically preserve experience across scenes. Our trajectory analysis shows that agents spend interaction and inference budgets establishing effective operations and identifying their preconditions, while fresh agents cannot reuse these discoveries. This motivates EmbodiedEvo, a self-evolving coding agent that distills physical feedback into reusable code Skills and diagnostic Lessons. Experience is acquired on one Playground scene per task, then frozen for independent agents in unseen scenes, without updating model weights. On RoboDojo-Sim, EmbodiedEvo achieves an average score of 44.48 and a success rate of 37.82%, surpassing all officially verified results. Paired tests show the largest gains on memory-dependent tasks and successful reuse of coordinated procedures, while fine contact control remains difficult. Experience from stronger models also improves weaker executors: Astra's experience raises Sol and Luna's success rates by 8.86 and 5.50 percentage points over no experience. Compared with raw interaction history, Skills and Lessons achieve comparable task performance at 53.0% lower execution cost.
Coding agents are beginning to move beyond purely digital tasks to tackle physical-world challenges, particularly in robotics. Existing robotics benchmarks, however, primarily focus on the performance of individual artifacts, such as policies or controllers, offering limited coverage of coding agents' broader engineering capabilities. Real-world robotics extends beyond control: agents must build, integrate, diagnose, and improve heterogeneous artifacts under resource constraints and reason from multimodal feedback. To evaluate these broader capabilities, we introduce RLE-Bench, a benchmark of robot-learning tasks spanning four representative robotics development workflows: interactive control, policy learning, perception and estimation, and mechanical design. We use diverse task-specific metrics to evaluate the artifacts submitted by the coding agents, from the success rate the agents achieved to the policy agents trained, the harness agent built, and the mechanical structures the agent designed. We aggregate these metrics into an overall RLE Index and report workflow-specific capability profiles, enabling systematic comparison of coding agents' capabilities across multiple capability dimensions. Beyond performance ranks, we also conduct in-depth case studies examining agent behavior on representative tasks, highlighting both current capabilities and limitations, and pointing to the opportunities robotics tasks have to offer for future agent training.
Haitong Ma, Chenxiao Gao, Rushi Qiang +2
Harvard University · Georgia Institute of Technology
Execution feedback can guide coding agents toward correct repository repairs, but only when the tests capture the behavior requested by the issue. Agent-generated tests can encode incomplete or incorrect behavioral targets; when the same trajectory writes both the patch and the test, their errors can agree and create false confidence. We introduce ExecCritic, combining a test--verify--revise scaffold with a role-specific reinforcement learning recipe for training agents within it. The scaffold separates test construction from source-code repair: a Test agent independently generates repository-native tests, a fail-closed harness qualifies and freezes them, and a Repair agent revises source code from their execution feedback without changing the tests. Both roles use Qwen-3.5-35B-A3B as the backbone and are trained separately. In Learn to Test, the Test agent learns to produce behaviorally valid tests that distinguish correct from incorrect patches. In Test to Improve, the Repair agent learns both direct task resolution and feedback-guided revision. On SWE-bench Verified, test quality determines whether feedback helps: holding the base Repair agent fixed, tests from the base Test agent reduce resolved rate from a no-test baseline of 61.2% to 57.3%, whereas tests from GPT-5.6-sol raise it to 65.3%. Role-specific post-training raises the Qwen Test agent's Base-to-Gold success from 22.2% to 62.2%; composing the two post-trained Qwen agents reaches 72.6%, an 11.4-point gain over the original no-test baseline without stronger-model or Oracle feedback at evaluation time. Code is publicly available at https://github.com/MSR-Orchard/execcritic.
Leitian Tao, Baolin Peng, Haorui Wang +7
University of Wisconsin–Madison · Microsoft Research · Georgia Tech