Embodied coding agents construct robot policies from natural-language instructions by generating executable programs and refining them from execution feedback. This refinement assumes that execution feedback reveals whether the task succeeded and, after a failure, provides sufficient evidence to guide program revision. Aerial manipulation violates this assumption in two ways. Payload constraints restrict onboard camera coverage and leave task-critical outcomes outside the robot's view (partial observability), while strong dynamic coupling and limited control margins make physical interactions fragile (limited interaction robustness) and leave failure causes unresolved. Re-examining the same feedback cannot recover observations that were never collected or separate failure causes that the available evidence does not distinguish. To acquire this missing evidence, we introduce Active Feedback (AF), an active "View and Probe" framework. When execution feedback leaves unresolved questions, the agent synthesizes a separate diagnostic program that runs in simulation to inspect unseen outcomes from informative viewpoints (View) or test hypotheses about failure causes through targeted physical interactions (Probe). Across six simulated aerial manipulation tasks with 60 trials per condition, AF reduces false completion declarations from 25 to 5 and raises final task success from 31.7% to 75% compared with terminal feedback under the same coding-agent configuration and task-execution limit. Using shared robot APIs, task programs refined with AF in simulation run on a physical aerial manipulator without task-program source-code modification and complete multi-stage manipulation tasks in the real world. These results show that executable diagnosis fills gaps in execution feedback for embodied coding agents, improving both completion assessment and program refinement in aerial manipulation.
Figures & tables
Figure 1: Fly-by-Code uses Active Feedback (AF) to address missing outcome observations and unresolved failure causes arising from partial observability and limited interaction robustness. Rather than relying only on feedback passively collected during task execution, Fly-by-Code generates an executable diagnostic program to acquire the evidence needed for outcome assessment and program revision. The program either navigates to an informative viewpoint ( View ) to inspect an unobserved outcome or performs a targeted physical interaction ( Probe ) to distinguish between possible failure causes. Using shared robot APIs, task programs refined in simulation run on a physical aerial manipulator without source-code modification and complete multi-stage tasks such as the illustrated four-step tool-manipulation sequence.
Figure 2: Overview of Fly-by-Code. Bottom: The program refinement loop. The coding agent translates a task prompt into an executable task program using shared robot APIs. Following execution, it enters the Active Feedback (AF) stage to acquire diagnostic evidence, iteratively refining the program until it decides to terminate the revision process. Top: During AF, the agent synthesizes a diagnostic program to resolve execution uncertainties. View actively navigates to informative viewpoints (e.g., inspecting a basket from above to verify placement). Probe conducts diagnostic physical tests (e.g., testing door articulation with an arc motion after a straight pull fails).
Figure 3: Recorded View and Probe rounds, abridged from Appendix D . Left: The task program reported both tools placed. The diagnostic program acquired views of the destinations and nearby floor, which place both tools near floor level, and the agent rejected the reported success. Right: After pushes kept rotating a carton, the agent compared contact responses across six push tests. The next task program converts the measurements into a push-response model for contact selection. Quotations, code, and image notes are recorded agent outputs, with omissions marked by […] or # [...] . Red circles were added by the authors.
Figure 4: Representative aerial manipulation tasks. Sort: Identify the target bottle and place it into the color-matched bin. Align: Push protruding boxes into alignment with the rack. Because the centers of mass are offset, varying contact points induce distinct translational or rotational motions. Cabinet: Open the cabinet doors, retrieve the bottle inside, and deliver it to a hanging basket. Payload: Transport filled bottles and baskets with minimal hall crossings under an undisclosed payload limit. The Sort and Align tasks operate in the warehouse simulation, whereas Cabinet and Payload use the laboratory digital twin. Overlaid robot poses and arrows illustrate sequential execution stages and motion trajectories.
Figure 5: Final task success across six tasks. AF achieves the highest success rate overall and matches or exceeds both CaP-X and CaP-X+Trace on every task. Each trial permits one initial execution and up to ten program revisions. Labels above the bars indicate successful trials out of ten total trials per task.
Task Outcomes (TP / FP / ND)
Overall
Method
Cabinet
1-Obj P&P
2-Obj P&P
Payload
Align
Sort
TP/FP/ND
Precision
CaP-X
1/ 1 /8
9/1/0
4/5/1
3/6/1
0/10/0
2/2/6
19/25/16
43.2%
CaP-X+Trace
0/3/7
10 / 0 /0
6/3/1
6/3/1
3/7/0
4/3/3
29/19/12
60.4%
AF (Ours)
4 / 1 /5
10 / 0 /0
9 / 1 /0
10 / 0 /0
7 / 3 /0
5 / 0 /5
45 / 5 /10
90.0%
Table 1: Completion assessment performance. Each triplet reports correct completion declarations (TP), incorrect completion declarations (FP), and trials reaching the step limit without a declaration (ND), with 10 trials per task and 60 overall. Precision is TP/(TP+FP). P&P denotes Pick-and-place.
Figure 6: Examples of Active Feedback (AF) for completion assessment and program refinement. (a) View: The robot moves above the basket to inspect the placement outcome. (b) Probe: A targeted push reveals the box’s response to physical contact. (c) Probe: A diagnostic lift test of a 1.2 kg basket load is refused, indicating that the proposed load is infeasible (left). The next task program revises the object grouping and successfully carries a 1.1 kg load (right).
Figure 7: Final task success across models and agent interfaces on Payload and Cabinet. Labels above the bars indicate successful trials / total trials, with ten trials per task and condition.
Task
Outcome
Successful trials
2-Objects Pick-and-place
Grasp hammer
5/5
Place hammer in basket
4/5
Grasp wrench
5/5
Place wrench on shelf
5/5
Full task
4/5
Cabinet
Open left door
5/5
Table 2: Physical task outcomes. Denominators include all evaluated trials per task, even those not reaching the subtask.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Execution-trace feedback. Images captured before and after a primitive call accompany its structured execution record. The observations follow the task execution rather than a dedicated verification trajectory.
Figure 9: Aerial manipulator and laboratory environment. (a) Simulated model of our custom-designed platform, with its principal components annotated. (b) The laboratory digital twin and the physical laboratory, showing corresponding structures and task objects despite differences in their arrangement.
API
Function
Observation and motion
get_observation
Retrieve wrist RGB-D images, camera parameters, and robot state.
get_ee_pose
Read the end-effector pose and gripper opening.
goto_pose
Move the grasp center to a target position with a specified approach direction and roll.
open_gripper , close_gripper
Open or close the gripper.
Image-based perception
Appendix
Table 3: Robot APIs available to generated programs.
Figure 10: Task progress across task-program executions. AF (Ours) achieves generally comparable or higher progress across individual tasks and shows a clear advantage in average progress across all six tasks. Turn 1 denotes the initial program execution, and trials that terminate early retain their final score through turn 11. Scores are normalized by the maximum score of each task-specific rubric (Appendix B.2 ).
Figure 11: Task progress across API and coding-agent configurations. Each panel compares configurations under the same feedback method. Astra API achieves higher final average progress with AF than with CaP-X on both tasks, with a larger improvement on Cabinet. Normalization and early-termination handling follow Figure 10 .
AF variant
Success
FP
Cost ($)
2-Objects Pick-and-place
Full AF
9/10
1
10.2
w/o semantic map
4/10
5
15.3
Sort
Full AF
5/10
0
29.3
w/o semantic map
2/10
3
37.6
Appendix
Table 5: Semantic map ablations within AF, with ten trials per task and condition. Success counts successful trials; FP counts false FINISH declarations. Cost is the mean estimated model cost per trial in USD.
Per trial
Per success
Task
Method
Success
Turns
Output (k)
USD
Turns
Output (k)
USD
All tasks
CaP-X
19/60
7.2
142
9.6
22.7
447
30.4
CaP-X+Trace
29/60
7.0
129
9.4
14.6
267
19.5
AF (Ours)
45/60
6.0
155
17.1
8.1
206
22.8
Cabinet
CaP-X
1/10
10.9
156
10.8
109.0
1562
107.7
CaP-X+Trace
0/10
10.9
174
13.7
–
–
–
Appendix
Table 6: Resource usage in the main evaluation. Turns denote task-program executions; output tokens include thinking tokens and are reported in thousands (k). USD denotes estimated token-based LLM cost, not actual subscription charges. Per-success values include resources spent on failed trials; – denotes no observed success. Bold indicates the highest success count and lowest resource per success within each task block.
Item
Specification
Airframe
Configuration
Omnidirectional coaxial quad-tiltrotor: four tilting arms, each carrying a coaxial pair of rotors (eight rotors, four tilt joints).
Dimensions
Airframe 0.62×0.69 m, height 0.20 m including rotors.
Mass
8.30 kg in flight configuration (battery, onboard computer, camera, and gripper included).
Table 8: Scene layout in the laboratory digital twin and the physical laboratory. Coordinates are in the shared world frame (meters). Simulation values are ground-truth poses from the scene description. Physical positions are motion-capture measurements at the closest corresponding point on each object, so the comparison is approximate rather than exact.
Task
Trial
Position (x,y,z) [m]
Heading [ ∘ ]
2-Objects Pick-and-place
1
(−0.66,0.00,1.86)
+1.8
2
(−0.90,−0.01,1.86)
−0.3
3
(−0.97,−0.04,1.86)
−0.1
4
(−0.87,−0.04,1.86)
−0.6
5
(−0.71,0.33,1.86)
+41.3
Cabinet
1
(−1.19,−0.39,1.86)
+20.4
Appendix
Table 9: Starting poses of the hardware trials. Airframe position and heading at task-program start after takeoff, in the shared world frame.
Figure 12: Real-world Cabinet execution. External views of one successful trial: opening the left door, opening the right door, grasping the bottle, and placing it in the hanging basket.
Figure 13: Cabinet in simulation and the real world. Wrist-camera views show opening the left door, opening the right door, grasping the bottle, and approaching the basket before release. The top and bottom rows show simulation and physical execution, respectively.
Trial
S1
S2
S3
S4
What happened
2-Objects Pick-and-place (S1 grasp hammer, S2 place in basket, S3 grasp wrench, S4 place on upper shelf)
1
✓
✓
✓
✓
–
2
✓
✓
✓
✓
–
3
✓
×
✓
✓
A perception error in the detected basket position shifted the release point, and the hammer struck the basket rim on release and fell outside.
4
✓
✓
✓
✓
–
5
✓
✓
✓
✓
–
Appendix
Table 10: Trial-level records of the real-world trials. ✓: completed, × : failed.
Coding agents can construct executable programs and revise them through feedback, offering a foundation for embodied control without target-task fine-tuning. Through a programmatic robot interface, the agent observes the scene, executes control programs, and adapts them to the resulting physical state, serving as the closed-loop policy. However, this within-episode adaptation does not automatically preserve experience across scenes. Our trajectory analysis shows that agents spend interaction and inference budgets establishing effective operations and identifying their preconditions, while fresh agents cannot reuse these discoveries. This motivates EmbodiedEvo, a self-evolving coding agent that distills physical feedback into reusable code Skills and diagnostic Lessons. Experience is acquired on one Playground scene per task, then frozen for independent agents in unseen scenes, without updating model weights. On RoboDojo-Sim, EmbodiedEvo achieves an average score of 44.48 and a success rate of 37.82%, surpassing all officially verified results. Paired tests show the largest gains on memory-dependent tasks and successful reuse of coordinated procedures, while fine contact control remains difficult. Experience from stronger models also improves weaker executors: Astra's experience raises Sol and Luna's success rates by 8.86 and 5.50 percentage points over no experience. Compared with raw interaction history, Skills and Lessons achieve comparable task performance at 53.0% lower execution cost.
How well can general-purpose multimodal models turn visual understanding and reasoning into embodied manipulation via executable code? We introduce CodeActionBench, a benchmark of 25 manipulation tasks that evaluates this capability through agentic Code-as-Policy. Without task-specific fine-tuning, demonstrations, external specialist perception or grasp modules, privileged scene state, or predefined task policies, agents should select visual evidence, form task-relevant 3D estimates, construct manipulation targets, and iteratively execute and revise their policies. A shared robot API provides RGB observations, calibrated geometric operations, robot feedback, and bounded motion, leaving task-dependent decisions to the evaluated agent. Fixed task instances, resource budgets, and a hidden physical-outcome verifier support controlled comparisons across models and harness configurations. Extensive evaluations across nine configurations and 675 attempts achieve success rates ranging from 2.7% to 73.3%. The strongest configuration, GPT-6 Astra with Codex CLI, solves 22 of 25 tasks at least once in three attempts, demonstrating the best performance while still leaving substantial room for improvement. Trajectory analyses reveal difficulties in spatial alignment, object retention, and completion judgment, including task failures despite successfully completed motions. CodeActionBench provides a controlled testbed for measuring how general-purpose models translate their capabilities into manipulation behavior and for examining typical failure scenarios in that process.
Coding agents can now write, run, and debug programs with little human help. Robot tasks, however, are usually specified by a sentence that leaves out how to grasp, in what order to make contact, and what the result should look like, and an agent given only the sentence must find these details by trial and error. We introduce ENCORE, which gives the agent a few demonstrations as evidence to read rather than as training data. A deterministic builder distills each demonstration into a pack of multi-view keyframes, gripper events, frame strips, and the full trajectory. A coding agent studies the pack, writes a policy program against a fixed perception and action API, refines it iteratively over a few development rollouts, and freezes it before a sealed evaluation that never reveals the success signal. On LIBERO-PRO, the agent's first program already succeeds in half of the perturbed tasks with demonstrations and in one task without them, and the frozen programs outperform the strongest prior agentic system run with the same language model (96.3% against 89.3%). On RoboDojo tasks whose instructions leave the goal unstated, no program succeeds without demonstrations. ENCORE also runs on a real bimanual robot, learning cube handover and cup inversion from five demonstrations each.
Yifan Kang, Zihan Wang, Zhiwen Fan +1
Massachusetts Institute of Technology · AI Open Source Foundation · Texas A&M University +1