cs.ROOct 8, 2026
SaveFly-by-Code: Embodied Coding Agents for Aerial Manipulation with Active Visual and Physical Feedback
Organizations: Seoul National University · Carnegie Mellon University · University of Maryland, College Park · All Purpose AI · Kyung Hee University
Abstract
Embodied coding agents construct robot policies from natural-language instructions by generating executable programs and refining them from execution feedback. This refinement assumes that execution feedback reveals whether the task succeeded and, after a failure, provides sufficient evidence to guide program revision. Aerial manipulation violates this assumption in two ways. Payload constraints restrict onboard camera coverage and leave task-critical outcomes outside the robot's view (partial observability), while strong dynamic coupling and limited control margins make physical interactions fragile (limited interaction robustness) and leave failure causes unresolved. Re-examining the same feedback cannot recover observations that were never collected or separate failure causes that the available evidence does not distinguish. To acquire this missing evidence, we introduce Active Feedback (AF), an active "View and Probe" framework. When execution feedback leaves unresolved questions, the agent synthesizes a separate diagnostic program that runs in simulation to inspect unseen outcomes from informative viewpoints (View) or test hypotheses about failure causes through targeted physical interactions (Probe). Across six simulated aerial manipulation tasks with 60 trials per condition, AF reduces false completion declarations from 25 to 5 and raises final task success from 31.7% to 75% compared with terminal feedback under the same coding-agent configuration and task-execution limit. Using shared robot APIs, task programs refined with AF in simulation run on a physical aerial manipulator without task-program source-code modification and complete multi-stage manipulation tasks in the real world. These results show that executable diagnosis fills gaps in execution feedback for embodied coding agents, improving both completion assessment and program refinement in aerial manipulation.
Figures & tables
Figure 1: Fly-by-Code uses Active Feedback (AF) to address missing outcome observations and unresolved failure causes arising from partial observability and limited interaction robustness. Rather than relying only on feedback passively collected during task execution, Fly-by-Code generates an executable diagnostic program to acquire the evidence needed for outcome assessment and program revision. The program either navigates to an informative viewpoint ( View ) to inspect an unobserved outcome or performs a targeted physical interaction ( Probe ) to distinguish between possible failure causes. Using shared robot APIs, task programs refined in simulation run on a physical aerial manipulator without source-code modification and complete multi-stage tasks such as the illustrated four-step tool-manipulation sequence.
Figure 2: Overview of Fly-by-Code. Bottom: The program refinement loop. The coding agent translates a task prompt into an executable task program using shared robot APIs. Following execution, it enters the Active Feedback (AF) stage to acquire diagnostic evidence, iteratively refining the program until it decides to terminate the revision process. Top: During AF, the agent synthesizes a diagnostic program to resolve execution uncertainties. View actively navigates to informative viewpoints (e.g., inspecting a basket from above to verify placement). Probe conducts diagnostic physical tests (e.g., testing door articulation with an arc motion after a straight pull fails).
Figure 3: Recorded View and Probe rounds, abridged from Appendix D . Left: The task program reported both tools placed. The diagnostic program acquired views of the destinations and nearby floor, which place both tools near floor level, and the agent rejected the reported success. Right: After pushes kept rotating a carton, the agent compared contact responses across six push tests. The next task program converts the measurements into a push-response model for contact selection. Quotations, code, and image notes are recorded agent outputs, with omissions marked by […] or # [...] . Red circles were added by the authors.
Figure 4: Representative aerial manipulation tasks. Sort: Identify the target bottle and place it into the color-matched bin. Align: Push protruding boxes into alignment with the rack. Because the centers of mass are offset, varying contact points induce distinct translational or rotational motions. Cabinet: Open the cabinet doors, retrieve the bottle inside, and deliver it to a hanging basket. Payload: Transport filled bottles and baskets with minimal hall crossings under an undisclosed payload limit. The Sort and Align tasks operate in the warehouse simulation, whereas Cabinet and Payload use the laboratory digital twin. Overlaid robot poses and arrows illustrate sequential execution stages and motion trajectories.
Figure 5: Final task success across six tasks. AF achieves the highest success rate overall and matches or exceeds both CaP-X and CaP-X+Trace on every task. Each trial permits one initial execution and up to ten program revisions. Labels above the bars indicate successful trials out of ten total trials per task.
| Task Outcomes (TP / FP / ND) | Overall | |||||||
| Method | Cabinet | 1-Obj P&P | 2-Obj P&P | Payload | Align | Sort | TP/FP/ND | Precision |
| CaP-X | 1/ 1 /8 | 9/1/0 | 4/5/1 | 3/6/1 | 0/10/0 | 2/2/6 | 19/25/16 | 43.2% |
| CaP-X+Trace | 0/3/7 | 10 / 0 /0 | 6/3/1 | 6/3/1 | 3/7/0 | 4/3/3 | 29/19/12 | 60.4% |
| AF (Ours) | 4 / 1 /5 | 10 / 0 /0 | 9 / 1 /0 | 10 / 0 /0 | 7 / 3 /0 | 5 / 0 /5 | 45 / 5 /10 | 90.0% |
Table 1: Completion assessment performance. Each triplet reports correct completion declarations (TP), incorrect completion declarations (FP), and trials reaching the step limit without a declaration (ND), with 10 trials per task and 60 overall. Precision is TP/(TP+FP). P&P denotes Pick-and-place.
Figure 6: Examples of Active Feedback (AF) for completion assessment and program refinement. (a) View: The robot moves above the basket to inspect the placement outcome. (b) Probe: A targeted push reveals the box’s response to physical contact. (c) Probe: A diagnostic lift test of a 1.2 kg basket load is refused, indicating that the proposed load is infeasible (left). The next task program revises the object grouping and successfully carries a 1.1 kg load (right).
Figure 7: Final task success across models and agent interfaces on Payload and Cabinet. Labels above the bars indicate successful trials / total trials, with ten trials per task and condition.
| Task | Outcome | Successful trials |
|---|---|---|
| 2-Objects Pick-and-place | Grasp hammer | 5/5 |
| Place hammer in basket | 4/5 | |
| Grasp wrench | 5/5 | |
| Place wrench on shelf | 5/5 | |
| Full task | 4/5 | |
| Cabinet | Open left door | 5/5 |
Table 2: Physical task outcomes. Denominators include all evaluated trials per task, even those not reaching the subtask.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Execution-trace feedback. Images captured before and after a primitive call accompany its structured execution record. The observations follow the task execution rather than a dedicated verification trajectory.
Figure 9: Aerial manipulator and laboratory environment. (a) Simulated model of our custom-designed platform, with its principal components annotated. (b) The laboratory digital twin and the physical laboratory, showing corresponding structures and task objects despite differences in their arrangement.
| API | Function |
|---|---|
| Observation and motion | |
| get_observation | Retrieve wrist RGB-D images, camera parameters, and robot state. |
| get_ee_pose | Read the end-effector pose and gripper opening. |
| goto_pose | Move the grasp center to a target position with a specified approach direction and roll. |
| open_gripper , close_gripper | Open or close the gripper. |
| Image-based perception | |
Table 3: Robot APIs available to generated programs.
Figure 10: Task progress across task-program executions. AF (Ours) achieves generally comparable or higher progress across individual tasks and shows a clear advantage in average progress across all six tasks. Turn 1 denotes the initial program execution, and trials that terminate early retain their final score through turn 11. Scores are normalized by the maximum score of each task-specific rubric (Appendix B.2 ).
Figure 11: Task progress across API and coding-agent configurations. Each panel compares configurations under the same feedback method. Astra API achieves higher final average progress with AF than with CaP-X on both tasks, with a larger improvement on Cabinet. Normalization and early-termination handling follow Figure 10 .
| AF variant | Success | FP | Cost ($) |
| 2-Objects Pick-and-place | |||
| Full AF | 9/10 | 1 | 10.2 |
| w/o semantic map | 4/10 | 5 | 15.3 |
| Sort | |||
| Full AF | 5/10 | 0 | 29.3 |
| w/o semantic map | 2/10 | 3 | 37.6 |
Table 5: Semantic map ablations within AF, with ten trials per task and condition. Success counts successful trials; FP counts false FINISH declarations. Cost is the mean estimated model cost per trial in USD.
| Per trial | Per success | |||||||
|---|---|---|---|---|---|---|---|---|
| Task | Method | Success | Turns | Output (k) | USD | Turns | Output (k) | USD |
| All tasks | CaP-X | 19/60 | 7.2 | 142 | 9.6 | 22.7 | 447 | 30.4 |
| CaP-X+Trace | 29/60 | 7.0 | 129 | 9.4 | 14.6 | 267 | 19.5 | |
| AF (Ours) | 45/60 | 6.0 | 155 | 17.1 | 8.1 | 206 | 22.8 | |
| Cabinet | CaP-X | 1/10 | 10.9 | 156 | 10.8 | 109.0 | 1562 | 107.7 |
| CaP-X+Trace | 0/10 | 10.9 | 174 | 13.7 | – | – | – | |
Table 6: Resource usage in the main evaluation. Turns denote task-program executions; output tokens include thinking tokens and are reported in thousands (k). USD denotes estimated token-based LLM cost, not actual subscription charges. Per-success values include resources spent on failed trials; – denotes no observed success. Bold indicates the highest success count and lowest resource per success within each task block.
| Item | Specification |
|---|---|
| Airframe | |
| Configuration | Omnidirectional coaxial quad-tiltrotor: four tilting arms, each carrying a coaxial pair of rotors (eight rotors, four tilt joints). |
| Dimensions | Airframe m, height m including rotors. |
| Mass | kg in flight configuration (battery, onboard computer, camera, and gripper included). |
| Motors | T-Motor VELOX V2812, 1155 KV (6S). |
| Propellers | T-Motor C7.5 4.6 three-blade cinematic propellers (7.5 in). |
Table 7: Physical platform specifications.
| Item | Simulation | Physical |
| Target objects | ||
| Hammer | ||
| Wrench | ||
| Water bottle | ||
| Task fixtures | ||
| Pegboard front face ( ) | ||
Table 8: Scene layout in the laboratory digital twin and the physical laboratory. Coordinates are in the shared world frame (meters). Simulation values are ground-truth poses from the scene description. Physical positions are motion-capture measurements at the closest corresponding point on each object, so the comparison is approximate rather than exact.
| Task | Trial | Position [m] | Heading [ ∘ ] |
|---|---|---|---|
| 2-Objects Pick-and-place | 1 | ||
| 2 | |||
| 3 | |||
| 4 | |||
| 5 | |||
| Cabinet | 1 |
Table 9: Starting poses of the hardware trials. Airframe position and heading at task-program start after takeoff, in the shared world frame.
Figure 12: Real-world Cabinet execution. External views of one successful trial: opening the left door, opening the right door, grasping the bottle, and placing it in the hanging basket.
Figure 13: Cabinet in simulation and the real world. Wrist-camera views show opening the left door, opening the right door, grasping the bottle, and approaching the basket before release. The top and bottom rows show simulation and physical execution, respectively.
| Trial | S1 | S2 | S3 | S4 | What happened |
| 2-Objects Pick-and-place (S1 grasp hammer, S2 place in basket, S3 grasp wrench, S4 place on upper shelf) | |||||
| 1 | ✓ | ✓ | ✓ | ✓ | – |
| 2 | ✓ | ✓ | ✓ | ✓ | – |
| 3 | ✓ | ✓ | ✓ | A perception error in the detected basket position shifted the release point, and the hammer struck the basket rim on release and fell outside. | |
| 4 | ✓ | ✓ | ✓ | ✓ | – |
| 5 | ✓ | ✓ | ✓ | ✓ | – |
Table 10: Trial-level records of the real-world trials. ✓: completed, : failed.