cs.CVOct 1, 2026
SaveFewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens
Organizations: Peking University · National University of Singapore · NVIDIA · Impossible Research
Abstract
Vision language model (VLM) agents can control robots through visual feedback and action primitives, but repeated model invocations and redundant observations incur substantial token overhead. We introduce PyRUA-Lean, an interactive code-execution framework that couples feedback-driven primitive composition with selective observation: the agent composes classical robot primitives and learned vision-language-action (VLA) policies into Python cells that perform conditional checks and local retries, returning only explicitly requested images and state feedback for replanning. Across 700 simulated task instances from LIBERO-PRO, RoboTwin 2.0, and RoboCasa365, we compare PyRUA-Lean with a tool-calling baseline using the same GPT-6 Astra planner and underlying robot primitives. Under equal LLM-call budgets, PyRUA-Lean increases overall success from 63.1% to 71.7%. On instances solved by both agents, it uses 49% fewer LLM calls and 65% fewer input tokens.
Figures & tables
Figure 2 : PyRUA-Lean combines primitive composition with selective feedback. (a) The tool-calling agent invokes robot primitives through tool calls. Operations that depend on preceding results generally require another VLM turn, and motion calls automatically return images and state. (b) PyRUA-Lean instead generates Python cells that compose robot primitives with helper functions, conditional checks, and local retries. Intermediate execution stays within the runtime, while explicitly requested images and state messages are recorded and returned at cell end for replanning. (c) A recorded placement example illustrates both mechanisms: one PyRUA-Lean cell replaces baseline steps 14–17, reducing four LLM turns to one. The cell computes a placement target from scene geometry (compose), conditionally executes lowering and release (compose), and requests an image only if the task remains unfinished (select). This successful execution returns five printed lines and no image, demonstrating how programmatic composition and selective feedback reduce repeated model interaction. Appendix A follows the whole episode.
| Tool (RPent) | Code (PyRUA-Lean) | |
| Primitives, the same in both | ||
| Arm motion | move_to , move_pose , move_delta , rotate_wrist , rotate_pitch | |
| Gripper | set_gripper , release , scripted_grasp | |
| VLA policy | pi0_pick , pi0_doubled , lingbot_act , rldx_skill , rldx_arm | |
| Mobile base | navigate_to , move_base | |
| Perception | segment , back_project , back_project_batch , sample_world_xyz , query_world_map | |
Table 1 : The two agents’ interfaces: each primitive is an RPent tool for tool calling and a robo method with the same name and arguments for code, both running RPent’s implementation. Names pooled over the benchmarks.
Figure 3 : Success rate if every episode were stopped once its token consumption reached the budget on the horizontal axis. Dashed lines mark the budget at which each agent first reaches tool calling’s final success rate, with its value on the axis; the large number is their ratio. RC365: RoboCasa365.
Figure 4 : Only code solved this LIBERO-PRO instance (“turn on the stove and put the moka pot on it”): both agents dropped the pot, and the code agent fitted a plane to the lid, turned the pot upright and set its base over the burner (Appendix D ).
Figure 5 : Token-savings decomposition on jointly solved instances. Tool-calling-to-PyRUA-Lean ratios quantify reductions in LLM calls (solid) and tokens per call (light); their product gives the total token-reduction factor (right). Ratios below indicate increased usage by PyRUA-Lean.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6 : The episode of Figure 2 , whole. Top: each box is one LLM call; the number is how many tools (tool calling) or primitives (code) it called, dark boxes returned camera images, and the outlined box solved the task. Bottom: token consumption so far; circles mark the solving call, reached after 816k tokens with tool calling and 156k with code. Left: the agent-view camera at the start and when the task was solved.
Figure 7 : Token consumption so far of the episodes in Table 5 , five per benchmark, LLM call by LLM call; circles mark the solving call. On RoboTwin 2.0 and RoboCasa365’s atomic tasks the median episodes of both agents are nearly identical from task to task, so their lines overlap.
Figure 12 : An instance only code solved: LIBERO-PRO, “turn on the stove and put the moka pot on it” (10 swap, task 2, seed 3). Both agents turned the stove on and dropped the pot. Tool calling asked Pi0 to grasp the fallen pot four times and ended the episode; the code agent grasped it itself, fitted a plane to the points of its lid to find how it leans, turned it upright, and set its base, not the gripper, over the burner. Top: the recorded camera views after the LLM calls named. Bottom: the code agent’s cells, excerpts exactly as the model wrote them; “ ... ” marks lines left out. The plane fit in call 23 raised an error (NumPy’s linear algebra refuses half-precision arrays); call 24 cast the points and ran it again.
Figure 13 : An instance only code solved: RoboCasa365 atomic, “Open the cabinet door.” (seed 4). Both agents located the door’s hinge. Tool calling pulled the handle a few centimeters per LLM call and was still pulling when its 40 calls ran out; the code agent took the hinge as the centre of the circle through three positions of the handle, two of which it copied from earlier outputs, and pulled along that arc in one cell, stopping if a move fell short or the grip slipped. The door opened, and a VLA call in call 34 completed the task. Top: the recorded camera views; bottom: the code agent’s cells, excerpts exactly as the model wrote them.
| Benchmark | Tasks | Seeds per task | Task instances | LLM calls per episode |
|---|---|---|---|---|
| LIBERO-PRO | 40 | 5 | 200 | 40 |
| RoboTwin 2.0 | 50 | 5 | 250 | 40 |
| RC365 atomic | 18 | 5 | 90 | 40 |
| RC365 composite | 32 | 5 | 160 | 100 |
| All | 140 | 700 |
Table 7 : The task instances of the main comparison. Each instance runs once with each agent.
| Tool , images after every move | Tool , images on demand | Code | |
|---|---|---|---|
| Success rate (%), all task instances | 83.0 | 68.5 | 94.0 |
| Failures: gave up / call budget | 9 / 25 | 17 / 46 | 5 / 7 |
| LLM calls per solved episode ∗ | 19.9 | 22.8 | 7.4 |
| Prompt tokens per solved episode ∗ | 1.07M | 1.07M | 222k |
Table 8 : Tool calling with camera images after every move (the LIBERO-PRO baseline of Table 3 ) and only on demand, against code, on all 200 task instances of LIBERO-PRO. ∗ On the 131 instances all three solved (means).