cs.ROOct 22, 2026
SaveFastHarness: Experience-Driven Review Scheduling for Vision-Language-Action Policies
Organizations: School of AI, Shanghai Jiao Tong University · Zhejiang University
Abstract
Agentic robot systems combine language-model reasoning with robot policies to monitor execution and recover from failures. However, reviewing every action chunk incurs substantial overhead, even in familiar situations. We introduce FastHarness, an experience-driven execution harness that adaptively schedules GPT-6 Astra supervision of a π0.5 policy. A fast path uses stage memory and reviewed outcomes to reduce review frequency as experience accumulates, while a complementary recovery path focuses supervision on correcting errors. Once recovery is verified, routine execution resumes, with failure experience retained to guide future interventions. The policy remains frozen and generates fresh actions from current observations. Across four simulated manipulation tasks, reviewer token use decreases by an average of 57.2% per task from the first evaluated episode to the mean of the final five episodes. Mean episode time decreases from 85.5 to 52.0 minutes over the same interval. On the same tasks, mean success increases from 80% for GPT-as-Policy (π0.5 + GPT-6 Astra) to 97.19% with FastHarness. These results highlight adaptive supervision as a practical route to more efficient agentic robotics without additional model training.
Figures & tables
Figure 1: From VLA execution to experience-driven supervision. VLA-only execution generates actions from current observations. GPT-as-Policy adds GPT-6 Astra review at every action chunk. Using the same policy and reviewer, FastHarness skips redundant reviews as task experience accumulates and concentrates supervision on error recovery. Dots denote GPT reviews; the check mark indicates verified recovery, after which adaptive review resumes.
Figure 2: FastHarness adds two paths around the original GPT-as-Policy execution backbone (gray, middle). The experience-driven fast path (top) routes fresh VLA proposals through a memory-guided scheduler, bypassing GPT-6 review when eligible; the gray dashed link denotes the original per-chunk review route. The recovery path (bottom) uses reviewed local repair and, when needed, verified waypoint goals. Ordinary scheduling resumes after verified resolution. Dashed teal feedback retains reviewed evidence across episodes. Image inserts illustrate recorded task scenes.
Figure 3: GPT-6 calls across repeated task executions. Each panel uses a task-specific call-count scale and spans all evaluated episodes: 16 for Build tower, 20 each for Fold clothes and Bottle disposal, and nine for Classify objects. Episode indices follow the evaluation order within each shared-memory chain.
Figure 4: Execution efficiency for the first evaluated episode and the final-five-episode mean. Avg. denotes the equal-task mean. (a) Recorded input and output tokens, with cached input counted once. The mean per-task reduction is 57.2%, and the equal-task mean falls from 1.83 to 0.93 million tokens. (b) Episode time in minutes; the equal-task mean falls from 85.5 to 52.0 minutes (39% lower).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| 1 | Match the current observation; obtain and validate a fresh policy proposal. |
|---|---|
| 2 | Compute the interval and control-step allowance; set the review flag from the ordered triggers. |
| 3 | while the review flag is set do |
| 4 | Review the observation, proposal, preceding transition, history, memory, and recovery context. |
| 5 | On the first review only: attribute outcome and intent error; store any eligible checkpoint. |
| 6 | Update the stage example, reset review counters, and update recovery. |
| 7 | Retain outcome/intent uncertainty; set the review flag if recovery requests re-review. |
Algorithm 1 Review scheduling and recovery before each action chunk in FastHarness.
| Component | Configuration |
|---|---|
| Environment | RoboDojo native runtime on Isaac Sim; two X5 arms; CPU physics. |
| Control and observations | Control step s (25 Hz); three RGB views (high, left wrist, right wrist), joint state and end-effector state; no depth input. |
| Policy | , OpenPI/JAX; configuration pi05_base_aloha_full_sim_arx-x5_seed_0 , checkpoint directory 59999 ; fixed weights. |
| Policy randomness | JAX seed 0; a fresh policy server for each episode. |
| Action interface | 14 dimensions: six joints and one gripper opening per arm; internal arm actions are increments relative to the current joint state; gripper 0 is closed and 1 is open. The policy predicts 50 actions; an executed proposal contains 1–15 control steps. |
| Reviewer | gpt-6-astra , reasoning effort xhigh ; structured review output via the Responses protocol. |
Table 2: Environment, policy, and harness configuration used for evaluation.
| Task | Native step limit | Evaluation and analysis |
|---|---|---|
| Build tower | 1,050 | 16 episodes: efficiency and completion |
| Fold clothes | 500 | 20 episodes: review allocation, automatic spans, time and tokens |
| Bottle disposal | 700 | 20 episodes: review allocation, automatic spans, time and tokens |
| Classify objects | 1,100 | 9 episodes: efficiency and completion; episodes 1–8: 16 recovery events |
Table 3: Task coverage and native control-step limits. Task success counts are reported in Table 6 .
| Observed event outcome | Count |
|---|---|
| Resumed normal scheduling | 16 |
| Closed without normal resumption | 0 |
| Still active at observation end | 0 |
| Total recovery events | 16 |
Table 4: Recovery outcomes in Classify objects episodes 1–8, comprising 16 events across eight episodes. Resuming normal scheduling is an intermediate recovery outcome, distinct from native task success. Events within an episode and shared memory chain are not independent trials.
| Task | First episode (M) | Final five (M) | Reduction |
|---|---|---|---|
| Build tower | 1.47 | 0.47 | 68% |
| Fold clothes | 0.72 | 0.25 | 65% |
| Bottle disposal | 1.64 | 0.58 | 64% |
| Classify objects | 3.50 | 2.42 | 31% |
Table 5: Recorded GPT-6 token usage (millions) in the first evaluated episode and the mean of the final five episodes. Counts include input and output tokens, with cached input counted once. Build tower’s first evaluated episode follows earlier task experience. Values are rounded; the unweighted mean of the four per-task reductions is 57.2%.
| Task | Episodes | Successes | Rate (%) |
|---|---|---|---|
| Build tower | 16 | 15 | 93.75 |
| Fold clothes | 20 | 19 | 95 |
| Bottle disposal | 20 | 20 | 100 |
| Classify objects | 9 | 9 | 100 |
Table 6: FastHarness completion counts and success rates, with an unweighted task mean of 97.19%. The evaluation protocol is specified in Appendix B .
| External method | Build tower | Fold clothes | Bottle disposal | Classify objects |
|---|---|---|---|---|
| 24 | 21.07 | 69.30 | 12.67 | |
| +GPT-6 | 60 | 100 | 100 | 60 |
| GPT-6 only | 0 | 40 | 20 | 100 |
Table 7: Success rates (%) from the GPT-as-Policy report ( Su et al., 2026 ) for the same four tasks.
Figure 5: Empirical cumulative distributions of automatic-span lengths in episodes 1–5 and 16–20. A rightward shift indicates longer execution spans between GPT-6 reviews.
Figure 6: Running mean recorded tokens per episode on common axes. Each point averages all episodes through the indicated index, including initial experience acquisition.