Recognizing task completion and failure enables robots to stop upon success and initiate recovery or replanning after failure. Existing methods augment Vision Language Action (VLA) policies with task status signal prediction to support these execution decisions. However, learning and integrating these signals can require fine-grained supervision, external monitoring models, or complex architectural designs. Moreover, the potential of task status signals to directly guide action generation remains underexplored. We present Echo, a framework that combines task status signal learning without frame-level progress annotations with a compact design for status prediction and action generation within a single VLA. Specifically, Echo represents task status with stop and failure signals, added as two dimensions of each action chunk and denoised together with robot actions. Their combination distinguishes ongoing execution, successful completion, and failure. To learn these signals without fine-grained progress annotations, Echo constructs successful, failed, and recovery trajectories with complementary supervision properties. To enable status signal-guided action generation, Echo controls interactions between the signals and actions and denoises the signals faster than the actions. These allow later action denoising steps to condition on cleaner status estimates while preserving accurate signal prediction. Extensive real-world manipulation experiments show that Echo accurately predicts task status and consistently improves policy performance.
Figures & tables
Figure 1: A low stop signal (SS, orange) value indicates ongoing execution, where a high failure signal (FS, purple) value marks recoverable failure and guides correction. High SS with low or high FS values indicates success (top) or terminal failure (bottom) respectively.
Figure 2: Overview of Echo. Four data types provide complementary action and signal supervision, with failed actions excluded from imitation. Stop (SS) and failure (FS) signals are appended to action chunks, with padding-based target construction illustrated on the left. The attention mask couples actions with both signals while blocking direct SS–FS interactions. Asynchronous denoising provides cleaner signals to guide subsequent action updates.
Sampled action chunks: No padding: [a1,…,aH] With padding: [\mathbf{a}_{1},\ldots,\mathbf{a}_{n},{\color[rgb]{0.0898,0.2891,0.4922}\text{pad},\ldots,\text{pad}}]
Table 1: Action and task-status supervision for four data types. H is the chunk horizon, n counts non-padding positions, and nt is defined in Eq. 8 . 0k and 1k denote k zeros and ones. Blue text marks padding positions. “All”, “Non-pad”, and “–” denote supervision over the full chunk, non-padding positions only, and no supervision, respectively.
Figure 3: Intermediate clean-sample estimates under synchronous denoising. (a) Right-arm vertical action estimates; darker curves indicate later denoising steps. (b) Stop (SS) and failure (FS) signal estimates, plotted on separate scales. Both signals converge earlier than actions, with SS already near its zero target at the earliest denoising step. This motivates resolving signals earlier to guide subsequent action updates.
Figure 4: Real-world experimental setups for five manipulation tasks. Arrows illustrate the intended manipulations.
Table 6Table 7Figure 8
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Objective
Evaluation positions
Insert Flower
Grasp a flower and insert it into a vase.
3 random ID and 3 OOD tabletop positions; 5 trials each (30 total).
Capsule Toy
Rotate the machine’s knob until a capsule is dispensed.
–
Insert Pen
Grasp a pen and insert it into a pen holder.
6 random tabletop positions; no explicit ID/OOD split (30 trials).
Make Tanghulu
Insert a bamboo skewer into a hawthorn fruit.
–
Open Drawer
Pull the drawer open.
1 ID and 1 OOD position; 15 trials each (30 total).
Appendix
Table 6: Task objectives and evaluation positions. All tasks share the initial robot state and the five-minute episode limit. ID and OOD denote in-domain and out-of-distribution positions, respectively.
Task
Deduction rule
Maximum deduction
Insert Flower
1 point per contact with the vase wall.
3 points
Capsule Toy
1 point per empty grasp attempt.
5 points
Insert Pen
1 point per contact with the pen holder.
3 points
Make Tanghulu
5 points if the hawthorn falls off the skewer after release.
5 points
Open Drawer
1 point per empty grasp attempt.
5 points
Appendix
Table 7: Task-specific scoring rules. Successful episodes receive 10 points minus the applicable deductions. Failed episodes receive 0 points, except for Make Tanghulu episodes in which the hawthorn falls off after insertion and release; these receive 5 points as partial credit.
Hyperparameter
Description
Value
H
Action chunk length
60
λA
Action loss weight
1
λS
Stop signal loss weight (non-padding chunks)
0.1
λF
Failure signal loss weight (non-padding chunks)
1
τclean
Signal clean-endpoint time (SS and FS)
0.5
δs
Stop signal threshold
0.5
Appendix
Table 8: Training, model and deployment hyperparameters.
Figure 7: Signal values estimated by CycleVLA. The model exhibits weak discriminative capability for real-world failures.
Figure 8: Signal values estimated by Robometer. The model sometimes lacks discriminative capability for real-world failures.
Figure 9: Hyperparameter sensitivity across five tasks. Top: frame-level Macro-F1 under independently varied SS and FS thresholds. Bottom: trajectory-level F1 under a shared SS/FS threshold and varying outcome-window lengths. Values are percentages. Capsule and Skewer denote Capsule Toy and Make Tanghulu, respectively. The main results use δs=δf=0.5 and W=15 frames.
(a) Overall performance
Task
Frame F1
Traj. F1
Frame Acc.
Insert Flower
92.18±0.71
96.67±3.33
96.00
Capsule Toy
88.31±0.57
100.00±0.00
94.58
Insert Pen
89.33±1.45
96.67±3.33
92.18
Make Tanghulu
91.11±1.13
93.33±4.63
95.67
Open Drawer
88.01±1.22
90.00±5.57
96.51
Appendix
Table 9: Detailed task-status prediction results for Echo on five tasks. All metrics are percentages. Frame F1 denotes frame-level Macro-F1 and Traj. F1 denotes trajectory-level F1; both are reported as mean ± SE over 30 episodes per task. Trajectory recall includes the number of correctly recognized outcomes over the number of ground-truth outcomes in parentheses. A dash indicates that no ground-truth failure examples are present.
Evaluation seed
Trials/model
Task success
Signal accuracy
π0.5
Echo
Echo
7
1,500
32.87
34.47
89.76
17
1,500
31.07
34.67
90.55
27
1,500
34.07
35.13
89.58
Average
4,500 total
32.67
34.76
89.97
Appendix
Table 10: Simulation results on the fixed LIBERO-Recovery scene set. Success rates and frame-level signal accuracy are percentages. Each seed has 1,500 trials per model. Average uses equal seed weights for task success and evaluated-frame counts as weights for signal accuracy. Echo uses τclean=0.7 .
Figure 10: Task status signals in simulation. Each column shows three observations, cropped to the task region, above the stop signal (SS, orange) and failure signal (FS, purple), averaged over all H=50 chunk predictions. Left: successful tomato-sauce placement; the green line marks first success at 8.05 s, followed by the shaded 3-second scoring window. Right: bowl placement reaching the 26-second timeout, with SS remaining below threshold. Both traces cover all 520 actions, including after success. Predictions are held between queries without smoothing or clipping; final images are recorded after action 520.
Vision-Language-Action (VLA) policies are typically adapted using successful demonstrations, which provide direct action supervision but rarely cover failure-prone states. Deployment failures expose these states, yet lack the corrective actions needed for conventional supervised learning. We propose FailPatch, a failure-driven residual patching framework that decouples action supervision from execution-reliability supervision. Successful demonstrations ground how the policy should act, while deployment trajectories indicate when its behavior becomes unreliable. We further observe that action hidden representations exhibit clear linear separability between reliable and failure-associated states while directly conditioning action generation. Building on these insights, FailPatch introduces a Null-gated Residual Expert Bank into the action hidden space of a frozen VLA policy. A unified Preserve--Redirect--Trust objective retains the original policy in reliable states, selects residual experts in failure-associated states and redirects representations from failure regions toward success-associated regions under bounded intervention. With only 0.52% trainable parameters, FailPatch improves success rates by 11.0 percentage points on four long-horizon RoboTwin tasks under clean evaluation, 9.5 percentage points under clean-to-random generalization, and 16.7 percentage points over the baseline across three real-world tasks. Project and code: https://github.com/yupeng-2003/FailPatch.
Peng Yu, Jiacheng Wang, Ziheng Zhang +6
Xi’an Jiaotong University · Dexmal · Nanjing University
Vision-Language-Action (VLA) models remain brittle in long-horizon, contact-rich manipulation because success-only imitation provides little supervision for execution drift, while failed rollouts are often discarded. We introduce RePO-VLA, a recovery-driven policy optimization framework that assigns distinct roles to success, recovery, and failure trajectories. RePO-VLA first applies Recovery-Aware Initialization (RAI), slicing recovery segments and resetting history so corrective actions depend on the current adverse state rather than the preceding failure. It then learns a Progress-Aware Semantic Value Function (PAS-VF), aligning spatiotemporal trajectory features with instructions and successful references. The resulting labels salvage useful failure prefixes via reliability decay, while low-value labels mark drift and terminal breakdowns, teaching differences among nominal, failed, and corrective actions. The data engine turns adverse states into planner-generated or human-collected corrective rollouts, teaching recovery to the success manifold. Value-Conditioned Refinement (VCR) trains the policy to prefer high-progress actions. At deployment, a fixed high value (v=1.0) biases actions toward the learned success manifold without online failure detectors or heuristic retries. We introduce FRBench, with standardized error injection and recovery-focused evaluation. Across simulated and real-world bimanual tasks, RePO-VLA improves robustness, raising adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials.
Weijia Liufu, Xiaoyu Guo, Ruiyi Chen +16
1Sun Yat-sen University · 2South China University of Technology · 3Peng Cheng Laboratory +3
Vision-Language-Action (VLA) models serve as unified policies for robotic manipulation, yet their expensive inference forces robots to pause between policy calls, resulting in stop-and-go execution that interrupts smooth motion and prolongs task completion. Extending the action chunk reduces policy calls and hence these pauses, but predicting farther into the future makes long-chunk execution unreliable. To understand where this unreliability arises, we analyze action errors within long chunks and find that they concentrate around transitions between manipulation subskills, growing sharply with chunk length. This suggests the importance of transition timing, i.e., when to switch subskills within a chunk. Motivated by this observation, we introduce RACE (Reliable Action-Chunk Extension), a framework that predicts the transition timing from an auxiliary one-step denoising pass and conditions action generation on it. By learning and conditioning on transition timing, RACE reduces errors at subskill transitions and enables reliable execution of longer action chunks. Across simulation benchmarks, RACE outperforms fine-tuning at the same chunk length; with 2x longer chunks, it surpasses recent state-of-the-art and efficient VLAs in success rate, and with 4x longer chunks, it remains competitive. On a real robot, RACE uses 4x longer chunks, which reduces the idle time caused by stop-and-go execution by about 5x, while achieving a higher success rate than fine-tuning with the same chunk length. Code and a real-robot demo are available at https://github.com/Seonghoon-Yu/RACE-VLA