DiMuRe: Diagnosis-Guided Multilevel Recovery for Robotic Task Execution
Organizations: Beijing Institute of Technology, Beijing, China · Shenzhen MSU-BIT University, Shenzhen, China · AInnovation Technology Group Co., Ltd.
Abstract
Disturbances can invalidate the conditions required by a robot's next action even when much of its task program remains useful. A recovery system must therefore choose a response suited to the current state. We present DiMuRe (diagnosis-guided multilevel recovery), a framework for robot-use agents that uses phase-specific evidence and method preconditions to select program reuse or predefined recovery skills, including bounded vision-language-action (VLA) grasping. After the original program exits, a multimodal large language model (MLLM) can generate corrective code from images and execution feedback. Local verification checks the outcome of each recovery attempt. We also introduce LIBERO-Disturb, a benchmark for robustness and recovery under mid-execution disturbances, supporting program-based agents and VLA policies. On 801 scheduled disturbed Encore development cases, Local DiMuRe (program reuse and recovery skills) achieves 77.65% complete-task success, compared with 58.68% for the original program under the same physical-step allowance. Full DiMuRe achieves 81.90% with terminal replanning, an additional step allowance, and early handoff. Local DiMuRe also uses fewer physical steps per scheduled episode than the original program.
Figures & tables
| Configuration | Clean SR (%) | Disturbed SR (%) |
|---|---|---|
| Encore | ||
| Original | 100.00 | 58.68 |
| Local DiMuRe (L1+L2) | 100.00 | 77.65 |
| Full DiMuRe | 100.00 | 81.90 |
| (a) Task outcomes and execution costs | ||||||
|---|---|---|---|---|---|---|
| Configuration | SR (%) | SR (pp) | Mean task time (s) | Mean physical steps | ||
| All cases | Successful cases | All cases | Successful cases | |||
| Encore: shared native allowance | ||||||
| Original program | 58.68 | 0.00 | 4.65 | 2.93 | 328.90 | 213.46 |
| L1-only | 73.66 | 5.24 | 3.68 | 317.32 | 229.20 | |
| L2-only | 65.54 | 5.91 | 3.95 | 349.72 | 229.49 | |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Meaning | Value | Unit |
|---|---|---|---|
| Minimum tool displacement for evaluating attachment | 0.025 | m | |
| Maximum accepted object/tool displacement residual | 0.035 | m | |
| Lower admissible gripper opening | 0.003 | m | |
| Upper admissible gripper opening | 0.070 | m |
| Symbol | Applicable procedure or invocation | Value | Unit |
|---|---|---|---|
| Shared candidate-attempt limit across the task | 3 | attempts | |
| Grasp-recovery entry | 250 | steps | |
| Placement-retry entry | 230 | steps | |
| Grasp recovery and placement retry | 150 | steps | |
| Grasp-recovery procedure | 450 | steps | |
| Placement-retry procedure | 500 | steps |
| Symbol | Meaning | Value | Unit |
|---|---|---|---|
| Minimum remaining budget for planner entry | 80 | steps | |
| Minimum remaining budget for beginning a round | 40 | steps | |
| Maximum planning rounds within the invocation | 32 | rounds | |
| Shared cap on generic reviews of unverified stop proposals | 2 | requests | |
| Maximum counted physical-command calls per fragment | 12 | commands | |
| Wall-clock execution limit per generated fragment | 45 | s |
| Setting | Value |
|---|---|
| Task programs | 30 fixed Encore programs |
| Random seeds | Environment reset: 51, 52, 53; disturbance RNG: |
| Evaluation sets | Original/Local/Full: 90 clean and 891 disturbed each; L1-only/L2-only: 801 disturbed each |
| Controlled components | Fixed task programs, perception models, and disturbance settings |
| Disturbance protocol | Trigger rules, perturbation parameters, and validity checks in Appendix C |
| Recovery configurations | Original: no added recovery; L1-only: program reuse; L2-only: geometry/VLA skills; Local: L1+L2; Full: Local plus terminal replanning |
| Metric | Definition and denominator |
|---|---|
| SR and SR | Native task successes divided by all scheduled episodes, including unreached or ineffective disturbances. SR is the difference from Original in percentage points, computed from unrounded rates. |
| Recovery and retention | Successful outcomes among Original failures and among Original successes, respectively, paired by scheduled case. |
| Mean task time and steps | Arithmetic means over all scheduled episodes, including failures, or over each configuration’s successful subset as labeled. Successful subsets can differ between configurations. |
| Recorded recovery time | Sum of logged stage intervals within the task interval, averaged over episodes entering that stage before the endpoint, including failed episodes. Local and terminal stages are measured separately. |
| Anomaly-response coverage | Auditable first-fault windows containing a qualifying alarm or target adaptation, divided by all auditable windows; repeated responses count once. |
| Task SR after response | Native task successes strictly after the first qualifying response, divided by responding windows. |
| Evaluation set | Original | Local | Full | |
|---|---|---|---|---|
| Main-text subset (without 6 cm) | 801 | 470/801 | 622/801 | 656/801 |
| Full disturbed set | 891 | 549/891 | 689/891 | 725/891 |
| Receiving-region shift, 6 cm | 90 | 79/90 | 67/90 | 69/90 |
| Receiving-region shift, 12 cm | 90 | 49/90 | 77/90 | 77/90 |
| Configuration | Applied | Effect verified | Effect check missing | Final check passed |
|---|---|---|---|---|
| Original | 631 | 105 | 696 | 470 |
| L1-only | 632 | 105 | 696 | 590 |
| L2-only | 632 | 105 | 696 | 523 |
| Local DiMuRe | 632 | 105 | 696 | 620 |
| Full DiMuRe | 632 | 105 | 696 | 654 |
| Configuration | Original failures recovered (%) | Original successes retained (%) |
|---|---|---|
| Full DiMuRe | 194/331 (58.61) | 462/470 (98.30) |
| Local DiMuRe (no L3) | 159/331 (48.04) | 463/470 (98.51) |
| L1-only | 125/331 (37.76) | 465/470 (98.94) |
| L2-only | 69/331 (20.85) | 456/470 (97.02) |
| Disturbance | Original | L1-only | L2-only | Local | Full | |
|---|---|---|---|---|---|---|
| Arm retraction | 90 | 83 | 83 | 84 | 84 | 84 |
| Grasp slip | 90 | 38 | 72 | 49 | 70 | 76 |
| Object bump | 90 | 90 | 90 | 90 | 90 | 90 |
| Object pose change | 81 | 26 | 26 | 47 | 47 | 47 |
| Receiving-region shift (12 cm) | 90 | 49 | 77 | 49 | 77 | 77 |
| Object retrieval | 90 | 24 | 58 | 29 | 58 | 69 |
| Fault family | Response (both) | Correct diagnosis (both) | Task success after response | Task success after correct diagnosis (both) | |
|---|---|---|---|---|---|
| Local | Full | ||||
| Grasp not established | 31/34 | 15/34 | 23/31 | 24/31 | 15/15 |
| Holding lost | 30/34 | 12/34 | 15/30 | 21/30 | 11/12 |
| Receiving region moved | 31/36 | 30/36 | 28/31 | 28/31 | 28/30 |
| Target state changed | 34/57 | 0/57 | 28/34 | 28/34 | N/A |
| Total | 126/161 | 57/161 | 94/126 | 101/126 | 54/57 |
| Configuration | L3 requests | Input tokens | Output tokens | Cached input tokens |
|---|---|---|---|---|
| Full DiMuRe | 1.93 | 204,421 | 570 | 30,825 |
| Disturbance | Trigger or target selection | Nominal intervention |
|---|---|---|
| Object XY displacement | Step 30; declared task target | Random horizontal shift of 6–10 cm; bounded by workspace, separation, and penetration checks |
| Object bump | Step 40; random free object excluding the nearest to the tool | Horizontal velocity increment of 0.3–0.6 m/s in a random direction |
| Slow object drift | Step 20; object nearest to the tool | Horizontal translation of 1 mm per step for 60 steps in a random direction |
| Object retrieval | Poll every 5 steps from step 80; held object displaced more than 8 cm in XY | Reset the object’s position to its initial position and zero its velocity |
| Object pose change | Poll every 10 steps from step 20 to 520; eligible resting target | Apply the predefined orientation assigned to the case, subject to clearance and settling checks |
| Object tipover | Step 40; object nearest to the tool | Rotate by about a random horizontal axis and raise by 3 cm |
| Setting | Value or rule |
|---|---|
| Initial state | Official state index 0 for each task |
| Random seeds | Environment 7; policy 7; disturbance 51 |
| Native physical-step limits | Spatial: 220; Object: 280; Goal: 300; Long: 520 |
| Initialization and action queue | 10 native wait steps; 5-action chunks |
| Trigger polling | Begin after 20 policy steps; retry after 10 steps if rejected |
| Transported-object eligibility | Lift cm; tool-to-object distance cm; closing command; bilateral contact for 3 consecutive steps |
| Condition | Runs | Successes | SR (%) | Applied | Safety stops |
|---|---|---|---|---|---|
| Clean | 40 | 39 | 97.5 | 0 | 0 |
| Empty hook | 40 | 39 | 97.5 | 0 | 0 |
| Receptacle relocation | 40 | 36 | 90.0 | 22 | 1 |
Robot Failure Recovery
Robot failure recovery studies how robots detect, diagnose, and respond to errors, damage, and unexpected physical interactions.
See all 81 papers