This paper presents Proactive Human-in-the-Loop (PHIL), a framework for proactive policy correction and preference-based post-training. PHIL enables operators to preview policy-predicted future motion and intervene before anticipated undesirable actions. During takeover, PHIL continues querying the policy and displaying its predicted trajectories in the physical workspace through an extended reality (XR) interface. PHIL pairs rejected policy predictions with corrective action chunks subsequently executed from the same physical state over the same horizon, and preference-based post-training encourages the policy to favor these corrections. This paper evaluates PHIL on four real-world manipulation tasks using a diffusion policy and a pretrained vision-language-action policy. With five correction episodes per condition, PHIL increases mean success rates across the evaluated tasks and demonstration budgets from 44.0% to 70.3% for the diffusion policy and from 57.8% to 76.9% for X-VLA, compared with HG-DAgger. Separately, policies trained with proactive corrections show fewer unnecessary recovery behaviors in successful episodes. Project page: https://intuitive-robots.github.io/PHIL-page/.
Figures & tables
Fig. 2: Reactive versus proactive correction under equal correction budgets (two interventions per panel; only the first annotated). Solid and dashed lines denote executed robot motion and previewed policy continuations, respectively. Top: in these examples, reactive intervention follows observed errors and requires recovery. Bottom: XR previews enable intervention before anticipated errors. During correction, rejected predictions A− from scheduled queries are paired with subsequent executed corrective trajectories A+ from the same query state. Takeover and release leave the query schedule unchanged; control returns to the policy when the preview is acceptable.
Fig. 3: Preference pairs from fixed B -step queries during human intervention. Each retained rejected H -step prediction At− pairs with the next H executed actions At+ , yielding At+≻At− . Here, takeover coincides with a query and positive windows fall entirely within the human intervention period; generally, windows may include policy actions after release. Windows overlap when B<H . Takeover and release do not reset queries. Only complete pairs are retained; the latest prediction displayed at release is accepted and excluded from negatives.
Fig. 4: Example of proactive human intervention with PHIL during the Cook the Carrot task. Colored spheres visualize the policy-predicted future end-effector trajectories in XR; green and red frames indicate policy control and human intervention, respectively. (1–2) The policy approaches the pot and moves it onto the stove. (3–4) The operator intervenes when the previewed motion fails to approach the carrot or moves toward the pot before grasping the carrot, providing corrections before these undesirable motions are executed. Policy predictions remain visible during human intervention, allowing the operator to assess when to return control. (5) Once the predicted motion is acceptable, policy control resumes to place the grasped carrot into the pot.
Fig. 5: Diffusion Policy performance across base-demonstration budgets. Panels (a–d) show full-task success rates for Pick Up Banana, Towel Folding, Cook the Carrot, and Lemon in the Drawer, respectively; panels (e, f) show normalized task completion scores for Cook the Carrot and Lemon in the Drawer. Each measured point summarizes 25 evaluation rollouts. The points at (0,0) represent an assumed zero-demonstration baseline and were not evaluated experimentally. HG-DAgger and Residual Policy use reactive corrections, whereas PHIL uses proactive corrections; all three train with five correction episodes and the original base demonstrations.
Fig. 6: Correction-collection and post-training ablations with Diffusion Policy. Panels (a–d) show success rates for Pick Up Banana, Towel Folding, Cook the Carrot, and Lemon in the Drawer, respectively. Base policies use three demonstrations for Pick Up Banana and ten for each remaining task. Each bar summarizes 25 evaluation rollouts after post-training with five correction episodes and the original base demonstrations. R, H, and P denote Reactive, Hybrid, and Proactive collection; Hybrid combines two proactive and three reactive episodes. Methods within each collection mode share the same recorded correction data. Dashed horizontal lines indicate Base Policy success rates.
Fig. 7: Unnecessary recovery in Diffusion Policies post-trained with HG-DAgger or PHIL. Panels (a–d): Pick Up Banana, Towel Folding, Cook the Carrot, and Lemon in the Drawer. Each point averages recovery counts over ten successful episodes per method, collection mode, and base-demonstration budget, excluding recovery required after actual errors. The x-axis shows correction frames as a percentage of base-demonstration plus correction frames, decreasing rightward. Orange circles/blue diamonds denote HG-DAgger/PHIL; filled/open markers denote reactive/proactive collection. Dashed/dotted lines show least-squares linear fits across three budgets per method for reactive/proactive collection, respectively.
Task
Method
Base demonstrations
10
20
30
SR
TCS
SR
TCS
SR
TCS
Towel Folding
Base Policy
32
–
36
–
60
–
HG-DAgger
32
–
40
–
64
–
PHIL
64
–
72
–
72
–
Cook the Carrot
Base Policy
40
73.3
56
77.3
56
85.3
TABLE I: X-VLA results with different numbers of base demonstrations. HG-DAgger and PHIL use corrections collected over five interactive rollout episodes.
Task
Stages
Diffusion Policy demos
X-VLA demos
Metrics
Rollouts
Pick Up Banana
1
3/6/9
—
SR
25
Towel Folding
1
10/20/30
10/20/30
SR
25
Cook the Carrot
3
10/20/30
10/20/30
SR, TCS
25
Lemon in the Drawer
3
10/20/30
10/20/30
SR, TCS
25
TABLE II: Task, demonstration-count, and evaluation summary. Demo counts refer to base-policy demonstrations.
Method
Success Rate (SR)
Task Completion Score (TCS)
Banana
Towel
Carrot
Lemon
Carrot
Lemon
3
6
9
10
20
30
10
20
30
10
20
30
10
20
30
10
20
30
Base Policy
36%
48%
84%
24%
40%
52%
4%
24%
44%
0%
0%
4%
38.7%
65.3%
80.0%
0.0%
0.0%
6.7%
HG-DAgger
56%
76%
76%
48%
52%
64%
24%
28%
40%
20%
20%
24%
57.3%
64.0%
77.3%
40.0%
44.0%
52.0%
Residual Policy
56%
76%
80%
48%
64%
64%
24%
28%
72%
12%
40%
48%
56.0%
62.7%
88.0%
41.3%
62.7%
65.3%
PHIL
56%
76%
96%
68%
72%
72%
52%
80%
80%
40%
68%
84%
84.0%
93.3%
93.3%
60.0%
78.7%
94.7%
TABLE III: Main diffusion-policy comparison with different numbers of base demonstrations. Pick Up Banana uses 3/6/9 base demonstrations; all other tasks use 10/20/30. Long-horizon tasks report both full-task success rate (SR) and normalized task completion score (TCS).
Objective
Collection
Banana
Towel
Carrot
Lemon
Base Policy
—
36%
24%
4%
0%
HG-DAgger
Reactive
56%
48%
24%
20%
HG-DAgger
Proactive
56%
40%
28%
36%
PHIL
Reactive
48%
36%
24%
16%
PHIL
Proactive
56%
68%
52%
40%
HG-DAgger
Hybrid
60%
52%
36%
60%
TABLE IV: Controlled diffusion-policy ablation in the lowest-data regime. Each comparison changes the correction-collection mode or the post-training objective while holding the remaining protocol fixed.
HG-DAgger / collection
Pick Up Banana
Towel Folding
Cook the Carrot
Lemon in the Drawer
3
6
9
10
20
30
10
20
30
10
20
30
Evaluation recovery count
Reactive
21
17
9
24
19
15
23
20
14
24
18
10
Proactive
0
0
1
0
0
1
2
4
4
0
4
5
Training correction-frame ratio (%)
Reactive
23.40
13.66
11.19
40.44
23.39
16.43
32.82
18.00
14.20
23.23
12.72
5.98
TABLE V: Unnecessary recovery behaviors in a separate evaluation of diffusion policies post-trained with HG-DAgger, and the corresponding training-data composition. The comparison covers four tasks and three base-demonstration budgets, with ten successful episodes per condition.