Corrections offer a natural modality for people to provide feedback to a robot, by (i) intervening in the robot's behavior when they believe the robot is failing (or will fail) the task objectives and (ii) modifying the robot's behavior to successfully fulfill the task. Each correction offers information on what the robot should and should not do, where the corrected behavior is more aligned with task objectives than the original behavior. Most prior work on learning from corrections involves interpreting a correction as a new demonstration (consisting of the modified robot behavior), or a preference (for the modified trajectory compared to the robot's original behavior). However, this overlooks one essential element of the correction feedback, which is the human's decision to intervene in the robot's behavior in the first place. This decision can be influenced by multiple factors including the robot's task progress, alignment with human expectations, dynamics, motion legibility, and optimality. In this work, we investigate whether the timing of this decision can offer a useful signal for inferring these task-relevant influences. In particular, we investigate three potential applications for this learning signal: (1) identifying features of a robot's motion that may prompt people to correct it, (2) quickly inferring the final goal of a human's correction based on the timing and initial direction of their correction motion, and (3) learning more precise constraints for task objectives. Our results indicate that correction timing results in improved learning for the first two of these applications. Overall, our work provides new insights on the value of correction timing as a signal for robot learning.
Figures & tables
Figure 1. A human intervenes during a robot’s pre-planned trajectory ξ , resulting in a corrected trajectory c . The timing information is represented by tc , while the spatial information includes cp , cp′ , and cl .
Feature
Equation
Expectation Alignment (Vel.) ( Gopinath and Argall, 2020 ; Gopinath et al., 2022 )
∥vt∥∥vtopt∥vt⋅vtopt
Expectation Alignment (Pos.) ( Korkmaz and Bıyık, 2025 ; Tian et al., 2023 )
∥xt−xtopt∥
Directness Alignment (Vel.) ( Gopinath and Argall, 2020 ; Gopinath et al., 2022 )
∥vt∥∥vtg∥vt⋅vtg
Velocity Consistency ( Xie et al., 2019 ; Yao et al., 2022 )
∥vt∥∥vt−1∥vt⋅vt−1
Legibility ( Dragan et al., 2013 )
∫γtdt∫P(g∣ξS→xt)γtdt
Task Progress (Dist. to Goal) ( Gopinath and Argall, 2020 ; Gopinath et al., 2022 )
∥xt−xg∥
Table 1. Time-series features for modeling robot behavior and human corrections, grounded in prior work linking motion characteristics to human perception and intervention.
Figure 2. Cumulative first-correction timing across trajectory completion. “Correction Ratio” denotes the proportion of first-correction events occurring up to each completion percentage.
Figure 3. F1 scores for the multi-feature model and Boltzmann baseline across correction timing percentages (portion of trajectory completed before the first correction). Statistical significance in all figures is indicated as p≤0.05∗ , p≤0.01∗∗ , p≤0.001∗∗∗ .
Figure 4. Predicted-to-actual correction ratio for the multi-feature model and Boltzmann baseline across correction timing percentages.
Figure 5. Mean absolute error (seconds) between predicted and true correction timing for the multi-feature model and Boltzmann baseline across correction timing percentages. Lower MAE indicates better accuracy.
Figure 6. F1 scores for the baseline and single-feature ablations across correction timing percentages. Significance denotes deviation from the baseline, indicating feature impact. A drop in F1 suggests positive contribution of the removed feature.
Figure 7. KLD between ground-truth and inferred goal distributions after each correction in the test set, aggregated over 50 random splits. Results are shown for each goal inference model (WHEN, WHERE, COMBINED) across correction timing percentages, aggregated over targets and shapes. Small circles denote outliers beyond 1.5× the interquartile range.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Corr. w/ x
Corr. w/ y
Corr. w/ z
Grasp position cp
0.092
0.039
-0.600
Initial correction velocity cp′
0.077
-0.010
0.264
Appendix
Table 2. Pearson correlation between normalized correction timing tc and correction spatial features.
Figure 8. KLD vs. different weight α
Figure 9. KLD between the ground-truth goal distributions and the inferred goal using the start of the correction information after each correction in the test set, evaluated over 50 random dataset splits. Results are shown for each goal inference model (WHEN, WHERE, and COMBINED) across different correction timing percentages. The results are aggregated across shapes.
Correcting robot manipulation policies through human intervention holds great promise for real-world deployment, yet human operators are inherently imperfect in both the actions they provide and the timing of their intervention signals. While the former has been extensively discussed in reinforcement learning (RL), the latter remains underexplored. At high control frequencies, human intervention signals are often delayed and inconsistent across time and state space. In this work, we present GAINS, a framework for leveraging inconsistent human intervention signals in RL. At the core of GAINS, we employ distributional RL with quantile Q-networks to model the return variability induced by sparse task rewards and inconsistent human interventions. Building on this distributional representation, we introduce a pessimistic exploration strategy that promotes safe and sample-efficient learning under human corrections. We evaluate GAINS on four diverse simulated manipulation tasks and two challenging real-world scenarios against state-of-the-art intervention-based methods. GAINS achieves a 22% higher task success rate than RLIF and improves recovery success by up to 43% in failure scenarios. These results highlight the importance of modeling return variability induced by human imperfection for real-world deployment of intervention-based learning.
Xinyi Zhang, Yinuo Zhao, Pei Ren +7
Beijing Institute of Technology · Beijing Innovation Center of Humanoid Robotics · City University of Hong Kong +1
Learning reward functions from demonstrations assumes that demonstrations provide adequate supervision over all features -- or task-relevant aspects of behavior. In practice, demonstrations are often imperfect: humans may under-emphasize certain features due to cognitive load or physical difficulty, or the training regime may fail to sufficiently cover all relevant situations. In either case, important features may be underspecified, leading to ambiguity in the learned reward function and misaligned behavior at deployment. We propose a framework that detects such underspecified features and actively solicits targeted corrective demonstrations. Our key insight is that demonstrations implicitly reveal which features are well specified: features that are consistently optimized show little variation across demonstrations, while features that are underspecified vary widely. We leverage this statistical signal to infer which features may have been insufficiently demonstrated. The robot then explains which features it is uncertain about in natural language and queries for demonstrations that explicitly address the identified gaps. We evaluate our approach in a simulated tabletop manipulation domain and in a user study with a real Franka robot. Targeted, explanation-guided queries significantly improve reward recovery compared to random querying and passive data collection, reducing ambiguity that would otherwise persist in learning from imperfect demonstrations.
Robots can acquire bad habits from a few defective moments in otherwise useful teleoperation. In mixed-quality robot data, normal and problematic demonstrations share most task behavior and differ only at a local action continuation. Full retraining is costly, while fine-tuning on clean data alone offers limited recovery under a small update budget. We ask whether the local difference can instead be fixed with only 1% of the full-training sample-backward budget. Using only episode-level retained/problematic labels, REDIRECT localizes the branch, assigns a coherent retained continuation to the original observations around it, and anchors shared behavior, without frame-level annotations or additional interaction. Across three randomized ManiSkill tasks and three seeds, REDIRECT raises the mean success rate from 68.2% to 90.3%, recovering 88.8% of the clean-retraining gap versus 22.1% for matched-compute fine-tuning. On a PiPER arm, it recovers 86.7-92.9% of the clean-only gap across cup insertion and towel folding. Local robot habits can therefore be repaired by spending the update budget on the behavioral difference rather than relearning shared behavior.