cs.RONov 6, 2026

Who Taught the Robot That? Removing One User's Corrections from a Robot Policy That Learns Online

Authors: Eshika Pathak, Leela Krishna

Organizations: University of Illinois Urbana-Champaign · Centific

Abstract

A robot that keeps learning after deployment is corrected by several people in turn, and one of them may later ask to be removed. We show that whether the robot can forget that person depends on how people correct it. The goal of removal is the robot that would exist if the person had never been there, which we compute exactly in two simulated tasks with scripted users. When users correct the robot by taking over control, an ordered log of training updates is enough: retraining on the remaining sessions in their original order gives a robot as close to the goal as training noise allows. Deleting the person's data and retraining in one pass does not, even when the other users' corrections do not depend on the removed person, because it loses the order of training. When users only nudge the robot's action, the other users' corrections carry the removed person's influence, and no method we tested recovers the goal robot. Continuing to train without the person removes most of their habits, but not what their data in the training buffer keep alive. The log can identify one clearly harmful user, but not rank users by influence better than counting their corrections.

Figures & tables

Explore similar work

Oct 2, 2026

REDIRECT: A 1% Fix for Bad Robot Habits

Robots can acquire bad habits from a few defective moments in otherwise useful teleoperation. In mixed-quality robot data, normal and problematic demonstrations share most task behavior and differ only at a local action continuation. Full retraining is costly, while fine-tuning on clean data alone offers limited recovery under a small update budget. We ask whether the local difference can instead be fixed with only 1% of the full-training sample-backward budget. Using only episode-level retained/problematic labels, REDIRECT localizes the branch, assigns a coherent retained continuation to the original observations around it, and anchors shared behavior, without frame-level annotations or additional interaction. Across three randomized ManiSkill tasks and three seeds, REDIRECT raises the mean success rate from 68.2% to 90.3%, recovering 88.8% of the clean-retraining gap versus 22.1% for matched-compute fine-tuning. On a PiPER arm, it recovers 86.7-92.9% of the clean-only gap across cup insertion and towel folding. Local robot habits can therefore be repaired by spending the update budget on the behavioral difference rather than relearning shared behavior.
Feb 20, 2026

Enhancing Goal Inference via Correction Timing

Corrections offer a natural modality for people to provide feedback to a robot, by (i) intervening in the robot's behavior when they believe the robot is failing (or will fail) the task objectives and (ii) modifying the robot's behavior to successfully fulfill the task. Each correction offers information on what the robot should and should not do, where the corrected behavior is more aligned with task objectives than the original behavior. Most prior work on learning from corrections involves interpreting a correction as a new demonstration (consisting of the modified robot behavior), or a preference (for the modified trajectory compared to the robot's original behavior). However, this overlooks one essential element of the correction feedback, which is the human's decision to intervene in the robot's behavior in the first place. This decision can be influenced by multiple factors including the robot's task progress, alignment with human expectations, dynamics, motion legibility, and optimality. In this work, we investigate whether the timing of this decision can offer a useful signal for inferring these task-relevant influences. In particular, we investigate three potential applications for this learning signal: (1) identifying features of a robot's motion that may prompt people to correct it, (2) quickly inferring the final goal of a human's correction based on the timing and initial direction of their correction motion, and (3) learning more precise constraints for task objectives. Our results indicate that correction timing results in improved learning for the first two of these applications. Overall, our work provides new insights on the value of correction timing as a signal for robot learning.
Oct 26, 2026

Environment Edits as a Corrective Signal for Reward Learning

Robot corrections allow people to adjust a robot's behavior, but a single correction changes many features. Also, preferences about object attributes and relationships between objects cannot be expressed by moving the robot. We propose environment edits as a complementary form of feedback. The robot's motion is held constant as the human edits the scene, so the only explanation for the edit is the moved object. We introduce Feature Learning from Environmental and Robot-Relational Edit Traces (FERRET), which learns preferences from both robot and environment corrections. We show that environment edits are able to disambiguate features and learn preferences that robot corrections alone cannot.