Large pretrained robot foundation models provide strong priors for manipulation, yet adapting them to target tasks with offline teleoperated demonstrations often fails to cover the contact-rich, recovery, and precision-critical states encountered during deployment. Existing methods mitigate this issue by augmenting offline demonstrations with human-in-the-loop data or online reinforcement learning. Despite this, offline post-training adapts slowly because it cannot immediately absorb deployment experience, while online RL typically relies on value functions or auxiliary actor-critic models, increasing both training complexity and instability. We present TIDE (Training In DEployment), a human-in-the-loop online post-training system. TIDE introduces a provenance-conditioned supervised learning framework that updates the policy online from human interventions and policy-generated rollouts collected during deployment, without a learned critic, reward model, or action scorer. Across four real-world bimanual manipulation tasks, TIDE improves milestone-normalized task progress and binary full-task completion over the warm-start policy and matched additional offline demonstrations, and performs comparably to the fixed-policy collection baseline while making updated checkpoints available during deployment. Each task-specific online post-training session, from the start of deployment rollouts until the final checkpoint is available, completes in less than one hour while learning and checkpoint handling run asynchronously.
Figures & tables
Fig. 2: The pipeline of TIDE. The robot executes policy actions with human takeover when needed. The system records executed actions and labels their human or policy source. The learner combines this online data with offline demonstrations to update the video model and action expert using provenance conditioning. Updated policies return to the robot at episode boundaries, while data collection and learning proceed asynchronously within the same deployment session.
Fig. 3: Real-world evaluation tasks. Initial and successful final states for four bimanual manipulation tasks.
Fig. 4: Real-robot setup. The bimanual UR5e platform uses Robotiq 2F85 grippers, one third-person RealSense D455 camera, and two wrist-mounted RealSense D405 cameras.
Fig. 5: Action representation study. Offline-only comparison of eight action spaces using the same data, initialization, and training budget. Joint-relative actions perform best and are used in subsequent experiments.
Fig. 6: Autonomous evaluation over 20 no-takeover rollouts per method–task pair. (a) Mean milestone-normalized task progress. (b) Binary full-task completion. Labels report completed rollouts out of 20.
Fig. 7: Qualitative autonomous rollouts before and after online post-training. For each task, the top and bottom rows show no-takeover rollouts from the warm-start policy and final TIDE checkpoint, respectively, with time progressing from left to right. The warm-start policy fails to complete the tasks in these examples, whereas TIDE completes the full manipulation sequences, illustrating improved autonomous execution after online post-training.
Fig. 8: Deployment diagnostics on Bit Insertion. (a) The intervention ratio decreases over deployment, indicating reduced human-supervision burden. (b) TIDE achieves higher normalized task progress than variants without provenance conditioning or using only human-executed online data, highlighting the value of source conditioning and learning from policy-executed experience.
HHCM, Istituto Italiano di Tecnologia, Genoa, Italy · DIBRIS, University of Genova, Genova, Italy · Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genova, Italy +1