cs.LGNov 19, 2026

Improving Imitation Policies for Multi-Phase Tasks through Residual Learning Allocation

Authors: Clinton Enwerem, Saleh Kemal, John S. Baras, Calin Belta

Organizations: Institute for Systems Research, University of Maryland, College Park, MD, USA

Abstract

Residual reinforcement learning corrects an imitation-trained base policy using a limited budget of environment interaction. In a task with several phases, this budget must be divided among the phases, and the phase in which the base policy fails is not necessarily the phase in which a correction most improves task completion. We formalize this distinction through the completion gain: the change in full-task completion probability when, during a single phase, a state-feedback controller replaces selected components of the base policy's action. We decompose the completion gain into the probability of reaching the phase and the conditional improvement in completion from the start of the phase, and we estimate it from paired episodes. The positive estimates determine which action components the residual policy corrects and how training episodes are distributed across phases. In stochastic point-mass docking experiments, most failures occurred in the final docking phase, whereas the largest completion gain occurred in the preceding channel phase; gain-based allocation improved completion over allocation proportional to failure counts and, in a variant with a longer approach, over uniform allocation at half of the budget. In simulated bimanual robotic assembly, the completion-gain estimates were positive only for insertion and release, although few failures occurred during release; in a second assembly configuration, a residual policy trained in the release phase improved completion from held-out states at the start of release by 16.9%.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When, Where, and What to Correct: Human-Guided Adaptive Residual Correction for Robot Policies

    Oct 27, 2026Anukriti Singh, Maxime Caniot, Pratap Tokekar +1Robot Policy AdaptationResidual Policy Learning

  2. Res-HIL: Human-Guided Residual Reinforcement Learning for Sample-Efficient Dexterous Manipulation

    Sep 24, 2026Mariia Iavorskaia, Christian Dietz, Sebastian Albrecht +1Contact-Rich Robotic ManipulationHuman-in-the-Loop AI

  3. PRISM: Personalized Refinement of Imitation Skills for Manipulation via Human Instructions

    Mar 5, 2026Arnau Boix-Granell, Alberto San-Miguel-Tello, Magí Dalmau-Moreno +1Robot Imitation LearningInteractive Imitation Learning