cs.ROOct 30, 2026

ResMerge: Residual Policy Learning and Merging for Continual Adaptation of Pre-trained Robot Policies

Authors: Kefei Zhu, Lingyun Xu, Yue Xin, Xiaoqi Li, Caixin Guo, Xincheng Du, Yuanpei Chen, Yaodong Yang, 

Organizations: Peking University

Abstract

Continual adaptation to new tasks is essential for robots deployed in the real world, but adapting pre-trained policies remains challenging due to parameter growth and the need for continual demonstration collection. Parameter-Efficient Continual Fine-Tuning (PECFT) isolates task-specific updates, but adapter-per-task designs require demonstrations and grow with the task set. We propose ResMerge, a two-stage framework that learns mergeable residual policies atop a frozen pre-trained policy: residual reinforcement learning (RL) acquires task-specific corrections without additional expert demonstrations during adaptation, and parameter-space merging consolidates compatible residuals to reduce stored parameters. We introduce Residual-Field Consistency (RFC), a simple, bounded, and interpretable metric that assesses compatibility in residual action-field space by comparing correction direction and magnitude. Experiments on LIBERO across two backbones and on a real-world Franka robot show improved continual learning performance over strong PECFT baselines, while selective merging reduces the residual-parameter footprint. Project website: https://resmerge.github.io/.

Figures & tables

Explore similar work

CardsList
  1. Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning

    May 19, 2026Dongjie Yu, Kun Lei, Zhennan Jiang +2Robot Policy AdaptationReinforcement Learning

  2. Continual Robot Policy Learning via Variational Neural Dynamics

    Jun 25, 2026Jiaxu Xing, Zhiyuan Zhu, Yunfan Ren +4Robot Policy LearningRobot Policy Adaptation