Continual adaptation to new tasks is essential for robots deployed in the real world, but adapting pre-trained policies remains challenging due to parameter growth and the need for continual demonstration collection. Parameter-Efficient Continual Fine-Tuning (PECFT) isolates task-specific updates, but adapter-per-task designs require demonstrations and grow with the task set. We propose ResMerge, a two-stage framework that learns mergeable residual policies atop a frozen pre-trained policy: residual reinforcement learning (RL) acquires task-specific corrections without additional expert demonstrations during adaptation, and parameter-space merging consolidates compatible residuals to reduce stored parameters. We introduce Residual-Field Consistency (RFC), a simple, bounded, and interpretable metric that assesses compatibility in residual action-field space by comparing correction direction and magnitude. Experiments on LIBERO across two backbones and on a real-world Franka robot show improved continual learning performance over strong PECFT baselines, while selective merging reduces the residual-parameter footprint. Project website: https://resmerge.github.io/.
Figures & tables
Fig. 2: ResMerge framework. In the Learn stage, a LoRA-parameterized residual policy Rcand is trained on the new task via residual RL with πbase frozen. In the Merge stage, Rcand is merged into the most compatible existing expert when their Residual-Field Consistency (RFC) meets or exceeds a compatibility threshold; otherwise it is added as a new expert.
Fig. 3: Overview of the experimental platforms and tasks. A Franka Panda with a Leap hand performs four sequential pinch- and power-grasp tasks; three frames illustrate each rollout. The bottom strip shows representative LIBERO-Spatial and LIBERO-Object tasks used for simulation comparisons.
Base Policy
LIBERO-Spatial
LIBERO-Object
Architecture
CL Method
MPE ↑
Demonstrations ↓
FWT ↑
NBT ↓
AUC ↑
FWT ↑
NBT ↓
AUC ↑
ViT-T
Seq. SFT
—
50 demos/task
63±2
76±1
16±1
76±3
73±3
27±2
ER
—
50 demos/task
63±1
29±2
50±2
70±2
28±1
57±1
PackNet
—
50 demos/task
57±4
15±1
59±3
58±3
18±2
56±4
TAIL
0.70
50 demos/task
51±2
56±1
60±3
58±2
50±2
70±1
DMPEL
0.69
50 demos/task
22±2
28±3
29±2
14±3
25±1
27±3
TABLE I: Continual learning on LIBERO. Results are mean ± standard deviation over five seeds. Demonstrations are counted only during adaptation.
Fig. 4: Sequential RFC analysis on the real robot. Streamlines show the compared correction fields; background color indicates local directional agreement (green) or opposition (red). Statistics use full observations and actions; the plotted projection is only a visualization. (a) RFC accepts (R1,R2) , producing RAB . (b) RFC rejects (RAB,R3) and retains R3 . (c) RFC accepts (R3,R4) , producing RCD . Two consolidated experts serve all four tasks.
(a) Continual learning metrics
Method
FWT (%)
NBT (pp)
AUC (%)
DMPEL
24
23
54
ResMerge
50
7
75
(b) Average success and residual-expert count
Method
Avg. SR (%)
Experts
TAIL
52.5
–
TABLE II: Real-world evaluation. We report continual learning metrics in (a) and average success with residual-expert counts in (b).
Strategy
Learn-SR
Merge-SR
FWT
NBT
AUC
No Merge
36
–
27
13
54
Merge-All
36
21
23
22
37
ResMerge
36
78
49
−8
79
TABLE III: Effect of merging strategy. Learn-SR and Merge-SR are mean success rates across all ten LIBERO-Object tasks with ViT-T. Merge-SR includes the RL refinement that follows consolidation.
Residual Architecture
Learn-SR (%)
Merge-SR (%)
MLP Residual Policy
36
14
LoRA Residual Policy (ours)
36
78
TABLE IV: Effect of residual architecture. LoRA and MLP residuals are compared before and after merging and refinement.
Parameter
OpenVLA-OFT
Real robot
Actor learning rate
3×10−4
10−5
Critic learning rate
3×10−4
3×10−4
Residual scaling factor α
0.05
0.05
TABLE V: Implementation settings by experimental platform. Real-robot architecture and evaluation details are specified separately in the text.
Threshold τ
FWT
NBT
AUC
0.1
23
22
37
0.3
49
−2
71
0.5
49
−8
79
0.7
27
22
47
TABLE VI: RFC threshold sensitivity. We vary τ on LIBERO-Object with ViT-T, keeping it fixed throughout each task sequence.
Metric
Pre-SR
Post-SR
FWT
NBT
AUC
Task Similarity
36.0
40.0
26.0
11.0
46.0
RFC
36.0
78.0
49.0
−8.0
78.7
TABLE VII: Comparison of merging metrics. Results are on LIBERO-Object with ViT-T. Post-merge success includes RL refinement.
Stage and residual pool
T1
T2
T3
T4
T1 : {R1}
85
35
55
55
T2 : {RAB}
75
60
55
50
T3 : {RAB,R3}
75
60
85
50
T4 : {RAB,RCD}
75
60
75
80
TABLE VIII: Real-robot success across the task sequence. Rows indicate the completed training stage and residual pool; columns indicate the evaluated task. Each cell is evaluated over 20 trials. Success rates are percentages.
School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR. · Shanghai Qizhi Institute, Shanghai, China. · Shanghai Jiao Tong University, Shanghai, China. +2