Vision-Language-Action (VLA) models leverage large-scale pretraining to ultimately achieve generalist manipulation. Deployed VLA policies must support continual learning to acquire new tasks over time. Teaching a VLA a new task generally requires finetuning it on demonstrations of that task. However, naively finetuning on downstream tasks causes the policy to forget earlier tasks and degrades generalist capabilities. This failure is known as catastrophic forgetting. Most continual learning methods counter it by replaying data from earlier tasks. However, the old task demonstrations are not always readily available. In this paper, we introduce SAMBAR, a continual learning algorithm that prevents catastrophic forgetting during VLA finetuning without requiring access to the demonstrations of any previously learned task. We propose to cast continual learning as a constrained optimization problem and solve it with the method of multipliers. In our approach, the method of multipliers drives the policy to learn the new task without the model parameters drifting far away from their previous values. In contrast to a standard regularization penalty, the method of multipliers raises the penalty as the constraint violation accumulates by using a dual variable. We also selectively anchor the parameters critical to previous tasks to preserve past knowledge, leaving other parameters free for new task acquisition. The combination of dual variable and selective anchoring, therefore, balances knowledge acquisition with knowledge retention. We evaluate our method, SAMBAR, on the LIBERO simulation benchmark and on hardware. When sequentially finetuning on a VLA, every replay-free baseline we compare against completely forgets the first task it learned, whereas SAMBAR retains every task it has learned.
Figures & tables
Fig. 1: Overview of SAMBAR for replay-free continual VLA learning. A policy learns tasks T1,T2,T3 sequentially using only the current task’s demonstrations. Orange , blue , and green indicate parameter importance for {\color[rgb]{0.8945,0.4219,0.043}T_{1}} , {\color[rgb]{0.3281,0.5586,0.8359}T_{2}} , and {\color[rgb]{0.4688,0.5781,0.2344}T_{3}} , respectively, with darker shades indicating greater importance. Standard supervised finetuning (SFT) can overwrite parameters important to earlier tasks, causing forgetting. SAMBAR combines an importance-weighted parameter anchor with the method of multipliers to limit changes to parameters important to prior tasks while allowing new-task adaptation.
Fig. 2: The three finetuning LIBERO tasks we learn in sequence. Starting from a policy pretrained on 32 LIBERO tasks, we finetune on pot-on-stove , then bowl-in-drawer , then bottle-on-rack , in the order shown left to right. Each panel shows the initial scene the policy is given. The first two are long-horizon tasks, while the third is a goal task.
Method
Pretrain SR ↑
Finetune SR ↑
Weighted Avg SR ↑
Finetune task: pot-on-stove
Pretrained (no finetuning)
93.2 ± 1.2
4.0 ± 11.1
-
SFT
89.0 ± 0.9
100.0 ± 0.0
89.1
RETAIN [ 31 ]
89.7 ± 1.2
100.0 ± 0.0
89.8
LoRA [ 19 ]
88.1 ± 0.6
100.0 ± 0.0
88.2
EWC [ 5 ]
89.5 ± 1.4
100.0 ± 0.0
89.6
TABLE I: Learning one new task from a broadly pretrained policy. We finetune a pi0.5 policy pretrained on 117 LIBERO tasks on finetune tasks, and evaluate it on the 117 pretraining tasks ( Pretrain SR ) and on the finetune task ( Finetune SR ). All entries are success rates in percent, averaged over 5 random seeds with 5 episodes per seed, reported as mean ± 95% confidence interval; higher is better ( ↑ ) everywhere. Weighted Avg SR is the task-count-weighted mean over all 118 tasks. Forgetting is mild in this regime, and SAMBAR learns each new task without significantly forgetting the 117 pretrained tasks.
Method
Pretrain SR ↑
Finetune SR ↑
Weighted Avg SR ↑
Finetune task: pot-on-stove
Pretrained (no finetuning)
93.9 ± 1.4
6.0 ± 16.7
-
Co-Training
93.8 ± 1.3
94.0 ± 6.8
93.8
SFT
56.1 ± 2.6
96.0 ± 6.8
57.3
RETAIN [ 31 ]
66.5 ± 2.1
94.0 ± 11.1
67.3
LoRA [ 19 ]
75.4 ± 1.8
100.0 ± 0.0
76.2
TABLE II: Learning one new task from a narrowly pretrained policy. We finetune a pi0.5 policy pretrained on 32 LIBERO tasks (8 from each suite) on two LIBERO-Long finetune tasks, and evaluate it on the 32 pretraining tasks ( Pretrain SR ) and on the finetune task ( Finetune SR ). All entries are success rates in percent, averaged over 5 random seeds with 10 episodes per seed, reported as mean ± 95% confidence interval; higher is better ( ↑ ) everywhere. Weighted Avg SR is the task-count-weighted mean over all 33 tasks. Forgetting is severe in this regime: every baseline forgets the pretraining tasks, whereas SAMBAR remembers the pretraining tasks.
Success on each newly learned task
Method
Pretrain SR ↑
pot-on-stove ↑
bowl-in-drawer ↑
bottle-on-rack ↑
Weighted Avg SR ↑
Co-Training
95.4 ± 1.0
100.0 ± 0.0
98.0 ± 5.6
88.0 ± 13.6
95.4
SFT
12.9 ± 1.9
0.0 ± 0.0
60.0 ± 21.5
84.0 ± 11.1
15.9
RETAIN [ 31 ]
17.4 ± 0.7
0.0 ± 0.0
68.0 ± 16.2
96.0 ± 6.8
20.6
LoRA [ 19 ]
45.4 ± 1.3
0.0 ± 0.0
18.0 ± 13.6
92.0 ± 10.4
44.6
EWC [ 5 ]
0.0 ± 0.0
0.0 ± 0.0
0.0 ± 0.0
100.0 ± 0.0
2.9
TABLE III: Sequential Continual Learning. Starting from the policy pretrained on 32 tasks, we finetune on pot-on-stove , then bowl-in-drawer , then bottle-on-rack . We evaluate the policy after learning all three tasks. All entries are success rates in percent, averaged over 5 random seeds, 10 episodes/seed and reported as mean ± 95% confidence interval; higher is better ( ↑ ) for every column. Weighted Avg SR is the task-count-weighted mean over all 35 tasks. After three tasks every baseline has completely forgotten pot-on-stove task, while SAMBAR still performs it 84.0% of the time along with high success rate on 32 pretrained tasks.
Pretraining tasks
Method
bird-in-bin ↑
two-birds-in-bin ↑
eraser-in-mug ↑
eraser-in-bowl ↑
Pretrain SR ↑
Finetune SR ↑
Weighted Avg SR ↑
Finetune task: cake-on-cake-tray
SFT
4/5
1/5
5/5
2/5
60.0
5/5
68.0
RETAIN ( α=0.5 )
5/5
4/5
3/5
5/5
85.0
5/5
88.0
EWC
5/5
4/5
5/5
2/5
80.0
4/5
80.0
SAMBAR (ours)
5/5
5/5
5/5
5/5
100.0
5/5
100.0
TABLE IV: Learning one new task on hardware. We pretrain pi0.5 on four tasks and then finetune it on each of the two finetuning tasks. We evaluate every task with 5 rollouts and report the individual tasks as the number of successful rollouts out of 5. Pretrain SR is the percentage of the 20 pretraining rollouts that succeed, Weighted Avg SR is the percentage of all 25 rollouts that succeed. Higher is better ( ↑ ) for every column. SAMBAR is the only method that keeps all four pretraining tasks intact while learning the new one.
Pretraining tasks
Finetune tasks
Method
bird-in- bin ↑
two-birds- in-bin ↑
eraser-in- mug ↑
eraser-in- bowl ↑
Pretrain SR ↑
cake-on- cake-tray ↑
remove-lid-and- place-bird ↑
Weighted Avg SR ↑
SFT
5/5
5/5
5/5
3/5
90.0
4/5
3/5
83.3
RETAIN ( α=0.5 )
5/5
5/5
4/5
4/5
90.0
5/5
0/5
76.7
EWC
5/5
5/5
5/5
5/5
100.0
4/5
2/5
86.7
SAMBAR (ours)
5/5
5/5
5/5
5/5
100.0
4/5
4/5
93.3
TABLE V: Sequential Continual Learning on hardware. Starting from the policy pretrained on the four tasks, we finetune on the cake-on-cake-tray task and then on the remove-lid-and-place-bird task, and evaluate all six tasks after learning the remove-lid-and-place-bird task. Entries follow the same convention as Table IV , with Weighted Avg SR now the percentage of all 30 rollouts that succeed.
Fig. 3: The hardware experiment tasks. We pretrain pi0.5 on the four tasks on the left and finetune on the two on the right. Each panel shows the initial scene the policy is given and the arrows shows the task policy needs to perform.
When deployed in open-ended robotic environments, Vision--Language--Action (VLA) models need to continually acquire new skills, yet suffer from severe catastrophic forgetting. We observe that this degradation is related to the deterioration of cross-modal information structure, where dependencies among visual observations, language instructions, and actions progressively diffuse during continual adaptation. But existing continual learning methods fail to preserve such cross-modal information dependencies. Thus, we propose Info-VLA, an information-preserving continual learning framework that maintains cross-modal information structure through two complementary constraints. Replay Anchor Contrastive Learning constructs stable alignment anchors from a frozen teacher model, preserving cross-modal alignment in the representation space. Cross-Modal Mutual Information Maximization further preserves dependency structure between visual and language representations through mutual information constraints. By jointly preserving historical alignment and cross-modal dependency information, Info-VLA balances stability and plasticity during continual learning. Furthermore, experiments on the LIBERO show that Info-VLA significantly outperforms existing methods in both task retention and adaptation.
Libang Zhao, Qixin Zeng, Hongyin Zhang +1
Westlake University, China · University of Southampton, UK
Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across diverse lifelong RL benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability-plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era. Code is available at https://github.com/UT-Austin-RobIn/continual-vla-rl
Continual learning is a long-standing challenge in robot policy learning, where a policy must acquire new skills over time without catastrophically forgetting previously learned ones. While prior work has extensively studied continual learning in relatively small behavior cloning (BC) policy models trained from scratch, its behavior in modern large-scale pretrained Vision-Language-Action (VLA) models remains underexplored. In this work, we found that pretrained VLAs are remarkably resistant to forgetting compared with smaller policy models trained from scratch. Simple Experience Replay (ER) works surprisingly well on VLAs, sometimes achieving zero forgetting even with a small replay data size. Our analysis reveals that pretraining plays a critical role in downstream continual learning performance: large pretrained models mitigate forgetting with a small replay buffer size while maintaining strong forward learning capabilities. Furthermore, we found that VLAs can retain relevant knowledge from prior tasks despite performance degradation during learning new tasks. This knowledge retention enables rapid recovery of seemingly forgotten skills through finetuning. Together, these insights imply that large-scale pretraining fundamentally changes the dynamics of continual learning, enabling models to continually acquire new skills over time with simple replay. Code and more information can be found at https://continual-vlas.github.io/forget-me-not/
Huihan Liu, Changyeon Kim, Bo Liu +2
The University of Texas at Austin · KAIST · Microsoft Superintelligence.