Vision-Language-Action (VLA) models pre-trained on large-scale, closed datasets have demonstrated remarkable success across diverse robotic manipulation tasks. However, their long-term real-world deployment necessitates continuously acquiring new skills while retaining previously learned capabilities. While pioneering works have explored continual VLA adaptation using techniques such as experience replay and reinforcement fine-tuning, they overlook a foundational mechanism: action normalization, which determines the underlying coordinate system in which policies perceive and execute physical actions. To bridge this gap, we systematically evaluate five normalization strategies across four real-world task streams covering single-arm and bimanual manipulation. Our analysis reveals that existing protocols induce severe failure modes due to inter-task coordinate drift, limited motion coverage, or train-test coordinate mismatches. Motivated by these insights, we formulate three core design principles: consistency, coverage, and causality (3C), and introduce foresight action normalization (FAN). FAN estimates normalization statistics once from a small, task-independent calibration set prior to continual learning and freezes them throughout adaptation. Across all evaluated streams, FAN achieves the highest performance and demonstrates consistent robustness, providing insightful guidance for building stable action representations in achieving effective lifelong VLA adaptation.
Figures & tables
Fig. 2: Robot platform and multi-view visual observations used during both data collection and policy deployment.
Fig. 3: Action normalization strategies during continual fine-tuning and inference. Unlike the four task-derived baselines, FAN estimates q(C) solely from a task-independent calibration set and keeps it fixed throughout the stream.
(a) Average Score (AS, ↑ )
Method
SA-1
SA-2
BI-1
BI-2
Avg. ↑
ANS-I
74.2
78.5
53.0
42.3
62.0
ANS-II
97.2
45.7
97.2
68.0
77.0
ANS-III
51.3
22.0
17.3
13.0
25.9
ANS-IV
73.7
73.0
56.8
32.5
59.0
FAN (ours)
98.4
94.4
95.7
92.7
95.3
TABLE I: Stream-level and cross-stream performance under different action-normalization strategies.
Fig. 4: Task-order sensitivity after matching tasks by identity. (a)–(b) Signed final-score differences Δ for the single-arm and bimanual tasks, computed as the score in Stream 2 minus that in Stream 1. Δ>0 and Δ<0 indicate higher final performance in Stream 2 and Stream 1, respectively. (c) Mean absolute difference across all ten matched tasks.
Fig. 5: Coverage ablation with fixed normalization statistics on SA-2 and BI-2. (a) Per-dimension raw-action intervals [q0.01,d(D),q0.99,d(D)] for D∈{D1,C,Cexp} . For q(D1) on SA-2, only the seven active right-arm dimensions are shown. (b)–(c) AS and FWT for the three fixed normalization ranges. Across the evaluated streams, the q(C) condition outperforms the narrower and broader alternatives in both overall performance and subsequent-task acquisition.
Fig. 6: Representative rollout keyframes obtained with the final FAN checkpoint show physical robot execution on selected single-arm and bimanual tasks.
Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across diverse lifelong RL benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability-plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era. Code is available at https://github.com/UT-Austin-RobIn/continual-vla-rl
In real-world robotic manipulation, states typically admit a neighborhood of near-equivalent actions. That is, for each state, there exist a feasible action neighborhood (FAN) rather than a single correct action, within which motions yield indistinguishable progress. However, prevalent VLA training methodologies are directly inherited from linguistic settings and do not exploit the FAN property, thus leading to poor generalization and low sample efficiency. To address this limitation, we introduce a FAN-guided regularizer that shapes the model's output distribution to align with the geometry of FAN. Concretely, we introduce a Gaussian prior that promotes locally smooth and unimodal predictions around the preferred direction and magnitude. In extensive experiments across both reinforced finetuning (RFT) and supervised finetuning (SFT), our method achieves significant improvement in sample efficiency, and success rate in both in-distribution and out-of-distribution (OOD) scenarios. By aligning with the intrinsic action tolerance of physical manipulation, FAN-guided regularization provides a principled and practical method for sample-efficient, and generalizable VLA adaptation.
Haochen Niu, Kanyu Zhang, Shuyu Yin +3
Shanghai Jiao Tong University, China · Huawei Technologies, China
Similar to the natural capabilities of humans to sequentially learn new tasks, robots with Vision-Language-Action (VLA) models should possess lifelong learning ability to learn a new task when deployed in open-world environments. However, most recently proposed lifelong learning models aim to effectively learn the current task (plasticity) or maintain high accuracy on previous tasks (stability), while the plasticity-stability trade-off remains largely unsolved in robotic manipulation models. To address this fundamental challenge, we propose a cache-efficient lifelong Vision-Language-Action learning framework for robotic manipulation (i.e., LifelongVLA), which alleviates the plasticity-stability trade-off with a dual-timescale adaptation mechanism while achieving low-cost robotic deployment with a cache-efficient replay strategy. More concretely, we propose a dual-timescale LoRA gating module to decompose VLA adaptation into two lightweight pathways: a short-term adapter for plasticity and a long-term adapter for stable consolidation. These pathways are integrated via a task-aware gate, enabling explicit control of the plasticity-stability trade-off. In the skill replay phase, a cache-efficient stochastic replay strategy is proposed to preserve more balanced retention signals without full-trajectory storage. Finally, experiments show that LifelongVLA outperforms existing baselines, demonstrating efficient skill expansion, robust retention of learned manipulation behaviors, and reduced reliance on retraining for real-world deployment on an xArm robot.
Yao He, Gan Sun, Wenqi Liang +2
South China University of Technology · University of Trento