cs.AINov 13, 2026

Bring Back Language to Actions: Restoring VLM capabilities in VLA Models via Parameter Merging

Authors: Mohamed Salim Aissi, Olivier Sigaud, Laure Soulier, Nicolas Thome

Organizations: Sorbonne Université, CNRS, ISIR, F-75005 Paris, France · Institut universitaire de France (IUF) · International Laboratory on Learning Systems (ILLS)

Abstract

Vision-Language-Action models (VLAs) are obtained by fine-tuning pre-trained Vision-Language Models (VLMs) on robot demonstration data, enabling action prediction from visual observations and language instructions. However, this fine-tuning induces catastrophic forgetting, substantially degrading the semantic and visual grounding capabilities of the original VLM. We address this by merging the parameters of a VLM and its VLA derivative without retraining either model. Concretely, we extract the task vector (the difference between VLA and VLM weights) and decompose it via Singular Value Decomposition. For each component, we learn a scalar mixing coefficient controlling how much of the VLA adaptation is retained, optimised to jointly preserve action prediction and vision-language capabilities. Experiments show that our merged model performs well on both axes simultaneously, consistently outperforming established merging baselines.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?

    Jun 26, 2026Guoheng Sun, Kaixi Feng, Shwai He +8Model CompressionEfficient VLA Models

  2. Rethinking VLM Representation for VLA Initialization

    May 25, 2026Weifeng Lin, Siyuan Huang, Hao Li +5VLM AdaptationLow-Rank Adaptation