cs.RONov 7, 2026

Spatial Representation Distillation and Knowledge Routing for Vision-Language-Action Models

Authors: Jungin Park, Chaoran Zhu, Changjae Oh, Kwanghoon Sohn

Organizations: DGIST, South Korea. · Yonsei University, South Korea. · Queen Mary University of London, UK.

Abstract

Understanding three-dimensional physical structure has become essential for vision-language-action models (VLAs). However, existing approaches either rely on explicit geometric inputs at inference time or adopt a single visual pathway, potentially compromising the visual-semantic knowledge inherited from pre-trained models. We present SPARK-VLA, Spatial Representation distillation and Knowledge routing for VLAs, a preserve-specialize-route framework for spatially adapting VLAs without requiring external 3D inputs or experts at inference. SPARK-VLA consists of two key components: (1) spatial representation distillation, which transfers 3D understanding from a frozen 3D expert to a LoRA-adapted visual branch by aligning attention responses while preserving the original visual pathway, and (2) a lightweight knowledge router, which selectively forwards action-relevant visual tokens to the language model using routing supervision derived from action-loss gradients. Across LIBERO, LIBERO-Plus, CALVIN, and real-world manipulation, SPARK-VLA achieves strong performance with only a 0.5B language backbone and approximately 53M trainable parameters, providing an effective and parameter-efficient approach to 3D-aware VLA learning. Code is available at https://github.com/park-jungin/spark-vla.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models

    Aug 5, 2026Xingyu Ding, Yuzhong Zhao, Yang Wu +4Robotic ManipulationVision-Language-Action Models

  2. 3DVLA: Enhancing Vision-Language-Action Models via 3D Spatial and Instance Understanding

    May 28, 2026Zhongyu Xia, Yousen Tang, Bingqing Wei +13D Spatial ReasoningRobotic Manipulation

  3. 3DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training

    Jun 3, 2026Jiaxin Shi, Xidong Zhang, Fucai Zhu +33D Spatial ReasoningRobotic Manipulation