cs.ROApr 7, 2026

Grounding Hierarchical Vision-Language-Action Models Through Explicit Language-Action Alignment

Authors: Theodor Wulff, Federico Tavella, Rahul Singh Maharjan, Manith Adikari, Angelo Cangelosi

Organizations: The University of Manchester, Manchester, United Kingdom

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Achieving robot transparency is a critical step toward effective human-robot collaboration. To be transparent, a robot's natural language communication must be consistent with its actions and explicitly grounded in the task and environment. Existing hierarchical Vision-Language-Action (VLA) models can generate language (e.g., through chain-of-thought) and low-level actions. However, current work does not consider explicit alignment between these modalities during training. To address this crucial gap, we propose a novel training framework that explicitly grounds hierarchical VLA sub-task descriptions with respect to the visual observation and action space. Our framework uses a contrastive model to assess the alignment between generated language and corresponding action trajectories. This contrastive model enables direct ranking of different language-trajectory pairs based on their alignment, allowing us to refine the grounding of our hierarchical VLA through offline preference learning. We apply our framework to the LanguageTable dataset, a benchmark dataset of human language-annotated trajectories, and provide critical insights into multimodal grounding representations, all while establishing a strong baseline that achieves performance comparable to fully supervised fine-tuning and minimizing the need for costly data annotations.

Figures & tables

Figure 1: Method Overview. We extend a regular VLA (left) into a hierarchical VLA by adding a high-level VLM module to break a high-level instruction down into executable low-level instructions (center), following recent trends on hierarchical VLAs [ 45 , 4 ] . To align the intermediate low-level instruction and the generated trajectory, we invoke a separately trained ranking model, which ranks NN sampled output pairs based on their grounding in the environment. Based on these scores, we select the chosen and rejected output pairs that serve as preference data to update our transparent VLA and increase the alignment of the multimodal outputs.
Figure 2: Action-Conditioned Grounding Model. We extend a pre-trained SigLIP 2 by conditioning the visual features of the SigLIP 2 Vision Encoder on the encoded trajectories. Using a contrastive loss, we align the vision-action pairs with the low-level instructions.
ModelBLEU ↑\uparrowROUGE ↑\uparrowMETEOR ↑\uparrowBERTScore ↑\uparrowMSE ↓\downarrowMAE ↓\downarrowCosSim ↓\downarrow
Low-Level OnlyN/AN/AN/AN/A0.043 ±\pm 0.020.158 ±\pm 0.04-0.029 ±\pm 0.22
Supervised0.111 ±\pm 0.050.405 ±\pm 0.120.313 ±\pm 0.120.984 ±\pm 0.000.046 ±\pm 0.020.164 ±\pm 0.04-0.044 ±\pm 0.23
GPLA (CLIP)0.062 ±\pm 0.040.298 ±\pm 0.120.217 ±\pm 0.120.976 ±\pm 0.000.045 ±\pm 0.020.164 ±\pm 0.04-0.036 ±\pm 0.22
GPLA (SigLIP 2)0.066 ±\pm 0.060.307 ±\pm 0.130.227 ±\pm 0.120.976 ±\pm 0.000.045 ±\pm 0.020.163 ±\pm 0.04-0.035 ±\pm 0.22
GPLA (Action-Conditioned)0.063 ±\pm 0.050.300 ±\pm 0.120.218 ±\pm 0.120.980 ±\pm 0.000.045 ±\pm 0.020.163 ±\pm 0.04-0.035 ±\pm 0.22
Supervised + GPLA (Action-Conditioned)0.051 ±\pm 0.050.308 ±\pm 0.120.226 ±\pm 0.120.980 ±\pm 0.000.046 ±\pm 0.020.163 ±\pm 0.04-0.042 ±\pm 0.23
Table 1: Comparison of model variants on the language- and trajectory-based metrics.
Table 2: Qualitative Examples on the LanguageTable Dataset [ 25 ] .
Figure 3: Quantitative Evaluation of Generated Low-level Instructions.
Figure 4: Quantitative Evaluation of Generated Trajectories.
Figure 5: t-SNE visualizations of all grounding models on the LanguageTable dataset using visual inputs and low-level instructions.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

ModalityAugmentationProbability
ImagesBrightness0.5
Contrast0.5
Saturation0.5
Crop and resize0.6
Vertical translation0.4
Horizontal translation0.4
Appendix
Table 3: Data augmentation techniques and their application probabilities, grouped by modality.
General
Gradient norm clipping:1.0
Action cut-off threshold:0.1
High-level VLM (fine-tuning)
Steps:1,500
Learning rate:10−510^{-5}
Effective Batch size:64
Appendix
Table 4: Hyperparameters grouped by model variant.

Explore similar work

CardsList

  1. H-VLA: Hierarchical Vision-Language-Action Model with Key-Action Reasoning and Motion Planning in a Unified Action Space

    Sep 19, 2026Xiongfeng Peng, Lu Xu, Yandong Wang +10Vision-Language-Action ModelsLanguage-Conditioned Robot Manipulation

    Advanced Research Lab, Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea

  2. RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

    Mar 22, 2026Dongyoung Kim, Sumin Park, Woomin Song +6Vision-Language-Action ModelsReasoning for VLA Models

    KAIST · RLWRLD · Yonsei University +1

  3. In-Context VLA: Endowing Vision-Language-Action Models with Language via In-Context Post-Training and Agentic Tool Use

    Aug 6, 2026Jiarui Yang, Wen Huang, Jiale Zhang +2ICL for RoboticsContinual Learning for VLA Models

    Nankai University · Tsinghua University · Swiss Federal Institute of Technology in Lausanne (EPFL)