cs.AIMar 22, 2026

RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

Authors: Dongyoung Kim, Sumin Park, Woomin Song, Seungku Kim, Taeyoung Kim, Huiwon Jang, Jinwoo Shin, Jaehyung Kim, +1 more

Organizations: KAIST · RLWRLD · Yonsei University · UC Berkeley

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Improving embodied reasoning in multimodal-large-language models (MLLMs) is essential for building vision-language-action models (VLAs) on top of them to readily translate multimodal understanding into low-level actions. Accordingly, recent work has explored enhancing embodied reasoning in MLLMs through supervision of vision-question-answering type. However, these approaches have been reported to result in unstable VLA performance, often yielding only marginal or even negative gains. In this paper, we propose a more systematic MLLM training framework RoboAlign that reliably improves VLA performance. Our key idea is to sample action tokens via zero-shot natural language reasoning and refines this reasoning using reinforcement learning (RL) to improve action accuracy. As a result, RoboAlign bridges the modality gap between language and low-level actions in MLLMs, and facilitate knowledge transfer from MLLM to VLA. To validate the effectiveness of RoboAlign, we train VLAs by adding a diffusion-based action head on top of an MLLM backbone and evaluate them on major robotics benchmarks. Remarkably, by performing RL-based alignment after SFT using less than 1% of the data, RoboAlign achieves performance improvements of 17.5%, 18.9%, and 106.6% over SFT baselines on LIBERO, CALVIN, and real-world environments, respectively.

Figures & tables

Figure 1: Performance on LIBERO. VLAs built upon MLLMs specialized for embodied reasoning (fine-tuned variants of Qwen2.5-VL-7B-Instruct) fail to significantly improve performance and often degrade it compared to the baseline VLA based on the original model. In contrast, RoboAlign achieves significant gains, as detailed in Section 5 .
Figure 2: Overview of RoboAlign framework. RoboAlign directly aligns MLLM representations with low-level action generation using reasoning-incentivized reinforcement learning ( Guo et al., 2025 ) . The framework consists of two stages: (i) Stage 1 integrates embodied reasoning, zero-shot reasoning, and FAST-tokenized low-level action generation via supervised fine-tuning, and (ii) Stage 2 optimizes responses through reinforcement learning to improve token-level action accuracy and better alignment. The resulting model serves as an MLLM tailored for effective VLA training.
Table 1: Example of RoboAlign response. Incorporating reasoning data during SFT effectively transfers zero-shot reasoning ability to FAST token generation process. Without such data, the model produces only minimal reasoning which reduces the diversity of reasoning samples and hinders RL.
Figure 3: Examples of observations. Visual inputs for training and evaluation (from left to right): BridgeV2 for FAST token training, CALVIN, LIBERO benchmark, and the real-robot.
Figure 4: Summary of VLA performance. Comparison of VLA performance across different MLLM training methods on LIBERO and CALVIN. RoboAlign achieves the highest gains.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Prompt for FAST token generation. We use this prompt template for both FAST token prediction and reinforcement learning.
Figure 6: Training metrics for reinforcement learning in RoboAlign . Validation performance during RL training: (a) accuracy reward increases steadily, while (b) response length remains stable.
Question: Your current task is move the orange cloth to the top of the right burner. Output the robot’s actions to perform this task through FAST tokens.
GT Answer:
<|action_start|><|action_486|><|action_265|><|action_268|><|action_116|><|action_269|><|action_end|>
RoboAlign (SFT) Answer:
’<think>To perform the task of moving the orange cloth to the top of the right burner, the robot needs to first approach the cloth, then grasp it, and finally move it to the right burner. The actions should be sequential and purposeful, focusing on the cloth and the burner in question.</think><answer>
<|action_start|><|action_266|><|action_709|><|action_268|><|action_116|><|action_269|><|action_end|>
Reward: 0.5 (Format 1.0 / Accuracy 0.0)
Appendix
Figure 7: Example of model response.

Explore similar work

CardsList

  1. ReFineVLA: Multimodal Reasoning-Aware Generalist Robotic Policies via Teacher-Guided Fine-Tuning

    Apr 20, 2026Tuan Van Vo, Tan Q. Nguyen, Khang Nguyen +5Robotic ManipulationVLM Reasoning

    VinRobotics, Hanoi. Vietnam · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE. · University of Texas at Arlington, Texas, USA +4

  2. VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoning

    Mar 15, 2026Chaoyang Wang, Wenrui Bao, Sicheng Gao +5Vision-Language-Action ModelsReasoning for VLA Models

    University of Central Florida · University of Würzburg · University of Southern California +1

  3. Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

    Jul 15, 2026Dwip Dalal, Shivansh Patel, Chahit Jain +8Vision-Language-Action ModelsVision-Language Alignment

    University of Illinois Urbana-Champaign · Texas A&M University · University of California, Irvine