cs.AIMar 22, 2026

RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

Authors: Dongyoung Kim, Sumin Park, Woomin Song, Seungku Kim, Taeyoung Kim, Huiwon Jang, Jinwoo Shin, Jaehyung Kim, +1 more

Organizations: KAIST · RLWRLD · Yonsei University · UC Berkeley

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Improving embodied reasoning in multimodal-large-language models (MLLMs) is essential for building vision-language-action models (VLAs) on top of them to readily translate multimodal understanding into low-level actions. Accordingly, recent work has explored enhancing embodied reasoning in MLLMs through supervision of vision-question-answering type. However, these approaches have been reported to result in unstable VLA performance, often yielding only marginal or even negative gains. In this paper, we propose a more systematic MLLM training framework RoboAlign that reliably improves VLA performance. Our key idea is to sample action tokens via zero-shot natural language reasoning and refines this reasoning using reinforcement learning (RL) to improve action accuracy. As a result, RoboAlign bridges the modality gap between language and low-level actions in MLLMs, and facilitate knowledge transfer from MLLM to VLA. To validate the effectiveness of RoboAlign, we train VLAs by adding a diffusion-based action head on top of an MLLM backbone and evaluate them on major robotics benchmarks. Remarkably, by performing RL-based alignment after SFT using less than 1% of the data, RoboAlign achieves performance improvements of 17.5%, 18.9%, and 106.6% over SFT baselines on LIBERO, CALVIN, and real-world environments, respectively.

Figures & tables

Figure 1: Performance on LIBERO. VLAs built upon MLLMs specialized for embodied reasoning (fine-tuned variants of Qwen2.5-VL-7B-Instruct) fail to significantly improve performance and often degrade it compared to the baseline VLA based on the original model. In contrast, RoboAlign achieves significant gains, as detailed in Section 5 .
Figure 2: Overview of RoboAlign framework. RoboAlign directly aligns MLLM representations with low-level action generation using reasoning-incentivized reinforcement learning ( Guo et al., 2025 ) . The framework consists of two stages: (i) Stage 1 integrates embodied reasoning, zero-shot reasoning, and FAST-tokenized low-level action generation via supervised fine-tuning, and (ii) Stage 2 optimizes responses through reinforcement learning to improve token-level action accuracy and better alignment. The resulting model serves as an MLLM tailored for effective VLA training.
Table 1: Example of RoboAlign response. Incorporating reasoning data during SFT effectively transfers zero-shot reasoning ability to FAST token generation process. Without such data, the model produces only minimal reasoning which reduces the diversity of reasoning samples and hinders RL.
Figure 3: Examples of observations. Visual inputs for training and evaluation (from left to right): BridgeV2 for FAST token training, CALVIN, LIBERO benchmark, and the real-robot.
Figure 4: Summary of VLA performance. Comparison of VLA performance across different MLLM training methods on LIBERO and CALVIN. RoboAlign achieves the highest gains.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Prompt for FAST token generation. We use this prompt template for both FAST token prediction and reinforcement learning.
Figure 6: Training metrics for reinforcement learning in RoboAlign . Validation performance during RL training: (a) accuracy reward increases steadily, while (b) response length remains stable.
Question: Your current task is move the orange cloth to the top of the right burner. Output the robot’s actions to perform this task through FAST tokens.
GT Answer:
<|action_start|><|action_486|><|action_265|><|action_268|><|action_116|><|action_269|><|action_end|>
RoboAlign (SFT) Answer:
’<think>To perform the task of moving the orange cloth to the top of the right burner, the robot needs to first approach the cloth, then grasp it, and finally move it to the right burner. The actions should be sequential and purposeful, focusing on the cloth and the burner in question.</think><answer>
<|action_start|><|action_266|><|action_709|><|action_268|><|action_116|><|action_269|><|action_end|>
Reward: 0.5 (Format 1.0 / Accuracy 0.0)
Appendix
Figure 7: Example of model response.

Explore similar work

CardsList
Apr 20, 2026

ReFineVLA: Multimodal Reasoning-Aware Generalist Robotic Policies via Teacher-Guided Fine-Tuning

Vision-Language-Action (VLA) models have gained much attention from the research community thanks to their strength in translating multimodal observations with linguistic instructions into desired robotic actions. Despite their advancements, VLAs often overlook explicit reasoning and learn the functional input-action mappings, omitting crucial logical steps, which are especially pronounced in interpretability and generalization for complex, long-horizon manipulation tasks. In this work, we propose ReFineVLA, a multimodal reasoning-aware framework that fine-tunes VLAs with teacher-guided reasons. We first augment robotic datasets with reasoning rationales generated by an expert teacher model, guiding VLA models to learn to reason about their actions. Then, we fine-tune pre-trained VLAs with the reasoning-enriched datasets with ReFineVLA, while maintaining the underlying generalization abilities and boosting reasoning capabilities. We also conduct attention map visualization to analyze the alignment among visual observation, linguistic prompts, and to-be-executed actions of ReFineVLA, reflecting the model is ability to focus on relevant tasks and actions. Through this additional step, we explore that ReFineVLA-trained models exhibit a meaningful agreement between vision-language and action domains, highlighting the enhanced multimodal understanding and generalization. Evaluated across a suite of simulated manipulation benchmarks on SimplerEnv with both WidowX and Google Robot tasks, ReFineVLA achieves state-of-the-art performance, in success rate over the second-best method on the both the WidowX benchmark and Google Robot Tasks.

Tuan Van Vo, Tan Q. Nguyen, Khang Nguyen +5

VinRobotics, Hanoi. Vietnam · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE. · University of Texas at Arlington, Texas, USA +4

Robotic ManipulationVLM ReasoningVision-Language-Action ModelsReasoning for VLA Models
AskPDF
Mar 15, 2026

VLA-Thinker: Boosting Vision-Language-Action Models through Thinking-with-Image Reasoning

Vision-Language-Action (VLA) models have shown promising capabilities for embodied intelligence, but most existing approaches rely on text-based chain-of-thought reasoning where visual inputs are treated as static context. This limits the ability of the model to actively revisit the environment and resolve ambiguities during long-horizon tasks. We propose VLA-Thinker, a thinking-with-image reasoning framework that models perception as a dynamically invocable reasoning action. To train such a system, we introduce a two-stage training pipeline consisting of (1) an SFT cold-start phase with curated visual Chain-of-Thought data to activate structured reasoning and tool-use behaviors, and (2) GRPO-based reinforcement learning to align complete reasoning-action trajectories with task-level success. Extensive experiments on LIBERO and RoboTwin 2.0 benchmarks demonstrate that VLA-Thinker significantly improves manipulation performance, achieving 97.5% success rate on LIBERO and strong gains across long-horizon robotic tasks. Project and Codes: https://cywang735.github.io/VLA-Thinker/ .

Chaoyang Wang, Wenrui Bao, Sicheng Gao +5

University of Central Florida · University of Würzburg · University of Southern California +1

Vision-Language-Action ModelsReasoning for VLA ModelsVisual CoTRobotic Manipulation
AskPDF
Jul 15, 2026

Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. However, BC finetuning progressively overwrites the pretrained representations that support visual and semantic generalization. Co-training on web image-text data, a common remedy, applies language and action losses to separate observations, leaving VLAs with language-action misalignment that standard manipulation benchmarks do not expose. We propose Anchor-Align, which augments BC with two objectives: Vision-Language Anchoring distills layer-wise representations from a frozen VLM copy to prevent this drift, while Language-Action Alignment converts each action target into a discrete motion-direction label and jointly trains language and action prediction on the same robot observation. We conduct real-world evaluations across eight manipulation settings on single-arm xArm7 and bimanual YAM robots, using two VLA architectures with regression and flow-matching action heads. Across these settings, Anchor-Align consistently improves over BC on novel targets, layouts, and motion-sensitive bimanual tasks requiring coordinated control. At scale in simulation, we demonstrate consistent improvements on OOD perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO, LIBERO-Plus, and CALVIN, respectively, suggesting that preserving pretrained representations and effective action learning are not fundamentally at odds. Project page: anchoralignvla.github.io

Dwip Dalal, Shivansh Patel, Chahit Jain +8

University of Illinois Urbana-Champaign · Texas A&M University · University of California, Irvine

Vision-Language-Action ModelsVision-Language AlignmentRobustness of VLA Models
AskPDF