$A^3$-VLA: Automatic Perception-Guided Attention Alignment for Robot Manipulation

Shengzhe Zhang; Qi Zhang; Dazhong Shen; Chao Wang; Shaopeng Zhai; Hui Xiong

cs.RONov 4, 2026

$A^3$ -VLA: Automatic Perception-Guided Attention Alignment for Robot Manipulation

Authors: Shengzhe Zhang, Qi Zhang, Dazhong Shen, Chao Wang, Shaopeng Zhai, Hui Xiong

Organizations: School of Computer Science and Technology, University of Science and Technology of China · Shanghai AI Laboratory · College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · School of Artificial Intelligence and Data Science, University of Science and Technology of China ·

Abstract

Vision-Language-Action Models (VLAs) enable robots to perform manipulation tasks by leveraging Vision-Language Models (VLMs) to process visual observations and language instructions. However, the action prediction objective itself lacks fine-grained task-specific perceptual supervision, making it prone to relying on task-irrelevant visual shortcuts, leading attention away from target objects and key interaction areas, thereby limiting its operational performance and generalization ability. To address this, we propose Automatic Perception-Guided Attention Alignment for Robot Manipulation ( $A^3$ -VLA), a framework that derives task-specific visual guidance from the manipulation process itself. Specifically, $A^3$ -VLA first generates heuristic candidate regions for target objects and the robot from interaction-induced visual changes, and refines them with a visual segmentation model offline. These regions serve as explicit perceptual guidance through two complementary objectives: an attention allocation loss that encourages sufficient focus on task-critical regions when language or robot state tokens query the visual input, and a contrastive learning loss that aligns the representations of candidate target regions with language instructions while separating them from task-irrelevant regions. Extensive experiments on simulated and real-world manipulation tasks demonstrate that $A^3$ -VLA consistently improves multiple VLA backbones, with an average success rate gain of 6.5% - 25.2% across benchmarks.

Figures & tables

Figure 1: A failure case of the attention maps of language instruction querying visual observations in $\pi_{0}$ and OpenVLA-OFT on a Select Poker task from VLABench, visualized across layers from shallow (left) to deep (right).

Figure 2: Overview of the $\text{A}^{3}$ -VLA framework.

Method	Chemistry	Fruit	Poker	Toy	Mahjong	Avg.
Octo Team et al. (2024)	0.02	0.00	0.00	0.02	0.04	0.02
OpenVLA Kim et al. (2025b)	0.08	0.06	0.04	0.10	0.06	0.07
RDT Liu et al. (2024)	0.12	0.16	0.08	0.10	0.06	0.10
$\pi_{0}$ -FAST Pertsch et al. (2025)	0.20	0.34	0.18	0.22	0.10	0.21
$\pi_{0}$ Black et al. (2024)	0.52	0.60	0.54	0.40	0.30	0.47
$\text{A}^{3}$ -VLA ( $\pi_{0}$ )	0.82	0.66	0.82	0.46	0.42	0.64

Table 1: Success Rate of VLAs on VLABench. The scores of the best baselines are underlined, while the scores of

\text{A}^{3}

-VLA are bold.

Method	Block Hammer	Click Bell	Playing Card	Burger & Fries	Avg.
OpenVLA	0.00	0.09	0.14	0.00	0.06
RDT	0.17	0.44	0.30	0.08	0.23
$\pi_{0}$	0.29	0.56	0.70	0.12	0.42
$\text{A}^{3}$ -VLA ( $\pi_{0}$ )	0.36	0.65	0.79	0.26	0.52
OpenVLA-OFT	0.06	0.26	0.39	0.10	0.20
$\text{A}^{3}$ -VLA (OpenVLA-OFT)	0.08	0.37	0.46	0.18	0.27

Table 2: Success Rate of VLAs on RoboTwin 2.0. The scores of the best baselines are underlined, while the scores of

\text{A}^{3}

-VLA are bold.

	Open Cooker		Pick & Place Bowl		Pick 2 Vegetables		Select Bottles		Desktop Sweep		Avg.
Method	SR	TP	SR	TP	SR	TP	SR	TP	SR	TP	SR	TP
$\pi_{0}$ Black et al. (2024)	0.85	0.93	0.75	0.88	0.40	0.58	0.25	0.30	0.50	0.65	0.55	0.67
$\pi_{0.5}$ Intelligence et al. (2025)	0.90	0.95	0.65	0.83	0.45	0.60	0.40	0.40	0.70	0.85	0.62	0.73
$\text{A}^{3}$ -VLA ( $\pi_{0}$ )	0.90	0.95	0.80	0.90	0.55	0.78	0.75	0.80	0.80	0.90	0.76	0.87

Table 3: Success Rate (SR) and Task Progress (TP) of VLAs on Real-world Settings. Bold scores represent the highest results among all the methods, while the second highest scores are underlined.

Table 6

Figure 3: The comparison of

L_{FM}

and progressive SR of

\pi_{0}

and

\text{A}^{3}

-VLA (

\pi_{0}

) on VLABench.

Figure 4: The visualization of the attention map $\overline{M}^{l2i}$ for language querying the image when testing $\pi_{0}$ , $\pi_{0.5}$ and $\text{A}^{3}$ -VLA ( $\pi_{0}$ ) on different tasks and benchmarks.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Performance comparison on VLABench of

\text{A}^{3}

-VLA w.r.t different hyper-parameters.

Figure 6: The manipulation process of $\text{A}^{3}$ -VLA ( $\pi_{0.5}$ ) and $\pi_{0.5}$ under perturbations on VLABench.

Figure 7: The visualization of the attention map $\overline{M}^{l2i}$ for language querying the image when testing $\pi_{0}$ , $\pi_{0.5}$ and $\text{A}^{3}$ -VLA ( $\pi_{0}$ ) on different tasks and benchmarks.

Figure 8: An Example of heuristic region proposals on VLABench.

Figure 9: An Example of heuristic region proposals on Robotwin.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,216 papers

Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models
Mar 16, 2026Yulin Luo, Hao Chen, Zhuangzhe Wu +10Spatial Grounding for VLA Models Vision-Language-Action Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics
AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignment
May 17, 2026Weijie Kong, Zhian Su, Wei Yu +1Language-Conditioned Robot Manipulation Vision-Language-Action Models
Grasp Lab, School of Mechanical Engineering of Zhejiang University, Hangzhou 310027, China · Torch Kernel Co., Ltd., Hangzhou, 310000, P. R. China
Gaze-Regularized Vision-Language-Action Models for Robotic Manipulation
Mar 24, 2026Anupam Pani, Yanchao YangVLMs for Robotics Robotic Manipulation
Institute of Data Science, University Of Hong Kong

Mar 16, 2026J/K move · Enter open · S save

Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models

Yulin Luo, Hao Chen, Zhuangzhe Wu, Bowen Sui+9

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics

Spatial Grounding for VLA Models Vision-Language-Action Models Vision-Language Models Robotic Manipulation Visual Token Pruning

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for robotic manipulation, in which reliable action prediction critically depends on accurately interpreting and integrating visual observations conditioned on language instructions. Although recent works have sought to enhance the visual capabilities of VLA models, most approaches treat the LLM backbone as a black box, providing limited insight into how visual information is grounded into action generation. Therefore, we perform a systematic analysis of multiple VLA models across different action-generation paradigms and observe that sensitivity to visual tokens progressively decreases in deeper layers during action generation. Motivated by this observation, we propose DeepVision-VLA, built on a Vision-Language Mixture-of-Transformers (VL-MoT) framework. This framework enables shared attention between the vision foundation model and the VLA backbone, injecting multi-level visual features from the vision expert into deeper layers of the VLA backbone to enhance visual representations for precise and complex manipulation. In addition, we introduce Action-Guided Visual Pruning (AGVP), which leverages shallow-layer attention to prune irrelevant visual tokens while preserving task-relevant ones, reinforcing critical visual cues for manipulation with minimal computational overhead. DeepVision-VLA outperforms prior state-of-the-art methods by 9.0% and 7.5% on simulated and real-world tasks, respectively, providing new insights for the design of visually enhanced VLA models.

Abstract

Vision-Language-Action Models

Explore similar work

Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models

AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignment

Gaze-Regularized Vision-Language-Action Models for Robotic Manipulation