Organizations: School of Computer Science and Technology, University of Science and Technology of China · Shanghai AI Laboratory · College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · School of Artificial Intelligence and Data Science, University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou) · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology
Vision-Language-Action Models (VLAs) enable robots to perform manipulation tasks by leveraging Vision-Language Models (VLMs) to process visual observations and language instructions. However, the action prediction objective itself lacks fine-grained task-specific perceptual supervision, making it prone to relying on task-irrelevant visual shortcuts, leading attention away from target objects and key interaction areas, thereby limiting its operational performance and generalization ability. To address this, we propose Automatic Perception-Guided Attention Alignment for Robot Manipulation (A3-VLA), a framework that derives task-specific visual guidance from the manipulation process itself. Specifically, A3-VLA first generates heuristic candidate regions for target objects and the robot from interaction-induced visual changes, and refines them with a visual segmentation model offline. These regions serve as explicit perceptual guidance through two complementary objectives: an attention allocation loss that encourages sufficient focus on task-critical regions when language or robot state tokens query the visual input, and a contrastive learning loss that aligns the representations of candidate target regions with language instructions while separating them from task-irrelevant regions. Extensive experiments on simulated and real-world manipulation tasks demonstrate that A3-VLA consistently improves multiple VLA backbones, with an average success rate gain of 6.5% - 25.2% across benchmarks.
Figures & tables
Figure 1: A failure case of the attention maps of language instruction querying visual observations in π0 and OpenVLA-OFT on a Select Poker task from VLABench, visualized across layers from shallow (left) to deep (right).
Figure 2: Overview of the A3 -VLA framework.
Method
Chemistry
Fruit
Poker
Toy
Mahjong
Avg.
Octo Team et al. (2024)
0.02
0.00
0.00
0.02
0.04
0.02
OpenVLA Kim et al. (2025b)
0.08
0.06
0.04
0.10
0.06
0.07
RDT Liu et al. (2024)
0.12
0.16
0.08
0.10
0.06
0.10
π0 -FAST Pertsch et al. (2025)
0.20
0.34
0.18
0.22
0.10
0.21
π0 Black et al. (2024)
0.52
0.60
0.54
0.40
0.30
0.47
A3 -VLA ( π0 )
0.82
0.66
0.82
0.46
0.42
0.64
Table 1: Success Rate of VLAs on VLABench. The scores of the best baselines are underlined, while the scores of A3 -VLA are bold.
Method
Block Hammer
Click Bell
Playing Card
Burger & Fries
Avg.
OpenVLA
0.00
0.09
0.14
0.00
0.06
RDT
0.17
0.44
0.30
0.08
0.23
π0
0.29
0.56
0.70
0.12
0.42
A3 -VLA ( π0 )
0.36
0.65
0.79
0.26
0.52
OpenVLA-OFT
0.06
0.26
0.39
0.10
0.20
A3 -VLA (OpenVLA-OFT)
0.08
0.37
0.46
0.18
0.27
Table 2: Success Rate of VLAs on RoboTwin 2.0. The scores of the best baselines are underlined, while the scores of A3 -VLA are bold.
Open Cooker
Pick & Place Bowl
Pick 2 Vegetables
Select Bottles
Desktop Sweep
Avg.
Method
SR
TP
SR
TP
SR
TP
SR
TP
SR
TP
SR
TP
π0 Black et al. (2024)
0.85
0.93
0.75
0.88
0.40
0.58
0.25
0.30
0.50
0.65
0.55
0.67
π0.5 Intelligence et al. (2025)
0.90
0.95
0.65
0.83
0.45
0.60
0.40
0.40
0.70
0.85
0.62
0.73
A3 -VLA ( π0 )
0.90
0.95
0.80
0.90
0.55
0.78
0.75
0.80
0.80
0.90
0.76
0.87
Table 3: Success Rate (SR) and Task Progress (TP) of VLAs on Real-world Settings. Bold scores represent the highest results among all the methods, while the second highest scores are underlined.
Table 6
Figure 3: The comparison of LFM and progressive SR of π0 and A3 -VLA ( π0 ) on VLABench.
Figure 4: The visualization of the attention map Ml2i for language querying the image when testing π0 , π0.5 and A3 -VLA ( π0 ) on different tasks and benchmarks.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Performance comparison on VLABench of A3 -VLA w.r.t different hyper-parameters.
Figure 6: The manipulation process of A3 -VLA ( π0.5 ) and π0.5 under perturbations on VLABench.
Figure 7: The visualization of the attention map Ml2i for language querying the image when testing π0 , π0.5 and A3 -VLA ( π0 ) on different tasks and benchmarks.
Figure 8: An Example of heuristic region proposals on VLABench.
Figure 9: An Example of heuristic region proposals on Robotwin.
Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for robotic manipulation, in which reliable action prediction critically depends on accurately interpreting and integrating visual observations conditioned on language instructions. Although recent works have sought to enhance the visual capabilities of VLA models, most approaches treat the LLM backbone as a black box, providing limited insight into how visual information is grounded into action generation. Therefore, we perform a systematic analysis of multiple VLA models across different action-generation paradigms and observe that sensitivity to visual tokens progressively decreases in deeper layers during action generation. Motivated by this observation, we propose DeepVision-VLA, built on a Vision-Language Mixture-of-Transformers (VL-MoT) framework. This framework enables shared attention between the vision foundation model and the VLA backbone, injecting multi-level visual features from the vision expert into deeper layers of the VLA backbone to enhance visual representations for precise and complex manipulation. In addition, we introduce Action-Guided Visual Pruning (AGVP), which leverages shallow-layer attention to prune irrelevant visual tokens while preserving task-relevant ones, reinforcing critical visual cues for manipulation with minimal computational overhead. DeepVision-VLA outperforms prior state-of-the-art methods by 9.0% and 7.5% on simulated and real-world tasks, respectively, providing new insights for the design of visually enhanced VLA models.
Yulin Luo, Hao Chen, Zhuangzhe Wu +10
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics
Recent advances in Vision-Language-Action (VLA) models have shown strong potential for general-purpose robotic manipulation. However, the visual representations of most VLA models are often dominated by global object appearance and struggle to focus on task-relevant functional interaction regions, which limits their robustness in unstructured environments. Existing affordance-based methods typically rely on explicit mask injection or external perception modules, requiring additional annotations while introducing cascading perception errors and inference overhead. To address these limitations, we propose AffordVLA, an affordance-enhanced VLA framework that internalizes manipulation-centric affordance perception into VLA visual representations through implicit representation alignment. Specifically, we construct a zero-shot affordance teacher to extract task-conditioned affordance visual representations from RGB observations and language instructions. AffordVLA aligns the intermediate visual representations of the VLA with the affordance visual representations extracted by the teacher, thereby implicitly injecting manipulation-centric affordance perception into VLA visual representations and improving action accuracy. Extensive simulation and real-world experiments demonstrate that AffordVLA and its affordance teacher achieve state-of-the-art performance and outperform strong baselines. Ablation analyses show that AffordVLA effectively reshapes VLA visual representations while preserving inference efficiency, leading to improved manipulation success rates and training efficiency.
Weijie Kong, Zhian Su, Wei Yu +1
Grasp Lab, School of Mechanical Engineering of Zhejiang University, Hangzhou 310027, China · Torch Kernel Co., Ltd., Hangzhou, 310000, P. R. China
Despite advances in Vision-Language-Action (VLA) models, robotic manipulation struggles with fine-grained tasks because current models lack mechanisms for active visual attention allocation. Human gaze naturally encodes intent, planning, and execution patterns -- offering a powerful supervisory signal for guiding robot perception. We introduce a gaze-regularized training framework that aligns VLA models' internal attention with human visual patterns without architectural modifications or inference-time overhead. Our method transforms temporally aggregated gaze heatmaps into patch-level distributions and regularizes the transformer's attention through KL divergence, creating an inductive bias toward task-relevant features while preserving deployment efficiency. When integrated into existing VLA architectures, our approach yields 4-12% improvements across manipulation benchmarks. The gaze-regularized models reach equivalent performance with fewer training steps and maintain robustness under lighting variations and sensor noise. Beyond performance metrics, the learned attention patterns produce interpretable visualizations that mirror human strategies, enhancing trust in robotic systems. Moreover, our framework requires no eye-tracking equipment and applies directly to existing datasets. These results demonstrate that human perceptual priors can significantly accelerate robot learning while improving both task performance and system interpretability.
Anupam Pani, Yanchao Yang
Institute of Data Science, University Of Hong Kong