Organizations: Harbin Institute of Technology, Shenzhen, China · DaiMon Robotics, China · Nanjing University, Nanjing, China · Renmin University of China, Beijing, China
Vision-Language-Action (VLA) models improve action generation by conditioning policies on rich vision-language information. However, current auto-regressive policies are constrained by three bottlenecks: (1) architectural bias drives models to overlook visual details, (2) an excessive number of visual tokens makes attention difficult to focus on the correct regions, and (3) task-irrelevant visual information introduces substantial noise - together severely impairing the quality of action. In this paper, we investigate how to effectively utilize different visual representations for action generation. To this end, we first empirically validate the above issues and show that VLA performance is primarily limited by how visual information is utilized, rather than by the quality of visual representations. Based on these insights, we introduce FocusVLA, a novel paradigm that directs the model's attention to task-relevant visual regions to effectively bridge vision to action. Specifically, we first propose Modality Cascaded Attention to eliminate shortcut pathways, thereby compelling VLA models to rely on task-relevant visual details for action generation. Furthermore, we propose Focus Attention, which dynamically selects task-relevant visual patches to control information quantity while explicitly modulating their influence to suppress task-irrelevant noise. Extensive experiments on both simulated and real-world robotic benchmarks demonstrate that FocusVLA not only effectively leverages visual details to perform dexterous manipulations, but also substantially improves performance and accelerates convergence across a variety of tasks.
Figures & tables
Figure 1: Illustration of structural limitations in existing auto-regressive Vision-Language-Action (VLA) models and the motivation of FocusVLA. OpenVLA-OFT suffers from the lack of direct utilization of visual tokens, limiting its ability to ground actions in fine-grained visual details. VLA-Adapter’s mixed attention introduces architectural shortcuts that enable the model to bypass concrete visual details by favoring the easier action-query pathway, leading to imprecise manipulation. Furthermore, it introduces a near-zero gating factor that substantially suppresses visual signals. In contrast, FocusVLA uses Modality Cascaded Attention to enforce sequential modality interaction, ensuring the model relies on task-relevant visual details before action reasoning and eliminating shortcut pathways. In addition, we propose Focus Attention to suppress task-irrelevant information, leading to precise and robust manipulation.
Figure 2: Visualization of attention maps on LIBERO simulation (left) and real-world environments (right). We extract the attention scores from the last layer, average them across attention heads and queries, and project them back onto the image plane for visualization. Specifically, for VLM, we use the attention from the action query to visual tokens, while for the policy, we use the attention from the action latent to visual tokens. VLA-Adapter exhibits highly scattered and distracted attention patterns, largely focusing on task-irrelevant regions due to structural shortcuts, thereby failing to capture concrete visual details. In contrast, FocusVLA produces concentrated and task-aligned attention maps that consistently focus on contact regions and manipulation targets. This targeted attention enables more precise action generation across both simulated and real-world settings.
Figure 3: Architectures of the four proposed policy variants. (a) Vanilla: A baseline utilizing visual tokens without constraints. (b) Pooling: Patch-level optimization by reducing visual token count via 2×2 pooling. (c) 1-param gate: Channel-level optimization that attenuates visual signal intensity using a single-parameter gate. (d) Cascaded attention: Structure-level optimization that alters feature interaction patterns through cascaded multi-head attention.
Figure 4: Success Rates of different policy architectures on the LIBERO-Long. We evaluate four different structures across three distinct visual representations as detailed in Section 3.2 . The experimental results reveal that existing policies often suffer from three critical biases: token quantity imbalance, low signal-to-noise ratio, and structural bias. By systematically addressing these issues through our proposed constraints, we achieve consistent and significant performance improvements across all visual representations. These findings demonstrate that VLA performance is primarily limited by how visual information is utilized, rather than by the quality of visual representations. “VLM” denotes the output visual features from the PrismaticVLM trained on Qwen2.5-0.5B [ 31 ] , while “DS” denotes the combined features of DINOv2 and SigLIP [ 21 , 34 ] . The red dashed line indicates the performance of a variant that does not use VLM visual features, relying solely on action queries.
Figure 5: The architecture of FocusVLA. Our policy explicitly addresses visual utilization bottlenecks through two core components: (1) Cascaded Attention: By enabling the action latent to query each modality independently, the model is forced to focus on and extract only the most task-relevant visual details, ensuring a more targeted feature utilization; and (2) Focus Attention: To resolve the quantity and quality limitations, this component employs patch-level pruning to discard irrelevant tokens and channel-level gated suppression to mitigate feature noise.
Method
Param
Spatial
Object
Goal
Long
Avg.
Multi-weights (one policy per task)
OpenVLA [ 14 ]
7B
84.7
88.4
79.2
53.7
76.5
OpenVLA-OFT [ 13 ]
7B
97.6
98.4
97.9
94.5
97.1
UniVLA [ 5 ]
7B
96.5
96.8
95.6
92.0
95.2
Spatial Forcing [ 16 ]
7B
99.4
99.6
98.8
96.0
98.5
SpatialVLA [ 23 ]
4B
88.2
89.9
78.6
55.5
78.1
Table 1: Comparisons with state-of-the-art methods on LIBERO benchmark.
Figure 6: Comparisons with state-of-the-art methods on RoboTwin 2.0 benchmark.
Attention Pattern
Patch-level Focus
Channel-level Focus
Visual Representation
Spatial SR (%)
Object SR (%)
Goal SR (%)
Long SR (%)
AVG. (%)
Mixed
512
w/o gate
VLM
94.4
95.6
93.2
91.0
93.6
Cascaded
512
w/o gate
VLM
98.0
98.6
96.2
95.0
97.0
Cascaded
128
w/o gate
VLM
96.8
99.0
95.4
93.8
96.3
Cascaded
256
w/o gate
VLM
99.0
99.2
98.2
95.4
98.0
Cascaded
384
w/o gate
VLM
98.4
99.6
97.8
95.2
97.8
Cascaded
512
w/o gate
VLM
98.0
98.6
96.2
95.0
97.0
Table 2: Component Analysis on LIBERO benchmark under the multi-weights setting. *DS denotes the combination of DINOv2 and SigLIP.
Figure 7: Comparison of training efficiency between VLA-Adapter and FocusVLA.
Figure 8: Visualization of Focus Attention. (a) Patch-level focus. In the primary view, background regions are largely removed with key objects retained, maintaining sufficient spatial cues for policy planning. In the wrist view, representations only concentrate on the gripper and target object, enabling precise grasp execution. (b) Channel-level focus. Without gating, noisy information may lead to instruction-inconsistent actions; with gating, irrelevant signals are suppressed, yielding more accurate and instruction-consistent behavior.
Figure 9: Experimental settings and results in real-world scenarios.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Backbone
Qwen2.5-0.5B
Number of layers (τ/M)
24
Hidden size
896
Attention heads
8
Utilization of VLM features
layers 1–24
Utilization of DINOv2 + SigLIP features
last layer
Appendix
Table 3: Implementation details and hyperparameters of FocusVLA.
Suite
1
2
3
4
5
6
7
8
9
10
Avg. ↑
Spatial
100.0
100.0
100.0
100.0
100.0
100.0
100.0
96.0
100.0
100.0
99.6
Object
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Goal
100.0
96.0
98.0
100.0
100.0
100.0
98.0
100.0
98.0
98.0
98.8
Long
96.0
98.0
100.0
100.0
98.0
100.0
94.0
98.0
84.0
94.0
96.2
Appendix
Table 4: Task-level success rates (%) on the LIBERO benchmark.
Task
DP
π0
VLA-Adapter
FocusVLA
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Beat Block Hammer
42
0
43
21
89
43
93
54
Blocks Ranking RGB
0
0
19
5
35
8
43
11
Click Alarmclock
61
5
63
11
44
4
81
20
Hanging Mug
8
0
11
3
8
1
18
5
Move Pillbottle Pad
1
0
21
1
10
1
20
2
Appendix
Table 5: Detailed success rates (%) on RoboTwin benchmark.
Figure 10: Visualizations of FocusVLA executing tasks in the LIBERO benchmark.
Figure 11: Comparison between FocusVLA and VLA-Adapter on the same LIBERO task.
Figure 12: Examples of unexpected situations during task execution in LIBERO simulation, including two failure trajectories and one successful recovery case.
Figure 13: Visualizations of FocusVLA executing tasks in the RoboTwin benchmark.
Figure 14: Comparison between FocusVLA and VLA-Adapter on the same RoboTwin task.
Figure 15: Failure cases on the RoboTwin benchmark.
Figure 16: Visualizations of FocusVLA executing tasks in the real-world experiments.
Figure 17: Comparison between FocusVLA and VLA-Adapter on the same real-world task.
Figure 18: Failure cases on the real-world experiments.
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics