Robust Bimanual Vision-Language-Action Models via Embarrassingly Simple Modality Masking
Authors: Dongzhou Cheng, Ziang Li, Yixiao Zhou, Haojuan Li, Jinghao Zhang, Lei Lei, Minjing Dong, Jie Gui, +1 more
Organizations: Southeast University · Shanghai Innovation Institute · Wuhan University · Zhejiang University · Shanghai Jiao Tong University · University of Science and Technology of China · City University of Hong Kong
Query-based Vision-Language-Action (VLA) models offer low-latency inference that is attractive for bimanual robotic manipulation, but we observe that they can still exhibit discontinuous actions and execution failures in complex dual-arm tasks. We hypothesize that unstable multi-view and language fusion is one contributing factor in these failures, often coinciding with attention spreading to distracting regions. To improve robustness, we introduce the Modality Masking Mechanism (M3), an embarrassingly simple, training-only strategy that requires no architectural changes or large-scale robot pretraining. M3 stochastically masks subsets of modality channels during training, exposing the policy to controlled partial observations and encouraging it to rely less on distracting cues and more on evidence that remains reliable. We evaluate M3 on ten bimanual tasks from RoboTwin 2.0 and on three long-horizon real-world tasks. Compared with the Adapter baseline, M3 improves average success by 21.7% in the Clean setting and 11.4% in Clean2Rand, where policies are trained on clean demonstrations and evaluated on randomized scenes, while also improving averaged real-world full-task success by over 30%. These results suggest that structured training-time masking is a practical way to improve the robustness of query-based VLA policies for bimanual manipulation.
Figures & tables
Figure 1: Left: representative multi-view, dual-arm platforms that reflect a prevalent form factor in current embodied manipulation research. (a) A plain query-based VLA exhibits trajectory discontinuities near the contact region. (b) The M3-trained policy produces more stable end-effector paths and more consistent contact alignment. (c) As shown in the bottom bar charts, M3 yields consistent improvements over the plain baseline across both simulation and real-world evaluations under clean conditions.
Figure 2: Attention structure and contact-rich rollout behavior. Top: Post-hoc attention-score maps for the Adapter baseline without M3 (left) and M3 (right). Rows are query tokens and columns are key tokens; E, L, R, T, and Q denote the egocentric, left-wrist, right-wrist, text, and action-query token groups, respectively. The M3 map exhibits patterns consistent with anchoring global context in the egocentric stream, reducing cross-view shortcuts, and encouraging more expressive query interactions. Bottom: At the handover-to-placement stage of the long-horizon Handover Block task, M3 maintains stable dual-arm coordination and completes placement in the shown rollout, whereas the Adapter baseline knocks the block over during contact and does not recover. Together, the panels juxtapose the intended change in attention structure with the qualitative execution difference observed in a representative rollout.
Figure 3: Overview of the Modality Masking Mechanism (M3). M3 is a training-time strategy that injects partial observability by stochastically masking modalities and action queries. As illustrated along the batch and token axes, different token subsets (comprising Egocentric (E), Left-wrist (L), Right-wrist (R) observations, Text Instructions (T), and Action Queries (Q)) are made visible (grey) at each training step. These partially observable inputs are processed by the VLM to produce M3-conditioned latents, which are then used by the action expert to decode actions. This process is intended to encourage more robust evidence selection and complementary representations without over-relying on any single input channel.
Figure 4: Example tasks from the RoboTwin 2.0 [ 6 ] benchmark, illustrating the visual distinction between the Clean (top row) and Clean2Rand (bottom row) evaluation settings. The six tasks shown are representative examples, with two selected from each of the short-, medium-, and long-horizon categories. Clean2Rand trains on clean demonstrations and evaluates on scenes with changed backgrounds, lighting, and distractor objects.
Category
Task Name
RDT ∗
π0∗
ACT
DP
Adapter
M3
Δ
Short Horizon
Click Bell
80
44
58
54
84
97
+13
Grab Roller
74
96
94
98
88
96
+8
Place Phone Stand
15
35
2
13
10
55
+45
Medium Horizon
Place Bread Basket
10
17
6
14
11
22
+11
Place A2B Right
1
27
0
13
4
28
+24
Place Shoe
35
28
5
23
34
63
+29
Table 1: Domain-clean performance across multi-horizon tasks on the RoboTwin 2.0 simulation platform. The best performance in each row is bolded, and the second-best is underlined. Δ denotes the relative improvement of M3 over the Adapter baseline. Models marked with ∗ are pretrained on large-scale robot data, while our method achieves superior performance through direct fine-tuning.
Category
Task Name
RDT ∗
π0∗
ACT
DP
Adapter
M3
Δ
Short Horizon
Click Bell
9
3
3
0
7
16
+9
Grab Roller
43
80
25
0
28
54
+26
Place Phone Stand
6
7
0
0
0
12
+12
Medium Horizon
Place Bread Basket
2
4
0
0
0
6
+6
Place A2B Right
1
6
0
0
0
2
+2
Place Shoe
7
6
0
0
0
11
+11
Table 2: Clean2Rand performance across multi-horizon tasks on the RoboTwin 2.0 simulation platform. All task-specific policies are trained on 50 clean demonstrations and evaluated on 100 held-out randomized scenes. Relative to the fine-tuned baseline, M3 improves average success by 11.4% . Unlike VLA policies such as π0 [ 5 ] and RDT-1B [ 28 ] , which rely on massive cross-embodiment, internet-scale or multi-robot pretraining to reduce the domain gap, M3 uses only clean, task-specific training data and achieves comparable or better generalization.
Figure 5: Qualitative comparison of real-world execution under clean and OOD conditions. The second column provides a close-up from the right-arm wrist camera. (a) In the clean setting, the Adapter baseline fails to release the bottle into the bin, whereas M3 completes the discard smoothly. (b) Under OOD clutter, the baseline suffers from gripper misalignment during handover, while M3 maintains stable alignment throughout the sequence.
Figure 6: Cross-backbone transfer. Under the domain-clean RoboTwin protocol, M3-OFT raises average success from 32.2% to 53.5% and exceeds the evaluated baselines.
Figure 7: Overview of real-world evaluation. Top: three representative tasks successfully executed by M3: (1) Bottle Cleanup, (2) Stack Shelf, and (3) Veggie Centering. Bottom-left: our bimanual robot platform. Bottom-center: full-task success rates under clean and OOD settings, where M3 shows consistent improvement over the Adapter baseline. Bottom-right: illustration of the two evaluation settings (clean vs. OOD with novel distractor objects). The baseline degrades more sharply under OOD clutter, suggesting that M3 appears to be more robust to unseen distractors. Detailed task definitions, protocols, additional demonstrations, and tabulated results are given in Sec. 4.1 and the Appendix .
Figure 8: Spatial attention under viewpoint variation. On Click Bell and Block Rank Size , the Adapter baseline spreads attention over distractors, the gripper, and background regions, whereas M3 more consistently concentrates on the target and contact region. Attention remains partly diffuse on the harder Block Rank Size .
Figure 9: Success rate vs. training steps on Place Phone Stand (short-horizon) and Handover Block (long-horizon). Under the same compute budget, M3 tends to reach higher success rates earlier and maintains a consistent margin over the Adapter baseline throughout training.
Table 3: Targeted masking ablations. (a) Vision view-level masking while non-view masking components follow the full-M3 setting. Rows compare masking the egocentric view, one wrist view (L or R), and both wrist views. (b) Query-mask ratio, including the unmasked Adapter baseline (0.0). Masking both wrist views and using a light query-mask ratio (0.1) achieve the highest average success rates in their respective comparisons.
Table 4: Ablation studies. (a) Component ablation: V, L, and Q denote vision, language, and query masking. (b) Comparison with dropout and augmentation baselines. Avg. SR is the mean success rate over Place Phone Stand , Place Shoe , and Handover Block .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Model
VRAM (GB)
Status
OpenVLA-OFT (7B)
≈23-24
OOM-prone
VLA-Adapter (0.5B)
≈10
Stable
Appendix
Table 5: VRAM footprint during simulation on RTX 4090 (24 GB). OpenVLA-OFT approaches the 24 GB ceiling and frequently triggers OOM failures, whereas VLA-Adapter (0.5B) operates with a substantially smaller footprint, enabling stable iteration [ 37 ] .
Task Name
Steps
Horizon
Horizon Group
Short Horizon Tasks
Click Bell
62
Short
Average: 95 steps Count: 3 tasks
Grab Roller
95
Short
Place Phone Stand
128
Short
Medium Horizon Tasks
Place Bread Basket
222
Medium
Average: 197 steps Count: 4 tasks
Appendix
Table 6: Task statistics and horizon categorization on the RoboTwin 2.0 benchmark.
Category
Task Name
Seen Instructions
Unseen Instructions
Short Horizon
Click Bell
Touch the bell at its top center.
Find the palm-sized bell and click its top center.
Grab Roller
Grab the roller on the table.
Use arms to grab the light brown cylindrical roller.
Place Phone Stand
Move the palm-sized phone onto the compact plastic phonestand.
Set the handheld phone with silver frame on the adjustable phone holder.
Medium Horizon
Place Bread Basket
Grab both the small bread and the golden bread loaf, drop into the beige plastic breadbasket.
Use one arm to grab the palm-sized braided golden bread, drop in the round beige breadbasket.
Place A2B Right
Place the manual blue stapler precisely to the right of the woodenblock with back grooves.
Put the hard black box for card deck to the right of the bright blue soap bar.
Place Shoe
Move the white shoe with rounded toe from the table to the mat in one fluid motion.
Grab the synthetic shoe upper from the table and set it on the mat.
Appendix
Table 7: Representative seen (training) and unseen (evaluation) language instructions. During evaluation, instructions are stochastically sampled from the complete unseen set.
Figure 10: Overview of the Modality Masking Mechanism (M3) for OpenVLA-OFT [ 18 ] , which is built upon the OpenVLA-7B [ 19 ] backbone. The legend follows the same color and token conventions as the main-paper overview figure. The three main differences from VLA-Adapter [ 37 ] include: (1) the adoption of parallel decoding with bidirectional attention, (2) the input of zero-query embeddings with shape (action axes)×(chunk size) , and (3) unlike VLA-Adapter, which interacts vision and query latents via an action expert at every layer, here the final-layer query latents are mapped pointwise to the minimal unit of the action signal. For example, if we infer T=25 action steps and each bimanual timestep has A=14 action axes, we provide A×T=14×25=350 query embeddings.
Category
Task Name
RDT ∗
π0∗
Adapter
+M3
OpenVLA-OFT
+M3
Δoft
Short Horizon
Click Bell
80
44
84
97
86
100
+14
Grab Roller
74
96
88
96
94
97
+3
Place Phone Stand
15
35
10
55
24
56
+32
Medium Horizon
Place Bread Basket
10
17
11
22
3
13
+10
Place A2B Right
1
27
4
28
8
10
+2
Place Shoe
35
28
34
63
17
50
+33
Appendix
Table 8: Domain-clean performance across multi-horizon tasks on the RoboTwin 2.0 simulation platform. The columns labeled +M3 denote the integration of our M3 training strategy into the preceding baseline (Adapter and OpenVLA-OFT, respectively). The best performance in each row is bolded, and the second-best is underlined. Δoft denotes the relative improvement of M3 over the OpenVLA-OFT baseline. Overall, these results provide additional supporting evidence that M3 may transfer beyond the primary VLA-Adapter backbone in the evaluated domain-clean setting.
Figure 11: Quantitative attention-misalignment analysis on three representative tasks. Discontinuous trajectories tend to exhibit higher unimportant attention than smooth trajectories over normalized rollout time.
Clean
OOD
Task
Phase
Adapter
M3
Adapter
M3
Bottle Cleanup
Brown-bottle disposal
68.8
85.4
20.8
70.8
Green-bottle lift
85.4
91.7
37.5
75.0
Bimanual handover
54.2
79.2
16.7
62.5
Final disposal
54.2
79.2
16.7
62.5
Full task
41.7 ± 9.6
66.7 ± 3.6
16.7 ± 7.2
58.3 ± 19.1
Appendix
Table 9: Per-phase and full-task success rates (%) for three real-world bimanual tasks, pooled over three evaluation rounds (clean: 3×16=48 trials; OOD: 3×8=24 trials per task). Full-task rows report the pooled rate together with the sample standard deviation across the three round-wise full-task success rates.
Figure 12: Qualitative attention maps and rollout timelines comparing the baseline with M3 across representative bimanual tasks.
Vision-Language-Action (VLA) models integrate vision-language understanding with executable robot actions, enabling end-to-end learning for robot control. However, our empirical analysis reveals that existing models exhibit severe trajectory overfitting when finetuned on limited datasets. To guide the model in effectively utilizing wrist camera information, we propose MaskVLA, a masking-based fine-tuning strategy. By randomly masking a small portion of the main camera's visual information, the model is guided to autonomously learn more fine-grained, task-relevant, and effective visual features. This process leads to the emergence of robust policies, thereby enhancing the model's capability to tackle complex manipulation tasks and improving its generalization performance. Our method has been comprehensively evaluated on RoboTwin 2.0, achieving an average success rate improvement of 23.2% and 16.8% compared to π0 and OpenVLA-OFT, respectively. Furthermore, experiments on real-world ALOHA robots also demonstrate the effectiveness of our approach.
Yuxuan Jiang, Jiaying Huang, Ge Wang +9
FNii-Shenzhen, The Chinese University of Hong Kong, Shenzhen, China · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China · Ising AI +2
Vision-language-action (VLA) models show strong capabilities in single and dual-arm robotic manipulation. Prior works show coordinated bimanual behaviors can emerge from end-to-end learning, leveraging large vision-language backbones with continuous action prediction. However, as bimanual tasks become tightly coupled and execution constraints become critical, implicit coordination alone is insufficient to ensure reliable, interpretable, and stable behavior. In this work, we propose Co-VLA, a coordination-aware bimanual manipulation framework introducing explicit structural priors into VLA models. We instantiate our method on a state-of-the-art vision-language backbone by replacing its monolithic action head with a Structured Action Expert (SAE) designed for bimanual coordination. Specifically, we introduce explicit structure at the action generation level with a modular coordination-aware loss that shapes shared and residual latents according to task-specific structures. The shared latent encodes task-level coordination intent, while residual latents capture execution adjustments for each arm. At deployment, a Latent-Aware Controller (LAC) interprets the learned representations to modulate synchronization strength, execution asymmetry, smoothness, and safety constraints in real time. LAC operates at the joint-command level and remains compatible with standard control pipelines without requiring force or impedance control. Experiments across simulation and real-world benchmarks show Co-VLA significantly outperforms monolithic baselines, achieving a 27% success rate gain in tight-coordination tasks, more than doubling performance in OOD real-world scenarios (from 13% to 27%), and reducing task completion time by up to 25%.
Yandong Wang, Jiaqian Yu, Xiongfeng Peng +8
Donghua University, Shanghai, China · Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea
Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning. While effective, this paradigm compromises the inherent generalization capabilities of Vision-Language Models (VLMs) and incurs catastrophic forgetting. To address these limitations, we propose M2-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly. However, it remains a key challenge to bridge the gap between the high-level semantic understanding of VLMs and the precise requirements of robotic control. To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features. Furthermore, to facilitate efficient trajectory learning under constrained model capacity, we propose a Meta Skill Module (MSM) that integrates strong inductive biases. Extensive experiments in both simulated and real-world environments demonstrate the effectiveness of our approach. Furthermore, generalization and ablation studies validate the architecture's zero-shot capabilities and confirm the contribution of each key component. Our code and pre-trained models will be made publicly available.
Siyao Xiao, Yuhong Zhang, Zhifang Liu +9
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · PengCheng Laboratory, Shenzhen, China. · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China. +1