Robust Bimanual Vision-Language-Action Models via Embarrassingly Simple Modality Masking

Dongzhou Cheng; Ziang Li; Yixiao Zhou; Haojuan Li; Jinghao Zhang; Lei Lei; Minjing Dong; Jie Gui

cs.ROAug 23, 2026

Robust Bimanual Vision-Language-Action Models via Embarrassingly Simple Modality Masking

Authors: Dongzhou Cheng, Ziang Li, Yixiao Zhou, Haojuan Li, Jinghao Zhang, Lei Lei, Minjing Dong, Jie Gui,

Organizations: Southeast University · Shanghai Innovation Institute · Wuhan University · Zhejiang University ·

Abstract

Query-based Vision-Language-Action (VLA) models offer low-latency inference that is attractive for bimanual robotic manipulation, but we observe that they can still exhibit discontinuous actions and execution failures in complex dual-arm tasks. We hypothesize that unstable multi-view and language fusion is one contributing factor in these failures, often coinciding with attention spreading to distracting regions. To improve robustness, we introduce the Modality Masking Mechanism (M3), an embarrassingly simple, training-only strategy that requires no architectural changes or large-scale robot pretraining. M3 stochastically masks subsets of modality channels during training, exposing the policy to controlled partial observations and encouraging it to rely less on distracting cues and more on evidence that remains reliable. We evaluate M3 on ten bimanual tasks from RoboTwin 2.0 and on three long-horizon real-world tasks. Compared with the Adapter baseline, M3 improves average success by 21.7% in the Clean setting and 11.4% in Clean2Rand, where policies are trained on clean demonstrations and evaluated on randomized scenes, while also improving averaged real-world full-task success by over 30%. These results suggest that structured training-time masking is a practical way to improve the robustness of query-based VLA policies for bimanual manipulation.

Figures & tables

Figure 1: Left: representative multi-view, dual-arm platforms that reflect a prevalent form factor in current embodied manipulation research. (a) A plain query-based VLA exhibits trajectory discontinuities near the contact region. (b) The M3-trained policy produces more stable end-effector paths and more consistent contact alignment. (c) As shown in the bottom bar charts, M3 yields consistent improvements over the plain baseline across both simulation and real-world evaluations under clean conditions.

Panel of Figure 2 — Figure 2: Attention structure and contact-rich rollout behavior. Top: Post-hoc attention-score maps for the Adapter baseline without M3 (left) and M3 (right). Rows are query tokens and columns are key tokens; E, L, R, T, and Q denote the egocentric, left-wrist, right-wrist, text, and action-query token groups, respectively. The M3 map exhibits patterns consistent with anchoring global context in the egocentric stream, reducing cross-view shortcuts, and encouraging more expressive query interactions. Bottom: At the handover-to-placement stage of the long-horizon Handover Block task, M3 maintains stable dual-arm coordination and completes placement in the shown rollout, whereas the Adapter baseline knocks the block over during contact and does not recover. Together, the panels juxtapose the intended change in attention structure with the qualitative execution difference observed in a representative rollout.

Figure 3: Overview of the Modality Masking Mechanism (M3). M3 is a training-time strategy that injects partial observability by stochastically masking modalities and action queries. As illustrated along the batch and token axes, different token subsets (comprising Egocentric (E), Left-wrist (L), Right-wrist (R) observations, Text Instructions (T), and Action Queries (Q)) are made visible (grey) at each training step. These partially observable inputs are processed by the VLM to produce M3-conditioned latents, which are then used by the action expert to decode actions. This process is intended to encourage more robust evidence selection and complementary representations without over-relying on any single input channel.

Figure 4: Example tasks from the RoboTwin 2.0 [ 6 ] benchmark, illustrating the visual distinction between the Clean (top row) and Clean2Rand (bottom row) evaluation settings. The six tasks shown are representative examples, with two selected from each of the short-, medium-, and long-horizon categories. Clean2Rand trains on clean demonstrations and evaluates on scenes with changed backgrounds, lighting, and distractor objects.

Category	Task Name	RDT ∗	$\pi_{0}^{*}$	ACT	DP	Adapter	M3	$\Delta$
Short Horizon	Click Bell	80	44	58	54	84	97	+13
	Grab Roller	74	96	94	98	88	96	+8
	Place Phone Stand	15	35	2	13	10	55	+45
Medium Horizon	Place Bread Basket	10	17	6	14	11	22	+11
	Place A2B Right	1	27	0	13	4	28	+24
	Place Shoe	35	28	5	23	34	63	+29

Table 1: Domain-clean performance across multi-horizon tasks on the RoboTwin 2.0 simulation platform. The best performance in each row is bolded, and the second-best is underlined.

\Delta

denotes the relative improvement of M3 over the Adapter baseline. Models marked with ∗ are pretrained on large-scale robot data, while our method achieves superior performance through direct fine-tuning.

Category	Task Name	RDT ∗	$\pi_{0}^{*}$	ACT	DP	Adapter	M3	$\Delta$
Short Horizon	Click Bell	9	3	3	0	7	16	+9
	Grab Roller	43	80	25	0	28	54	+26
	Place Phone Stand	6	7	0	0	0	12	+12
Medium Horizon	Place Bread Basket	2	4	0	0	0	6	+6
	Place A2B Right	1	6	0	0	0	2	+2
	Place Shoe	7	6	0	0	0	11	+11

Table 2: Clean2Rand performance across multi-horizon tasks on the RoboTwin 2.0 simulation platform. All task-specific policies are trained on 50 clean demonstrations and evaluated on 100 held-out randomized scenes. Relative to the fine-tuned baseline, M3 improves average success by 11.4% . Unlike VLA policies such as

\pi_{0}

[ 5 ] and RDT-1B [ 28 ] , which rely on massive cross-embodiment, internet-scale or multi-robot pretraining to reduce the domain gap, M3 uses only clean, task-specific training data and achieves comparable or better generalization.

Figure 5: Qualitative comparison of real-world execution under clean and OOD conditions. The second column provides a close-up from the right-arm wrist camera. (a) In the clean setting, the Adapter baseline fails to release the bottle into the bin, whereas M3 completes the discard smoothly. (b) Under OOD clutter, the baseline suffers from gripper misalignment during handover, while M3 maintains stable alignment throughout the sequence.

Figure 6: Cross-backbone transfer. Under the domain-clean RoboTwin protocol, M3-OFT raises average success from $32.2\%$ to $53.5\%$ and exceeds the evaluated baselines.

Figure 7: Overview of real-world evaluation. Top: three representative tasks successfully executed by M3: (1) Bottle Cleanup, (2) Stack Shelf, and (3) Veggie Centering. Bottom-left: our bimanual robot platform. Bottom-center: full-task success rates under clean and OOD settings, where M3 shows consistent improvement over the Adapter baseline. Bottom-right: illustration of the two evaluation settings (clean vs. OOD with novel distractor objects). The baseline degrades more sharply under OOD clutter, suggesting that M3 appears to be more robust to unseen distractors. Detailed task definitions, protocols, additional demonstrations, and tabulated results are given in Sec. 4.1 and the Appendix .

Figure 8: Spatial attention under viewpoint variation. On Click Bell and Block Rank Size , the Adapter baseline spreads attention over distractors, the gripper, and background regions, whereas M3 more consistently concentrates on the target and contact region. Attention remains partly diffuse on the harder Block Rank Size .

Figure 9: Success rate vs. training steps on Place Phone Stand (short-horizon) and Handover Block (long-horizon). Under the same compute budget, M3 tends to reach higher success rates earlier and maintains a consistent margin over the Adapter baseline throughout training.

Table 3: Targeted masking ablations. (a) Vision view-level masking while non-view masking components follow the full-M3 setting. Rows compare masking the egocentric view, one wrist view (L or R), and both wrist views. (b) Query-mask ratio, including the unmasked Adapter baseline (0.0). Masking both wrist views and using a light query-mask ratio (0.1) achieve the highest average success rates in their respective comparisons.

Table 4: Ablation studies. (a) Component ablation: V, L, and Q denote vision, language, and query masking. (b) Comparison with dropout and augmentation baselines. Avg. SR is the mean success rate over Place Phone Stand , Place Shoe , and Handover Block .

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Model	VRAM (GB)	Status
OpenVLA-OFT (7B)	$\approx 23\text{-}24$	OOM-prone
VLA-Adapter (0.5B)	$\approx 10$	Stable

Appendix

Table 5: VRAM footprint during simulation on RTX 4090 (24 GB). OpenVLA-OFT approaches the 24 GB ceiling and frequently triggers OOM failures, whereas VLA-Adapter (0.5B) operates with a substantially smaller footprint, enabling stable iteration [ 37 ] .

Task Name	Steps	Horizon	Horizon Group
Short Horizon Tasks
Click Bell	62	Short	Average: 95 steps Count: 3 tasks
Grab Roller	95	Short
Place Phone Stand	128	Short
Medium Horizon Tasks
Place Bread Basket	222	Medium	Average: 197 steps Count: 4 tasks

Appendix

Table 6: Task statistics and horizon categorization on the RoboTwin 2.0 benchmark.

Category	Task Name	Seen Instructions	Unseen Instructions
Short Horizon	Click Bell	Touch the bell at its top center.	Find the palm-sized bell and click its top center.
	Grab Roller	Grab the roller on the table.	Use arms to grab the light brown cylindrical roller.
	Place Phone Stand	Move the palm-sized phone onto the compact plastic phonestand.	Set the handheld phone with silver frame on the adjustable phone holder.
Medium Horizon	Place Bread Basket	Grab both the small bread and the golden bread loaf, drop into the beige plastic breadbasket.	Use one arm to grab the palm-sized braided golden bread, drop in the round beige breadbasket.
	Place A2B Right	Place the manual blue stapler precisely to the right of the woodenblock with back grooves.	Put the hard black box for card deck to the right of the bright blue soap bar.
	Place Shoe	Move the white shoe with rounded toe from the table to the mat in one fluid motion.	Grab the synthetic shoe upper from the table and set it on the mat.

Appendix

Table 7: Representative seen (training) and unseen (evaluation) language instructions. During evaluation, instructions are stochastically sampled from the complete unseen set.

Figure 10: Overview of the Modality Masking Mechanism (M3) for OpenVLA-OFT [ 18 ] , which is built upon the OpenVLA-7B [ 19 ] backbone. The legend follows the same color and token conventions as the main-paper overview figure. The three main differences from VLA-Adapter [ 37 ] include: (1) the adoption of parallel decoding with bidirectional attention, (2) the input of zero-query embeddings with shape $(\text{action axes})\times(\text{chunk size})$ , and (3) unlike VLA-Adapter, which interacts vision and query latents via an action expert at every layer, here the final-layer query latents are mapped pointwise to the minimal unit of the action signal. For example, if we infer $T=25$ action steps and each bimanual timestep has $A=14$ action axes, we provide $A\times T=14\times 25=350$ query embeddings.

Category	Task Name	RDT ∗	$\pi_{0}^{*}$	Adapter	+M3	OpenVLA-OFT	+M3	$\Delta^{\text{oft}}$
Short Horizon	Click Bell	80	44	84	97	86	100	+14
	Grab Roller	74	96	88	96	94	97	+3
	Place Phone Stand	15	35	10	55	24	56	+32
Medium Horizon	Place Bread Basket	10	17	11	22	3	13	+10
	Place A2B Right	1	27	4	28	8	10	+2
	Place Shoe	35	28	34	63	17	50	+33

Appendix

Table 8: Domain-clean performance across multi-horizon tasks on the RoboTwin 2.0 simulation platform. The columns labeled +M3 denote the integration of our M3 training strategy into the preceding baseline (Adapter and OpenVLA-OFT, respectively). The best performance in each row is bolded, and the second-best is underlined.

\Delta^{\text{oft}}

denotes the relative improvement of M3 over the OpenVLA-OFT baseline. Overall, these results provide additional supporting evidence that M3 may transfer beyond the primary VLA-Adapter backbone in the evaluated domain-clean setting.

Figure 11: Quantitative attention-misalignment analysis on three representative tasks. Discontinuous trajectories tend to exhibit higher unimportant attention than smooth trajectories over normalized rollout time.

		Clean		OOD
Task	Phase	Adapter	M3	Adapter	M3
Bottle Cleanup	Brown-bottle disposal	68.8	85.4	20.8	70.8
	Green-bottle lift	85.4	91.7	37.5	75.0
	Bimanual handover	54.2	79.2	16.7	62.5
	Final disposal	54.2	79.2	16.7	62.5
	Full task	41.7 $\pm$ 9.6	66.7 $\pm$ 3.6	16.7 $\pm$ 7.2	58.3 $\pm$ 19.1

Appendix

Table 9: Per-phase and full-task success rates (%) for three real-world bimanual tasks, pooled over three evaluation rounds (clean:

3\!\times\!16\!=\!48

trials; OOD:

3\!\times\!8\!=\!24

trials per task). Full-task rows report the pooled rate together with the sample standard deviation across the three round-wise full-task success rates.

Figure 12: Qualitative attention maps and rollout timelines comparing the baseline with M3 across representative bimanual tasks.

Figure 13: Qualitative rollout comparisons on representative long-horizon bimanual manipulation tasks.

Figure 22: Veggie Centering: OOD Clutter.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,154 papers

Sep 20, 2026

MaskVLA: Visual Masking Against Trajectory Overfitting of Vision-Language-Action Model

Vision-Language-Action (VLA) models integrate vision-language understanding with executable robot actions, enabling end-to-end learning for robot control. However, our empirical analysis reveals that existing models exhibit severe trajectory overfitting when finetuned on limited datasets. To guide the model in effectively utilizing wrist camera information, we propose MaskVLA, a masking-based fine-tuning strategy. By randomly masking a small portion of the main camera's visual information, the model is guided to autonomously learn more fine-grained, task-relevant, and effective visual features. This process leads to the emergence of robust policies, thereby enhancing the model's capability to tackle complex manipulation tasks and improving its generalization performance. Our method has been comprehensively evaluated on RoboTwin 2.0, achieving an average success rate improvement of 23.2% and 16.8% compared to

π_0

and OpenVLA-OFT, respectively. Furthermore, experiments on real-world ALOHA robots also demonstrate the effectiveness of our approach.

Yuxuan Jiang, Jiaying Huang, Ge Wang +9

FNii-Shenzhen, The Chinese University of Hong Kong, Shenzhen, China · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China · Ising AI +2

Robotic Manipulation Vision-Language-Action Models Robustness of VLA Models

Ask PDF

Jun 18, 2026

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems

Vision-language-action (VLA) models show strong capabilities in single and dual-arm robotic manipulation. Prior works show coordinated bimanual behaviors can emerge from end-to-end learning, leveraging large vision-language backbones with continuous action prediction. However, as bimanual tasks become tightly coupled and execution constraints become critical, implicit coordination alone is insufficient to ensure reliable, interpretable, and stable behavior. In this work, we propose Co-VLA, a coordination-aware bimanual manipulation framework introducing explicit structural priors into VLA models. We instantiate our method on a state-of-the-art vision-language backbone by replacing its monolithic action head with a Structured Action Expert (SAE) designed for bimanual coordination. Specifically, we introduce explicit structure at the action generation level with a modular coordination-aware loss that shapes shared and residual latents according to task-specific structures. The shared latent encodes task-level coordination intent, while residual latents capture execution adjustments for each arm. At deployment, a Latent-Aware Controller (LAC) interprets the learned representations to modulate synchronization strength, execution asymmetry, smoothness, and safety constraints in real time. LAC operates at the joint-command level and remains compatible with standard control pipelines without requiring force or impedance control. Experiments across simulation and real-world benchmarks show Co-VLA significantly outperforms monolithic baselines, achieving a 27% success rate gain in tight-coordination tasks, more than doubling performance in OOD real-world scenarios (from 13% to 27%), and reducing task completion time by up to 25%.

Yandong Wang, Jiaqian Yu, Xiongfeng Peng +8

Donghua University, Shanghai, China · Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea

Bimanual Robotic Manipulation Vision-Language-Action Models

Ask PDF

Apr 27, 2026

$M^2$ -VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills

Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning. While effective, this paradigm compromises the inherent generalization capabilities of Vision-Language Models (VLMs) and incurs catastrophic forgetting. To address these limitations, we propose

M^2

-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly. However, it remains a key challenge to bridge the gap between the high-level semantic understanding of VLMs and the precise requirements of robotic control. To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features. Furthermore, to facilitate efficient trajectory learning under constrained model capacity, we propose a Meta Skill Module (MSM) that integrates strong inductive biases. Extensive experiments in both simulated and real-world environments demonstrate the effectiveness of our approach. Furthermore, generalization and ablation studies validate the architecture's zero-shot capabilities and confirm the contribution of each key component. Our code and pre-trained models will be made publicly available.

Siyao Xiao, Yuhong Zhang, Zhifang Liu +9

Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · PengCheng Laboratory, Shenzhen, China. · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China. +1

Language-Conditioned Robot Manipulation VLM Adaptation Generalization in Robotic Manipulation Robotic Manipulation Vision-Language-Action Models Robot Learning

Ask PDF

Abstract

Vision-Language-Action Models

Explore similar work

MaskVLA: Visual Masking Against Trajectory Overfitting of Vision-Language-Action Model

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems

M2M^2M2-VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills

$M^2$ -VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills