VLA-Fabric: Learning Interaction for Composable Multi-Arm VLA Agents

Chenglong Zhang; Shuaijun Liu; Feiyang You; Shuyang Hao; Chengyu Wu; Ningxin Su

cs.ROOct 16, 2026

VLA-Fabric: Learning Interaction for Composable Multi-Arm VLA Agents

Authors: Chenglong Zhang, Shuaijun Liu, Feiyang You, Shuyang Hao, Chengyu Wu, Ningxin Su

Organizations: The Hong Kong University of Science and Technology (Guangzhou)

Abstract

Vision-language-action (VLA) models are rapidly expanding what individual robots can accomplish, yet many real-world manipulation settings require multiple arms to operate as a coordinated system. Existing approaches organize multi-arm coordination in different ways, including centralized joint policies, high-level task or role assignment, and locally executed policies that react to teammates. However, how complete action-producing VLAs should explicitly interact with one another during action generation remains largely unexplored. We introduce VLA-Fabric, which organizes one complete VLA per arm while preserving local perception and action ownership, and enables coordination through learned interaction across policy boundaries. We first systematically study where such interaction should occur, how peer information should be aggregated, and how the interaction should be learned, revealing complementary coordination functions spanning shared context, peer-specific latent information, and action-stage interaction. We then re-instantiate these functions in a structurally different VLA architecture, showing that the coordination principle is not tied to a particular tensor layout or backbone implementation. Finally, we compose teams of three and four arms and evaluate them in simulation and physical experiments. On all four physical tasks, Full interaction improves success over independently trained local policies by 16--40 percentage points. Together, these results suggest that explicit interaction among complete VLA policies offers a scalable path toward networked embodied intelligence.

Figures & tables

Figure 1: From multi-arm coordination to VLA-Fabric. Complete arm-local VLAs collaborate through learned interaction; we study interaction discovery, cross-architecture transfer, and larger-team scaling.

Panel of Figure 2 — Figure 2: Discovering the VLA-Fabric interaction architecture. (a) Search space over interaction entry, operator, and adaptation. (b) Controlled results on Handover Box and Shoes Table. Gold: 100k RAW follow-up; dashed: TwinVLA.

Figure 3: Re-instantiating VLA-Fabric across VLA architectures. The same interaction functions enter different computational boundaries: SingleVLA fuses final readouts, whereas

\pi_{0.5}

exchanges evolving Action Expert states.

A. Bimanual capability
Policy	Setting	Box	Shoes	Mic	Scan
RDT-1B ( Liu et al., 2025 )	ICLR 2025	69.4	66.0	90.0	4.0
$\pi_{0}$ ( Black et al., 2025 )	RSS 2025	78.8	82.4	98.0	18.0
TwinVLA ( Im et al., 2026 )	ICLR 2026	78.0	84.8	84.0	11.0
SingleVLA-Fabric	Full	80.0	70.0	84.0	5.5
$\pi_{0.5}$ -Fabric	Full	80.5	94.5	97.5	33.0

Table 1: Bimanual capability and fixed-weight interaction removals (success, %).

Panel of Figure 4 — Figure 4: Two- to four-agent VLA-Fabric teams and larger-team simulation tasks (Table 2 ).

Task	Independent	Full
Stack3	90.5	91.0
Frame3	67.5	79.5
Frame4	9.5	89.5
Arch4	0.0	5.0

Table 2: Larger-team success (%).

Panel of Figure 5 — Figure 5: Representative physical executions of the four real-world tasks; each row shows six chronological global-camera frames.

Task	Independent	Full
Basket3	72.0	88.0
Relay3	16.0	56.0
Relay4	40.0	72.0
Frame4	36.0	60.0

Table 3: Physical success (%).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Design	Common interaction	Private K/V	Action interaction
I0	None	Off	None
I1	None	On	None
I2	One-way	On	None
I3-AVG	Averaged K/V	On	None
I3-RAW	Operator-level	On	None
I4-AVG	Averaged K/V	On	Linear

Appendix

Table A1: Interaction design space. Each design preserves complete local VLA policies.

Design	Adaptation	Handover Box	Shoes Table
I0: Independent	Single-stage	47.0	39.5
I1: Private K/V	Single-stage	56.0	55.5
I2: One-way Common	Single-stage	39.5	21.5
I3-AVG	Single-stage	41.0	35.0
I3-RAW	Single-stage	45.5	56.5
I4-AVG: Linear	Single-stage	52.5	13.0

Appendix

Table A2: Complete architecture-search results. Success rates (%) use 200 paired conditions per task, disjoint from model selection.

Design	Handover Box	Shoes Table
I4-RAW: Linear	80.0	66.0
I5-RAW: Transformer	74.0	68.5

Appendix

Table A3: Extended-budget Common-first comparison, with 110k total updates including initialization. Success rates (%) use 200 paired conditions per task.

Task	Demos	Total updates	Trials
Handover Box	50	50k	200
Shoes Table	50	40k	200
Handover Mic	50	30k	200
Scan Object	50	60k	200

Appendix

Table B1: Task-specific

\pi_{0.5}

Full settings. Update counts include all adaptation, including initialization where used.

Methods	Tasks	Source / setting	Trials
RDT-1B, $\pi_{0}$ , TwinVLA	Box, Shoes	Im et al. (2026) , Table 9 / Easy	500
TwinVLA	Mic, Scan	Im et al. (2026) , Table 10 / Easy	100
RDT-1B, $\pi_{0}$	Mic, Scan	RoboTwin Team (2026) / Single, clean2clean	100

Appendix

Table B2: Reference evaluation settings. Trial counts are per task.

Task	Arms	Success criterion
Stack3	3	Complete the ordered cube stack with all assigned arms released.
Frame3	3	Place the triangular frame over the peg after joint lifting and transport; release and retreat with the frame stable.
Frame4	4	Place the quadrilateral frame over the peg after joint lifting and transport; release and retreat with the frame stable.
Arch4	4	Position the supports and place the beam to form a stable completed arch.

Appendix

Table C1: Larger-team simulation settings. Each policy setting uses 200 evaluation trials; completion requires the full task outcome.

Task	Arms	Role assignment and completion criterion
Basket3	3	Each arm loads an assigned vegetable; success requires all vegetables to be placed in the shared basket.
Relay3	3	One arm positions the receiving target, while two arms transfer the object; success requires handover and placement within the target.
Relay4	4	Two sender–receiver pairs relay vegetables to the basket; both pairs must complete their handovers and basket placements.
Frame4	4	Four arms jointly lift, align, and lower the frame over the peg; success requires stable placement after release.

Appendix

Table C2: Physical task definitions and successful completion.

Multi-Agent Coordination

Multi-agent coordination studies how autonomous agents align actions, share information, and divide work to achieve individual or collective goals.

See all 520 papers

MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization
Aug 26, 2026Zaibin Zhang, Junlan Xiao, Zhongbo Zhang +11Vision-Language-Action Models Instruction Generalization in VLA Models
Dalian University of Technology · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen +4
CoLA: Multi-VLA Coordination via Learned Adapters
Oct 11, 2026Ahaskar Kashyap, Jiawen Zong, Boyu Ma +4Multi-Agent Coordination Multi-Robot Coordination
MARS Lab · Nanyang Technological University, Singapore · Autonomous Agents Research Group
Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems
Jun 18, 2026Yandong Wang, Jiaqian Yu, Xiongfeng Peng +8Bimanual Robotic Manipulation Vision-Language-Action Models
Donghua University, Shanghai, China · Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea

Aug 26, 2026J/K move · Enter open · S save

MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization

Zaibin Zhang, Junlan Xiao, Zhongbo Zhang, Yifan Wang+10

Dalian University of Technology · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen+4

Vision-Language-Action Models Instruction Generalization in VLA Models Bimanual Robotic Manipulation

Open paper AskPDF arXiv

Multi-arm collaboration is becoming a core capability in embodied manipulation. Recent vision-language-action (VLA) models integrate perception, language, and control, but most represent language as a single global instruction and do not provide an explicit mechanism for assigning and composing arm-specific behaviors. This design limits transfer to collaboration patterns that differ from those observed during training. We present MA-VLA, a unified framework for multi-arm collaboration via atomic action assignment. MA-VLA decomposes cooperative behavior into mid-level atomic prompts and allocates them to individual arms, enabling explicit subgoal specification and compositional reuse across tasks. To reduce reliance on fixed execution roles, we introduce Arm Shuffle, a training-time permutation of the observation, state, and assigned atomic prompts for each arm. This permutation enforces role-agnostic instruction following and supports recomposition into unseen coordination patterns, which we term multi-arm compositional generalization. We also construct a benchmark in which test-time collaboration patterns are absent in training set. Across simulation and real-world evaluations, prior state-of-the-art VLAs largely fail under these unseen collaborations, while MA-VLA consistently succeeds. These results indicate that structured, per-arm atomic action assignment offers a practical route to scalable generalization in multi-arm embodied systems. Code, models, and data are available at https://github.com/zhangzaibin/future-robots

Abstract

Multi-Agent Coordination

Explore similar work

MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization

CoLA: Multi-VLA Coordination via Learned Adapters

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems