CoLA: Multi-VLA Coordination via Learned Adapters

Ahaskar Kashyap; Jiawen Zong; Boyu Ma; Chuhao Zhou; Jingliang Li; Stefano V. Albrecht; Jianfei Yang

cs.ROOct 11, 2026

CoLA: Multi-VLA Coordination via Learned Adapters

Authors: Ahaskar Kashyap, Jiawen Zong, Boyu Ma, Chuhao Zhou, Jingliang Li, Stefano V. Albrecht, Jianfei Yang

Organizations: MARS Lab · Nanyang Technological University, Singapore · Autonomous Agents Research Group

Abstract

Vision-Language-Action (VLA) models are typically deployed as isolated single-agent policies. When several such agents share a workspace, they can coordinate implicitly, by observing one another, or explicitly, by exchanging messages, but it remains unclear when explicit communication becomes necessary. We introduce CoLA (Coordination via Learned Adapters), which keeps each agent's VLA backbone frozen and trains a lightweight message channel and a per-arm diffusion action head on top, with fully decentralised execution. Across three tasks that differ in how much each agent can observe, the channel matters only when information is private. On an occluded tray handover, where the box's colour marker is visible to one agent only, CoLA reaches 76.0% success versus 1.3% with messages zeroed throughout training. On two-arm and three-arm handover tasks, removing messages has no measurable effect with a shared overhead camera (90.6% vs. 88.3% at two arms; 63.5% vs. 60.0% at three), but lowers success on the two-arm task's wrist-only variant (74.8% vs. 64.3%). This architecture generalises across different VLA backbones as well. With a frozen $π_{0.5}$ backbone on the same occluded tray task, CoLA matches the overall success rate of a LoRA-fine-tuned baseline (37.3% vs. 34.7%) using a fraction of the trainable parameters. When CoLA places the box, it selects the correct tray 71.8% of the time, whereas the independently fine-tuned baseline, whose placing arm never sees the marker, selects it at chance level (34.7% against 33.3%).

Figures & tables

Figure 1: CoLA framework overview. (a) Decentralised inference: each agent’s frozen VLA backbone (Octo-Base or $\pi_{0.5}$ ) encodes its local observation. Its CoLA module compresses the backbone feature into a compact message ( $d_{m}=64$ ), decodes the partner’s message, and concatenates the result with its own representation to form ${\bm{z}}_{i}$ . This vector conditions a U-Net action head via FiLM, which predicts a coordinated action chunk ( $K=10$ ). (b) Adapter and action-head training: backbone features are cached offline and the backbones stay frozen. Only the CoLA modules are trained with a behavioural cloning loss, with gradients flowing end-to-end through the communication channel. (c) Multi-agent broadcast: each agent broadcasts its message to all other agents and receives theirs, so the same design scales to $N\geq 3$ agents.

Figure 2: Multi-Agent Manipulation Progression. Key milestones across evaluated tasks: (top) Occluded Tray-Handover Task, (middle) 2-arm Reactive Handover, and (bottom) 3-arm Handover Chain.

Method	Occluded Tray-HO (wrist only)	2-Arm HO (wrist only)	2-Arm HO (overhead)	3-Arm HO (overhead)
CoLA (with message)	76.0	74.8 $\pm$ 2.6	90.6 $\pm$ 1.7	63.5 †
CoLA (message zeroed)	1.3	64.3 $\pm$ 6.7	88.3 $\pm$ 2.9	60.0 †
Decentralised, head-only	22.7	43.8 $\pm$ 2.0	68.3 $\pm$ 2.6	20.5 $\pm$ 2.7
Centralised, head-only	16.0	21.0 $\pm$ 2.2	65.9 $\pm$ 4.7	—
Centralised, full fine-tune	10.7	80.0 $\pm$ 2.6	92.1 $\pm$ 2.0	—
Decentralised, full fine-tune	12.0	69.3 $\pm$ 1.8	82.4 $\pm$ 3.1	31.6 $\pm$ 4.5

Table 1: Main results. Success rate (%), ordered from least to most shared observability. HO = handover. Handover columns: mean

\pm

std over 200-episode evaluation blocks ( section 4.2 ); every row is a single training run. Tray: single 150-episode block. –: not evaluated.

Method	Params (per arm)	Train (GPU-h)	Placed (%)	Correct $\mid$ placed (%)	Overall (%)
CoLA	$8.5$ M	$0.2$	52.0 [44.1, 59.8]	71.8 [61.0, 80.6]	37.3 [30.0, 45.3]
Decentralised, LoRA fine-tune	$467.0$ M	$24.4$	100.0 [97.5, 100]	34.7 [27.5, 42.6]	34.7 [27.5, 42.6]

Table 2: Cross-backbone transfer on the occluded tray-handover task. CoLA on a frozen

\pi_{0.5}

backbone versus independently LoRA-fine-tuned per-arm

\pi_{0.5}

policies. Placed : the box reaches any tray. Correct

\mid

placed : the box reaches the correct tray, given that it reaches one (chance

33.3\%

). Overall : the box reaches the correct tray, over all episodes. Brackets give 95% Wilson intervals; bold marks a value whose interval does not overlap the other method’s.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Value
Backbone feature ( ${\bm{f}}_{i}$ )	Octo-Base readout_action token, $d=768$ , frozen
Proprio encoder ( $\phi_{\text{prop}}$ )	Dense $(7\to 64)$ –ReLU–Dense $(64\to 64)$ ; dropout $p=0.1$ in training
Self representation ( ${\bm{h}}_{i}$ )	$\mathbb{R}^{832}$ (wrist-only); $\mathbb{R}^{1600}$ with the overhead feature
$\pi_{0.5}$ state input	Own $7$ -d state as prompt tokens (CoLA and LoRA alike); CoLA also via $\phi_{\text{prop}}$
Message encoder ( $\phi_{\text{enc}}$ )	Dense $(832\to 96)$ –ReLU–Dense $(96\to 64)$ , $d_{m}=64$ (input $1600$ with overhead)
Message decoder ( $\phi_{\text{dec}}$ )	Dense $(64\to 96)$ –ReLU–Dense $(96\to 768)$ , one per ordered pair

Appendix

Table 3: CoLA specifications and hyperparameters. Parameter counts are exact, read from the trained checkpoints; times are on one NVIDIA A100-SXM4-40GB. Channel

=\phi_{\text{enc}}+\phi_{\text{dec}}

; the proprioceptive encoder (

4{,}672

parameters per arm) is counted with the action head. Percentages are relative to the frozen backbone (

93

M Octo-Base,

{\sim}3.3

\pi_{0.5}

Model	Arm	Instruction
CoLA (Octo-Base and $\pi_{0.5}$ ), all tasks	all	coordinate with partner
Octo baselines, decentralised	A	pick up the red box and hand it to the other arm
	B, C	take the red box from the other arm
Octo baselines, centralised	–	pick up the red box with one arm and hand it to the other
$\pi_{0.5}$ LoRA baseline (tray)	A	hand the box to the other arm
	B	take the box from the other arm and put it in a tray

Appendix

Table 4: Language instructions. The box is red in every task, and in the tray task only its marker changes colour, so no instruction reveals the hidden information. The role-specific instructions give the baselines more task information than CoLA; on the tray task, the

\pi_{0.5}

baseline’s Arm B is also told that a tray exists.

Task	Method	Steps	Trainable	GPU-h
Occluded tray handover (wrist only), 2 arms
	CoLA (with message)	$20{,}250$	$14.5$ M	$0.6$
	CoLA (message zeroed)	$20{,}250$	$14.5$ M	$0.6$
	Decentralised, head-only	$20{,}250$	$137.0$ M	$6.0$
	Centralised, head-only	$20{,}250$	$68.5$ M	$6.4$
	Decentralised, full fine-tune	$20{,}250$	$323.0$ M	$6.1$

Appendix

Table 5: Trainable capacity and training cost by task and method. Every method within a task uses the same optimiser steps at batch size

128

, which matches the optimisation budget but not compute or capacity: CoLA trains on cached backbone features, so it does a fraction of the work per step. Trainable parameters are the modules updated by gradient descent, summed over the policies a method deploys (two for the 2-arm tasks, three for the 3-arm task); the Octo baselines’ action head is U-Net at horizon

12

(

68.5

M), and the centralised variants differ only in output width (

7\to 14

), adding

{<}0.1

M. GPU-hours are job wall-clock on one NVIDIA A100-SXM4-40GB, summed over the policies a method deploys; CoLA trains all arms in a single model. They exclude CoLA’s one-time feature extraction (

5

–

12

min per Octo-Base dataset, amortised over every CoLA run and ablation on that dataset). The centralised policy encodes both arms’ cameras in every forward pass, which is why on the tray task it costs more than the two decentralised policies combined.

Figure 3: Qualitative Trajectory Comparison on Handover. Top: CoLA enables Arm A to maintain grasp stability until Arm B firmly closes its gripper. Bottom: Uncoordinated baseline desynchronises, causing premature release and dropped objects.

Method	0	1000	2000	3000	4000	Mean $\pm$ std
CoLA (with message)	88.0	91.0	91.5	92.5	90.0	90.6 $\pm$ 1.7
CoLA (message zeroed)	90.5	85.0	89.5	–	–	88.3 $\pm$ 2.9
Decentralised, head-only	66.5	66.0	69.0	67.5	72.5	68.3 $\pm$ 2.6
Centralised, head-only	62.5	62.0	68.5	63.5	73.0	65.9 $\pm$ 4.7
Centralised, full fine-tune	91.0	91.0	94.5	94.0	90.0	92.1 $\pm$ 2.0
Decentralised, full fine-tune	78.0	85.5	81.0	82.5	85.0	82.4 $\pm$ 3.1

Appendix

Table 6: 2-Arm handover (overhead camera), per evaluation seed. Success rate (%) per 200-episode block; the seed sets the randomised start state. Octo-Base baselines use their wrist camera and the overhead camera.

Method	0	1000	2000	Mean $\pm$ std
CoLA (with message)	75.5	77.0	72.0	74.8 $\pm$ 2.6
CoLA (message zeroed)	61.5	59.5	72.0	64.3 $\pm$ 6.7
Decentralised, head-only	42.0	43.5	46.0	43.8 $\pm$ 2.0
Centralised, head-only	19.5	23.5	20.0	21.0 $\pm$ 2.2
Centralised, full fine-tune	81.5	81.5	77.0	80.0 $\pm$ 2.6
Decentralised, full fine-tune	71.0	69.5	67.5	69.3 $\pm$ 1.8

Appendix

Table 7: 2-Arm handover (wrist cameras only), per evaluation seed. Success rate (%) per 200-episode block at evaluation-seed offsets 0, 1000 and 2000.

Method	0	1000	2000	3000	4000	Mean $\pm$ std
CoLA (with message)	63.5	–	–	–	–	63.5 †
CoLA (message zeroed)	60.0	–	–	–	–	60.0 †
Decentralised, head-only	20.0	16.5	21.0	21.0	24.0	20.5 $\pm$ 2.7
Decentralised, full fine-tune	36.0	24.0	33.0	32.0	33.0	31.6 $\pm$ 4.5

Appendix

Table 8: 3-Arm handover chain (overhead camera), per evaluation seed. Success = both transfers completed and arm C holds the box for 20 steps; 200-episode blocks. CoLA rows have one block each; paired on the same 200 start states, message on vs. zeroed is 127 vs. 120 successes (exact McNemar

p=0.50

Method	Blocks	$n$	Success	A $\to$ B	B $\to$ C $\mid$ A $\to$ B	Held $\mid$ B $\to$ C
CoLA (with message)	1	200	63.5	79.5	79.9	100.0
CoLA (message zeroed)	1	200	60.0	75.0	80.0	100.0
Decentralised, head-only	5	1000	20.5	42.9	51.3	93.2
Decentralised, full fine-tune	5	1000	31.6	50.1	64.1	98.4

Appendix

Table 9: 3-Arm chain, per-stage rates (%) , pooled over all evaluated blocks: each conditional is computed as the total over the total across blocks, not as a mean of per-block rates. A

\to

B: first transfer. B

\to

\mid

\to

B: second transfer among episodes that completed the first. Held

\mid

\to

C: arm C holds the box for the required 20 steps, among episodes that completed the second transfer. Success is the product of the three stages, e.g.

42.9\times 51.3\times 93.2=20.5

for the head-only baseline.

Method	$n$	Success	Handover	Reached tray	Correct $\mid$ handover	Correct $\mid$ tray
CoLA (with message)	150	76.0 [68.6, 82.1]	92.7	88.7	82.0	85.7
CoLA (message zeroed)	150	1.3 [0.4, 4.7]	8.0	6.0	16.7	22.2
Decentralised, head-only	150	22.7 [16.7, 30.0]	88.7	86.0	25.6	26.4
Centralised, head-only	150	16.0 [11.0, 22.7]	68.0	43.3	23.5	36.9
Centralised, full fine-tune	150	10.7 [6.7, 16.6]	79.3	10.7	13.4	100.0
Decentralised, full fine-tune	150	12.0 [7.7, 18.2]	94.7	36.7	12.7	32.7

Appendix

Table 10: Occluded tray task, seed-0 block. All methods on the same 150 episodes (seed block 0, 50 per marker colour). Success = correct tray over all episodes, with 95% Wilson interval. Handover and Reached tray are rates over all episodes; Correct

\mid

handover and Correct

\mid

tray condition on those stages (chance on Correct

\mid

tray is 33.3%).

Ablation	Setting	$n$	Correct	Correct $\mid$ handover
Message width $d_{m}$	16	150	73.3	79.7
Message width $d_{m}$	32	150	42.0	47.4
Message width $d_{m}$	64 (default)	150	76.0	82.0
Message width $d_{m}$	128	150	73.3	76.9
Lighting (OOD)	bright	30	3.3	50.0
Lighting (OOD)	dark	30	10.0	15.8

Appendix

Table 11: CoLA ablations on the occluded tray task. Message width: the same 150 episodes (seeds 0, 50 per marker colour) for every width. Lighting: out-of-distribution scene lighting, 30 episodes each.

Task / protocol	Representation	Dim	Frame acc.	Episode acc.
Tray, 5-fold CV (150 ep.)	self_a	832	100.0	100.0
Tray, 5-fold CV (150 ep.)	msg_a	64	44.2	73.3
Tray, 5-fold CV (150 ep.)	msg_a (zeroed model)	64	39.7	33.3
Handover (wrist only)			$R^{2}$ ( $y$ / $z$ )	RMSE cm ( $y$ / $z$ )
Box position	self_a	832	1.000 / 1.000	0.04 / 0.08
Box position	msg_a	64	1.000 / 0.985	0.12 / 0.45

Appendix

Table 12: Linear probes on CoLA messages. Tray: multinomial logistic regression to marker colour, split by episode (chance 33.3%); frame = per-timestep accuracy, episode = majority vote. Handover: linear regression to box position. “Zeroed” probes the encoder of the model trained with the channel severed.

Figure 4: Linear probes of channel content. Left: on the occluded tray-handover task, the message decodes box colour well above chance while an untrained control does not, indicating channel-specific information transfer. Right: on the wrist-only handover, the message and an untrained control both decode Arm A’s presentation pose near ceiling, so pose is linearly recoverable from the raw features regardless of training, and this probe cannot isolate the channel’s contribution.

Backbone	Task	Success	Transfer	A touches box	B touches box
Octo-Small	Full task	0.0	0.0	0.0	0.0
Octo-Small	HO	0.0	0.0	29.0	0.0
Octo-Base	Full task	0.0	0.0	2.5	1.0
Octo-Base	HO	0.0	0.0	11.0	0.0

Appendix

Table 13: Zero-shot pretrained Octo (no fine-tuning; Bridge action statistics; overhead camera; 200 episodes, 350 control steps). Full task: pick up and hand over. HO: episodes start with arm A holding the box.

Task	Arm A	Arm B	Arm C
2-Arm handover (wrist only)	98.8	98.8	—
2-Arm handover (overhead)	98.9	98.7	—
Occluded tray handover	100.0	100.0	—
3-Arm handover chain	99.7	99.5	99.8

Appendix

Table 14: Validation gripper accuracy (%) per arm, taken at each run’s best-validation epoch, for the CoLA runs behind table 1 . —: arm not present in the task.

Figure 5: Validation loss for all training runs in table 1 , one panel per coordination regime, ordered as in the main table. Octo steps are converted to epochs so both families share the matched $150$ -epoch budget. Decentralised rows train one policy per arm; the line is their mean and the shading their range.

Explore similar work

MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization
Aug 26, 2026Zaibin Zhang, Junlan Xiao, Zhongbo Zhang +11Vision-Language-Action Models Instruction Generalization in VLA Models
Dalian University of Technology · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen +4
Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden States
May 16, 2026Miranda Muqing Miao, Subin Kim, Brandon Yang +1VLM Adaptation Vision-Language-Action Models
University of Pennsylvania
Co-VLA: Consensus-based Federated Training for Vision-Language-Action Models
Sep 17, 2026Haolong Li, Guner Dilsad Er, Michael Muehlebach +1Vision-Language-Action Models Federated Fine-Tuning
Intelligent Perception in Technical Systems, University of Augsburg, Germany · Max Planck Institute for Intelligent Systems, Germany

Aug 26, 2026J/K move · Enter open · S save

MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization

Zaibin Zhang, Junlan Xiao, Zhongbo Zhang, Yifan Wang+10

Dalian University of Technology · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen+4

Vision-Language-Action Models Instruction Generalization in VLA Models Bimanual Robotic Manipulation

Open paper AskPDF arXiv

Multi-arm collaboration is becoming a core capability in embodied manipulation. Recent vision-language-action (VLA) models integrate perception, language, and control, but most represent language as a single global instruction and do not provide an explicit mechanism for assigning and composing arm-specific behaviors. This design limits transfer to collaboration patterns that differ from those observed during training. We present MA-VLA, a unified framework for multi-arm collaboration via atomic action assignment. MA-VLA decomposes cooperative behavior into mid-level atomic prompts and allocates them to individual arms, enabling explicit subgoal specification and compositional reuse across tasks. To reduce reliance on fixed execution roles, we introduce Arm Shuffle, a training-time permutation of the observation, state, and assigned atomic prompts for each arm. This permutation enforces role-agnostic instruction following and supports recomposition into unseen coordination patterns, which we term multi-arm compositional generalization. We also construct a benchmark in which test-time collaboration patterns are absent in training set. Across simulation and real-world evaluations, prior state-of-the-art VLAs largely fail under these unseen collaborations, while MA-VLA consistently succeeds. These results indicate that structured, per-arm atomic action assignment offers a practical route to scalable generalization in multi-arm embodied systems. Code, models, and data are available at https://github.com/zhangzaibin/future-robots

Abstract

Explore similar work

MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization

Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden States

Co-VLA: Consensus-based Federated Training for Vision-Language-Action Models