Vision-Language-Action (VLA) models are typically deployed as isolated single-agent policies. When several such agents share a workspace, they can coordinate implicitly, by observing one another, or explicitly, by exchanging messages, but it remains unclear when explicit communication becomes necessary. We introduce CoLA (Coordination via Learned Adapters), which keeps each agent's VLA backbone frozen and trains a lightweight message channel and a per-arm diffusion action head on top, with fully decentralised execution. Across three tasks that differ in how much each agent can observe, the channel matters only when information is private. On an occluded tray handover, where the box's colour marker is visible to one agent only, CoLA reaches 76.0% success versus 1.3% with messages zeroed throughout training. On two-arm and three-arm handover tasks, removing messages has no measurable effect with a shared overhead camera (90.6% vs. 88.3% at two arms; 63.5% vs. 60.0% at three), but lowers success on the two-arm task's wrist-only variant (74.8% vs. 64.3%). This architecture generalises across different VLA backbones as well. With a frozen π0.5 backbone on the same occluded tray task, CoLA matches the overall success rate of a LoRA-fine-tuned baseline (37.3% vs. 34.7%) using a fraction of the trainable parameters. When CoLA places the box, it selects the correct tray 71.8% of the time, whereas the independently fine-tuned baseline, whose placing arm never sees the marker, selects it at chance level (34.7% against 33.3%).
Figures & tables
Figure 1: CoLA framework overview. (a) Decentralised inference: each agent’s frozen VLA backbone (Octo-Base or π0.5 ) encodes its local observation. Its CoLA module compresses the backbone feature into a compact message ( dm=64 ), decodes the partner’s message, and concatenates the result with its own representation to form zi . This vector conditions a U-Net action head via FiLM, which predicts a coordinated action chunk ( K=10 ). (b) Adapter and action-head training: backbone features are cached offline and the backbones stay frozen. Only the CoLA modules are trained with a behavioural cloning loss, with gradients flowing end-to-end through the communication channel. (c) Multi-agent broadcast: each agent broadcasts its message to all other agents and receives theirs, so the same design scales to N≥3 agents.
Table 1: Main results. Success rate (%), ordered from least to most shared observability. HO = handover. Handover columns: mean ± std over 200-episode evaluation blocks ( section 4.2 ); every row is a single training run. Tray: single 150-episode block. –: not evaluated.
Method
Params (per arm)
Train (GPU-h)
Placed (%)
Correct ∣ placed (%)
Overall (%)
CoLA
8.5 M
0.2
52.0 [44.1, 59.8]
71.8 [61.0, 80.6]
37.3 [30.0, 45.3]
Decentralised, LoRA fine-tune
467.0 M
24.4
100.0 [97.5, 100]
34.7 [27.5, 42.6]
34.7 [27.5, 42.6]
Table 2: Cross-backbone transfer on the occluded tray-handover task. CoLA on a frozen π0.5 backbone versus independently LoRA-fine-tuned per-arm π0.5 policies. Placed : the box reaches any tray. Correct ∣ placed : the box reaches the correct tray, given that it reaches one (chance 33.3% ). Overall : the box reaches the correct tray, over all episodes. Brackets give 95% Wilson intervals; bold marks a value whose interval does not overlap the other method’s.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Value
Backbone feature ( fi )
Octo-Base readout_action token, d=768 , frozen
Proprio encoder ( ϕprop )
Dense (7→64) –ReLU–Dense (64→64) ; dropout p=0.1 in training
Self representation ( hi )
R832 (wrist-only); R1600 with the overhead feature
π0.5 state input
Own 7 -d state as prompt tokens (CoLA and LoRA alike); CoLA also via ϕprop
Message encoder ( ϕenc )
Dense (832→96) –ReLU–Dense (96→64) , dm=64 (input 1600 with overhead)
Message decoder ( ϕdec )
Dense (64→96) –ReLU–Dense (96→768) , one per ordered pair
Appendix
Table 3: CoLA specifications and hyperparameters. Parameter counts are exact, read from the trained checkpoints; times are on one NVIDIA A100-SXM4-40GB. Channel =ϕenc+ϕdec ; the proprioceptive encoder ( 4,672 parameters per arm) is counted with the action head. Percentages are relative to the frozen backbone ( 93 M Octo-Base, ∼3.3 B π0.5 ).
Model
Arm
Instruction
CoLA (Octo-Base and π0.5 ), all tasks
all
coordinate with partner
Octo baselines, decentralised
A
pick up the red box and hand it to the other arm
B, C
take the red box from the other arm
Octo baselines, centralised
–
pick up the red box with one arm and hand it to the other
π0.5 LoRA baseline (tray)
A
hand the box to the other arm
B
take the box from the other arm and put it in a tray
Appendix
Table 4: Language instructions. The box is red in every task, and in the tray task only its marker changes colour, so no instruction reveals the hidden information. The role-specific instructions give the baselines more task information than CoLA; on the tray task, the π0.5 baseline’s Arm B is also told that a tray exists.
Task
Method
Steps
Trainable
GPU-h
Occluded tray handover (wrist only), 2 arms
CoLA (with message)
20,250
14.5 M
0.6
CoLA (message zeroed)
20,250
14.5 M
0.6
Decentralised, head-only
20,250
137.0 M
6.0
Centralised, head-only
20,250
68.5 M
6.4
Decentralised, full fine-tune
20,250
323.0 M
6.1
Appendix
Table 5: Trainable capacity and training cost by task and method. Every method within a task uses the same optimiser steps at batch size 128 , which matches the optimisation budget but not compute or capacity: CoLA trains on cached backbone features, so it does a fraction of the work per step. Trainable parameters are the modules updated by gradient descent, summed over the policies a method deploys (two for the 2-arm tasks, three for the 3-arm task); the Octo baselines’ action head is U-Net at horizon 12 ( 68.5 M), and the centralised variants differ only in output width ( 7→14 ), adding <0.1 M. GPU-hours are job wall-clock on one NVIDIA A100-SXM4-40GB, summed over the policies a method deploys; CoLA trains all arms in a single model. They exclude CoLA’s one-time feature extraction ( 5 – 12 min per Octo-Base dataset, amortised over every CoLA run and ablation on that dataset). The centralised policy encodes both arms’ cameras in every forward pass, which is why on the tray task it costs more than the two decentralised policies combined.
Figure 3: Qualitative Trajectory Comparison on Handover. Top: CoLA enables Arm A to maintain grasp stability until Arm B firmly closes its gripper. Bottom: Uncoordinated baseline desynchronises, causing premature release and dropped objects.
Method
0
1000
2000
3000
4000
Mean ± std
CoLA (with message)
88.0
91.0
91.5
92.5
90.0
90.6 ± 1.7
CoLA (message zeroed)
90.5
85.0
89.5
–
–
88.3 ± 2.9
Decentralised, head-only
66.5
66.0
69.0
67.5
72.5
68.3 ± 2.6
Centralised, head-only
62.5
62.0
68.5
63.5
73.0
65.9 ± 4.7
Centralised, full fine-tune
91.0
91.0
94.5
94.0
90.0
92.1 ± 2.0
Decentralised, full fine-tune
78.0
85.5
81.0
82.5
85.0
82.4 ± 3.1
Appendix
Table 6: 2-Arm handover (overhead camera), per evaluation seed. Success rate (%) per 200-episode block; the seed sets the randomised start state. Octo-Base baselines use their wrist camera and the overhead camera.
Method
0
1000
2000
Mean ± std
CoLA (with message)
75.5
77.0
72.0
74.8 ± 2.6
CoLA (message zeroed)
61.5
59.5
72.0
64.3 ± 6.7
Decentralised, head-only
42.0
43.5
46.0
43.8 ± 2.0
Centralised, head-only
19.5
23.5
20.0
21.0 ± 2.2
Centralised, full fine-tune
81.5
81.5
77.0
80.0 ± 2.6
Decentralised, full fine-tune
71.0
69.5
67.5
69.3 ± 1.8
Appendix
Table 7: 2-Arm handover (wrist cameras only), per evaluation seed. Success rate (%) per 200-episode block at evaluation-seed offsets 0, 1000 and 2000.
Method
0
1000
2000
3000
4000
Mean ± std
CoLA (with message)
63.5
–
–
–
–
63.5 †
CoLA (message zeroed)
60.0
–
–
–
–
60.0 †
Decentralised, head-only
20.0
16.5
21.0
21.0
24.0
20.5 ± 2.7
Decentralised, full fine-tune
36.0
24.0
33.0
32.0
33.0
31.6 ± 4.5
Appendix
Table 8: 3-Arm handover chain (overhead camera), per evaluation seed. Success = both transfers completed and arm C holds the box for 20 steps; 200-episode blocks. CoLA rows have one block each; paired on the same 200 start states, message on vs. zeroed is 127 vs. 120 successes (exact McNemar p=0.50 ).
Method
Blocks
n
Success
A → B
B → C ∣ A → B
Held ∣ B → C
CoLA (with message)
1
200
63.5
79.5
79.9
100.0
CoLA (message zeroed)
1
200
60.0
75.0
80.0
100.0
Decentralised, head-only
5
1000
20.5
42.9
51.3
93.2
Decentralised, full fine-tune
5
1000
31.6
50.1
64.1
98.4
Appendix
Table 9: 3-Arm chain, per-stage rates (%) , pooled over all evaluated blocks: each conditional is computed as the total over the total across blocks, not as a mean of per-block rates. A → B: first transfer. B → C ∣ A → B: second transfer among episodes that completed the first. Held ∣ B → C: arm C holds the box for the required 20 steps, among episodes that completed the second transfer. Success is the product of the three stages, e.g. 42.9×51.3×93.2=20.5 for the head-only baseline.
Method
n
Success
Handover
Reached tray
Correct ∣ handover
Correct ∣ tray
CoLA (with message)
150
76.0 [68.6, 82.1]
92.7
88.7
82.0
85.7
CoLA (message zeroed)
150
1.3 [0.4, 4.7]
8.0
6.0
16.7
22.2
Decentralised, head-only
150
22.7 [16.7, 30.0]
88.7
86.0
25.6
26.4
Centralised, head-only
150
16.0 [11.0, 22.7]
68.0
43.3
23.5
36.9
Centralised, full fine-tune
150
10.7 [6.7, 16.6]
79.3
10.7
13.4
100.0
Decentralised, full fine-tune
150
12.0 [7.7, 18.2]
94.7
36.7
12.7
32.7
Appendix
Table 10: Occluded tray task, seed-0 block. All methods on the same 150 episodes (seed block 0, 50 per marker colour). Success = correct tray over all episodes, with 95% Wilson interval. Handover and Reached tray are rates over all episodes; Correct ∣ handover and Correct ∣ tray condition on those stages (chance on Correct ∣ tray is 33.3%).
Ablation
Setting
n
Correct
Correct ∣ handover
Message width dm
16
150
73.3
79.7
Message width dm
32
150
42.0
47.4
Message width dm
64 (default)
150
76.0
82.0
Message width dm
128
150
73.3
76.9
Lighting (OOD)
bright
30
3.3
50.0
Lighting (OOD)
dark
30
10.0
15.8
Appendix
Table 11: CoLA ablations on the occluded tray task. Message width: the same 150 episodes (seeds 0, 50 per marker colour) for every width. Lighting: out-of-distribution scene lighting, 30 episodes each.
Task / protocol
Representation
Dim
Frame acc.
Episode acc.
Tray, 5-fold CV (150 ep.)
self_a
832
100.0
100.0
Tray, 5-fold CV (150 ep.)
msg_a
64
44.2
73.3
Tray, 5-fold CV (150 ep.)
msg_a (zeroed model)
64
39.7
33.3
Handover (wrist only)
R2 ( y / z )
RMSE cm ( y / z )
Box position
self_a
832
1.000 / 1.000
0.04 / 0.08
Box position
msg_a
64
1.000 / 0.985
0.12 / 0.45
Appendix
Table 12: Linear probes on CoLA messages. Tray: multinomial logistic regression to marker colour, split by episode (chance 33.3%); frame = per-timestep accuracy, episode = majority vote. Handover: linear regression to box position. “Zeroed” probes the encoder of the model trained with the channel severed.
Figure 4: Linear probes of channel content. Left: on the occluded tray-handover task, the message decodes box colour well above chance while an untrained control does not, indicating channel-specific information transfer. Right: on the wrist-only handover, the message and an untrained control both decode Arm A’s presentation pose near ceiling, so pose is linearly recoverable from the raw features regardless of training, and this probe cannot isolate the channel’s contribution.
Backbone
Task
Success
Transfer
A touches box
B touches box
Octo-Small
Full task
0.0
0.0
0.0
0.0
Octo-Small
HO
0.0
0.0
29.0
0.0
Octo-Base
Full task
0.0
0.0
2.5
1.0
Octo-Base
HO
0.0
0.0
11.0
0.0
Appendix
Table 13: Zero-shot pretrained Octo (no fine-tuning; Bridge action statistics; overhead camera; 200 episodes, 350 control steps). Full task: pick up and hand over. HO: episodes start with arm A holding the box.
Task
Arm A
Arm B
Arm C
2-Arm handover (wrist only)
98.8
98.8
—
2-Arm handover (overhead)
98.9
98.7
—
Occluded tray handover
100.0
100.0
—
3-Arm handover chain
99.7
99.5
99.8
Appendix
Table 14: Validation gripper accuracy (%) per arm, taken at each run’s best-validation epoch, for the CoLA runs behind table 1 . —: arm not present in the task.
Figure 5: Validation loss for all training runs in table 1 , one panel per coordination regime, ordered as in the main table. Octo steps are converted to epochs so both families share the matched 150 -epoch budget. Decentralised rows train one policy per arm; the line is their mean and the shading their range.
Multi-arm collaboration is becoming a core capability in embodied manipulation. Recent vision-language-action (VLA) models integrate perception, language, and control, but most represent language as a single global instruction and do not provide an explicit mechanism for assigning and composing arm-specific behaviors. This design limits transfer to collaboration patterns that differ from those observed during training. We present MA-VLA, a unified framework for multi-arm collaboration via atomic action assignment. MA-VLA decomposes cooperative behavior into mid-level atomic prompts and allocates them to individual arms, enabling explicit subgoal specification and compositional reuse across tasks. To reduce reliance on fixed execution roles, we introduce Arm Shuffle, a training-time permutation of the observation, state, and assigned atomic prompts for each arm. This permutation enforces role-agnostic instruction following and supports recomposition into unseen coordination patterns, which we term multi-arm compositional generalization. We also construct a benchmark in which test-time collaboration patterns are absent in training set. Across simulation and real-world evaluations, prior state-of-the-art VLAs largely fail under these unseen collaborations, while MA-VLA consistently succeeds. These results indicate that structured, per-arm atomic action assignment offers a practical route to scalable generalization in multi-arm embodied systems. Code, models, and data are available at https://github.com/zhangzaibin/future-robots
Zaibin Zhang, Junlan Xiao, Zhongbo Zhang +11
Dalian University of Technology · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen +4
Vision-Language-Action (VLA) models leverage powerful perceptual priors from web-scale Vision-Language Model (VLM) pre-training, yet they remain surprisingly brittle in practice, frequently failing at simple robotic tasks. To mitigate this, we propose Contrastive Conceptor Activation Steering (COAST). COAST builds on the notion of a "conceptor", a linear operator that soft-projects data into the principal components of a target distribution. COAST uses conceptors to identify success-critical subspaces for a target robotic task from a few examples of success and failure rollouts. At inference time, it steers VLA latents into these identified success subspaces to improve task outcomes. Across three architecturally distinct neural policies (flow-matching VLA, autoregressive VLA, and Diffusion Policy), COAST improves absolute mean simulation and real-robot task success rate by over 20 and 40% respectively. The activation subspace geometry reveals that failure modes share substantial structure across tasks while success representations remain largely task-specific. When tasks share similar failure modes, this structure enables previously fitted conceptors to improve performance on new tasks without refitting. Ultimately, our results suggest that current VLAs retain substantial task-relevant knowledge in their latent representations, and that the action expert's decoding bottleneck could be mitigated by steering its residual stream toward task-relevant subspaces. COAST provides a lightweight, training-free path to unlocking these latent capabilities by steering the model towards its own "success" distributions.
Vision-language-action models (VLAs) have emerged as a promising paradigm for general-purpose robot learning, with performance improving as models and datasets scale. Scaling robot data collection, however, remains challenging because data are naturally distributed across robots, tasks, and locations, making centralization costly or impractical. Federated learning offers a way to train on decentralized robot data, but applying it to VLAs requires accounting for heterogeneous robot client data distributions. We present Co-VLA, which applies consensus optimization using the Alternating Direction Method of Multipliers~(ADMM) to federated VLA training. We show that the same algorithm supports both full-model training and parameter-efficient fine-tuning with both fixed-rank and rank-adaptive adapters. The name Co-VLA reflects both consensus and collaboration: clients with different local robot datasets collaboratively train a shared model without sharing their data. Our experiments demonstrate that Co-VLA achieves performance comparable to centralized training in both full-model training and parameter-efficient fine-tuning settings. The project website and videos of our real-world experiments are available at https://embodiedvision.github.io/co-vla/.
Haolong Li, Guner Dilsad Er, Michael Muehlebach +1
Intelligent Perception in Technical Systems, University of Augsburg, Germany · Max Planck Institute for Intelligent Systems, Germany