Federated Subspace Guided Vision-Language-Action Policy Distillation for Non-IID Multi-Robot Manipulation

Biprodip Pal; Kaushik Roy; Yanming Zhu; Brendan Tidd; Alan Wee-Chung Liew; Peyman Moghadam

cs.ROSep 26, 2026

Federated Subspace Guided Vision-Language-Action Policy Distillation for Non-IID Multi-Robot Manipulation

Authors: Biprodip Pal, Kaushik Roy, Yanming Zhu, Brendan Tidd, Alan Wee-Chung Liew, Peyman Moghadam

Organizations: School of ICT, Griffith University, Australia · CSIRO Robotics, CSIRO, Australia

Abstract

Federated learning offers a natural way for multiple robots to jointly improve manipulation policies without requiring centralized access to training demonstrations. However, non-IID task and environment distributions can induce representation drift and mutually incompatible robot-policy updates, making naive parameter aggregation destructive. We present FedDRMan, a federated subspace-guided distillation framework for heterogeneous robot manipulation. At each communication round, the server model provides a frozen teacher for local behavior cloning, while low-rank multimodal subspace and action-distribution distillation preserve globally useful representation geometry and policy behavior. To address heterogeneous aggregation, FedDRMan groups clients by update compatibility and maintains a persistent model for each cluster. The server then spectrally rebalances each compatible aggregate to mitigate attenuation of weaker task-relevant robot-policy update directions. Extensive experiments on LIBERO across diverse non-IID settings, heterogeneity levels, client participation variation, together with ablations and aggregation analyses, show that FedDRMan substantially improves knowledge transfer and consistently outperforms strong federated baselines achieving a peak mean success rate of 80.7%, 11.6 percentage points above the strongest evaluated federated baseline.

Figures & tables

Fig. 1 : Overview of FedDRMan. Multimodal subspace distillation limits representation drift during local adaptation, while compatible-update aggregation and spectral rebalancing mitigates destructive aggregation of conflicting non-IID robot-policy updates and preserve task-relevant update directions.

Method	Object		Goal		Spatial		Mean SR (%) $\uparrow$	Comm. MB / Params
Method	SR (%) $\uparrow$	Best Round	SR (%) $\uparrow$	Best Round	SR (%) $\uparrow$	Best Round	Mean SR (%) $\uparrow$	Comm. MB / Params
Centralized BC	$56.0\pm 2.0$	35	$60.4\pm 2.0$	50	$46.0\pm 4.0$	50	$54.1$	- / -
Local BC	$27.2\pm 2.0$	50	$39.6\pm 4.0$	50	$37.2\pm 3.0$	50	$34.7$	- / -
FedAvg [ 4 ]	$75.2\pm 3.0$	50	$70.0\pm 2.0$	25	$62.0\pm 3.0$	25	$69.1$	21.53 / 5.38M
FedProx [ 5 ]	$70.8\pm 2.0$	40	$72.0\pm 2.0$	35	$52.0\pm 4.0$	35	$64.9$	21.53 / 5.38M
FedKD [ 8 ]	$20.0\pm 2.0$	40	$56.0\pm 4.0$	35	$44.0\pm 3.0$	25	$40.0$	21.53 / 5.38M

TABLE I : Experimental results of same-suite task non-IID(

\alpha{=}0.5

) federated evaluation across three different LIBERO task suites. All metrics are measured based on SR(%). Bold denotes the best federated result.

Method	Object-Goal		Object-Spatial		Goal-Spatial		Object-Goal-Spatial			Mean SR (%) $\uparrow$
Method	Obj	Goal	Obj	Spat	Goal	Spat	Obj	Goal	Spat	Mean SR (%) $\uparrow$
Centralized BC	$78.0{\pm}2.0$	$78.0{\pm}2.0$	$62.0{\pm}1.0$	$46.0{\pm}2.0$	$58.0{\pm}2.0$	$54.0{\pm}3.0$	$44.0{\pm}4.0$	$60.0{\pm}3.0$	$42.0{\pm}3.0$	58.0
Local BC	$15.0{\pm}2.0$	$48.0{\pm}3.0$	$15.0{\pm}2.0$	$35.0{\pm}2.0$	$48.0{\pm}3.0$	$35.0{\pm}2.0$	$15.0{\pm}2.0$	$48.0{\pm}3.0$	$35.0{\pm}2.0$	32.7
FedAvg [ 4 ]	$18.0{\pm}2.0$	$0.0{\pm}0.0$	$18.0{\pm}3.0$	$0.0{\pm}0.0$	$12.0{\pm}1.0$	$12.0{\pm}0.0$	$2.0{\pm}1.0$	$14.0{\pm}3.0$	$8.0{\pm}2.0$	9.3
FedProx [ 5 ]	$16.0{\pm}1.0$	$10.0{\pm}2.0$	$16.0{\pm}1.0$	$2.0{\pm}1.0$	$12.0{\pm}2.0$	$0.0{\pm}0.0$	$4.0{\pm}2.0$	$0.0{\pm}0.0$	$2.0{\pm}1.0$	6.9
FedKD [ 8 ]	$20.0{\pm}2.0$	$8.0{\pm}2.0$	$4.0{\pm}2.0$	$6.0{\pm}2.0$	$14.0{\pm}3.0$	$14.0{\pm}4.0$	$0.0{\pm}0.0$	$2.0{\pm}0.0$	$0.0{\pm}0.0$	$7.6$

TABLE II : Experimental results of cross-suite task and environment non-IID (

\alpha{=}0.5

) federated evaluation across different LIBERO suite combinations. All metrics are measured based on SR(%). Bold denotes the best federated result.

Fig. 2 : Task sensitivity analysis across different suites.

Panel of Figure 5 — Fig. 3 : Sensitivity of FedDRMan to varying Dirichlet non-IID severity ( $\alpha$ ).

Panel of Figure 6 — Fig. 4 : Analysis of sensitivity to client participation variation.

$\mathcal{L}_{\mathrm{BC}}$	$\mathcal{L}_{\mathrm{sub}}$	$\mathcal{L}_{\mathrm{action}}$	SRB	CUA	SR (Object) $\uparrow$	SR (Goal) $\uparrow$
$\checkmark$	$\times$	$\times$	$\times$	$\checkmark$	70.0	60.0
$\checkmark$	$\times$	$\checkmark$	$\times$	$\checkmark$	35.0	55.0
$\checkmark$	$\checkmark$	$\times$	$\times$	$\checkmark$	60.0	50.0
$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	80.0	60.0
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	85.0	70.0
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	0.0	0.0

TABLE III : Component-wise ablation of FedDRMan on the Object-Goal cross-suite task and environment non-IID setting.

Panel of Figure 8 — Fig. 5 : Aggregation effectiveness.

Explore similar work

An Empirical Study and Open Testbed for Federated Fine-Tuning of Vision-Language-Action Models
Sep 19, 2026Zhekai Duan, Kevin Ziyang Xie, Xinyu Tan +5Vision-Language-Action Models VLMs for Robotics
Department of Computer Science, University College London, U.K. · Cisco Research, San Jose, CA, USA.
Recursive Harness Distillation across Agents for Robot Manipulation
Sep 27, 2026Seungyeon Kim, Junhoo Lee, Minkyu Kim +2Robotic Manipulation Robot Policy Generalization
Seoul National University · Korea Advanced Institute of Science and Technology
FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity
Aug 2, 2026Ganghyeon Lee, Inha Lee, Junhee Lee +3Depth Estimation Non-IID Federated Learning
Ulsan National Institute of Science and Technology (UNIST), Republic of Korea

Sep 19, 2026, cs.ROJ/K move · Enter open · S save

An Empirical Study and Open Testbed for Federated Fine-Tuning of Vision-Language-Action Models

Zhekai Duan, Kevin Ziyang Xie, Xinyu Tan, Shikai Geng+4

Department of Computer Science, University College London, U.K. · Cisco Research, San Jose, CA, USA.

Vision-Language-Action Models VLMs for Robotics Robustness of VLA Models

Open paper AskPDF arXiv

Adapting a pretrained Vision-Language-Action (VLA) model to a new robot, environment, or task requires demonstrations that are collected locally and often discarded. Federated learning is a promising approach to exploiting such distributed demonstrations by learning a shared policy. However, whether it can adapt large pretrained VLAs remains an open question, and a lack of reproducible benchmarks for pretrained VLAs and reusable training frameworks makes existing results difficult to compare. In this paper, we conduct a systematic study of federated fine-tuning of three modern pretrained VLA policies on the 40 simulated tasks of the LIBERO manipulation benchmark, and on six real-world tasks in two real-robot experiments, with demonstrations collected across two and three sites, respectively. Our study analyzes the key choices in this setting, spanning multiple federated parameter scopes, three aggregation algorithms, and evaluation under distribution shift. Based on the study, we derive a series of lessons, including the dominance of the federated scope over the choice of aggregation algorithm and the difficulty of matching centralized fine-tuning on physical robots, where cross-site heterogeneity is stronger than simulation captures. We also highlight opportunities for federated VLA learning, such as the ability to match centralized fine-tuning on heterogeneous data, to remain at least as robust as centralized fine-tuning under distribution shift, and to personalize, with each client federating part of the policy and keeping the rest local, which helps where the policy's pretraining is weak but leaves no usable global model. We open-source \decentvla{}, the model- and runtime-agnostic testbed behind the study, to facilitate future research and fair comparisons in federated VLA learning.

Abstract

Explore similar work

An Empirical Study and Open Testbed for Federated Fine-Tuning of Vision-Language-Action Models

Recursive Harness Distillation across Agents for Robot Manipulation

FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity