PFedMA: Who Should Join Next as Federated Multi-Agent Orchestration Evolves?

Shuai Li; Shengyuan Liu; Fan Qi; Xiaoshan Yang

cs.AIOct 14, 2026

PFedMA: Who Should Join Next as Federated Multi-Agent Orchestration Evolves?

Authors: Shuai Li, Shengyuan Liu, Fan Qi, Xiaoshan Yang

Organizations: Tianjin University of Technology · Inner Mongolia University · Institute of automation, Chinese academy of science

Abstract

Federated Multi-Agent Systems (FedMAS) offer a promising paradigm for collaborative reasoning among LLM-based agents while keeping raw client data decentralized. However, reliable multi-round collaboration remains challenging. Agent selection must take into consideration evolving task requirements and the agent's historical reliability, while privacy rules must adapt to changing collaboration contexts and meet the client's personalized needs. To address these challenges, we propose PFedMA, a federated multi-agent framework for feedback driven multi-round collaboration under heterogeneous privacy requirements. Across rounds, PFedMA maintains historical capability and reliability states to guide agent selection, evaluates task responses before aggregation, and refines subsequent queries according to accumulated collaboration feedback. PFedMA enables bidirectional local to global and global to local private rule evolution across heterogeneous clients. Clients generalize locally updated privacy rules and selectively incorporate globally maintained rules, thereby facilitating cross-client privacy rule evolution while preserving personalized constraints. Extensive experiments across reasoning, privacy sensitive, and adversarial settings show that PFedMA improves task performance, reduces measured privacy leakage, and improves system robustness.

Figures & tables

Figure 1: Overview of PFedMA. Task response feedback guides query refinement and client routing, while privacy rules evolve through cross client sharing and personalized adaptation. Together, they jointly determine the participants and the content to be shared, thereby laying the foundation for continued collaboration. The algorithm is provided in Appendix A .

RQ1: Performance comparison across multiple benchmarks
Method	Math	GSM8K	AIME24	MedQA	AlpacaEval 2.0	MedMCQA	Avg
Single Agent	$42.63_{\pm 3.51}$	$79.11_{\pm 1.39}$	$6.67_{\pm 0.73}$	$63.35_{\pm 4.31}$	$58.51_{\pm 0.18}$	$57.82_{\pm 0.21}$	$51.35_{\pm 1.72}$
Router-R1	$76.06_{\pm 2.73}$	$84.02_{\pm 2.91}$	$73.40_{\pm 4.34}$	$75.61_{\pm 1.71}$	$\underline{75.43_{\pm 3.94}}$	$69.57_{\pm 1.07}$	$75.68_{\pm 2.78}$
KABB	$77.51_{\pm 1.12}$	$85.66_{\pm 0.60}$	$71.93_{\pm 3.00}$	$\bm{78.15_{\pm 0.73}}$	$72.38_{\pm 4.30}$	$\bm{72.05_{\pm 1.70}}$	$76.28_{\pm 1.91}$
OPTAGENT	$75.02_{\pm 1.27}$	$\underline{86.26_{\pm 4.31}}$	$62.43_{\pm 2.43}$	$66.27_{\pm 1.21}$	$69.01_{\pm 1.78}$	$64.82_{\pm 0.91}$	$70.64_{\pm{1.99}}$
EPEAgents	$70.03_{\pm 0.39}$	$80.10_{\pm 1.76}$	$68.57_{\pm 3.16}$	$77.20_{\pm 1.99}$	$70.88_{\pm 1.36}$	$63.54_{\pm 0.81}$	$71.72_{\pm 1.58}$

Table 1: Performance and privacy comparison of different frameworks. Best results are shown in bold , and second best results are underlined . For Avg calculation, lower is better metrics are converted using

100-score

before averaging.

RQ3: Evaluation of system security and robustness
Method	ASR $\downarrow$	RR $\uparrow$	BP $\uparrow$	PNA $\uparrow$	NRP $\uparrow$	Avg $\uparrow$
Single Agent	$54.82_{\pm 1.23}$	$5.34_{\pm 2.41}$	$\underline{77.75_{\pm 3.84}}$	$66.57_{\pm 2.03}$	$30.08_{\pm 3.20}$	$44.98_{\pm 2.54}$
Router-R1	$52.07_{\pm 0.81}$	$9.10_{\pm 0.36}$	$76.73_{\pm 0.74}$	$76.70_{\pm 0.65}$	$36.76_{\pm 0.88}$	$49.44_{\pm 0.69}$
KABB	$36.20_{\pm 0.69}$	$18.50_{\pm 0.54}$	$75.29_{\pm 0.83}$	$\bm{88.00_{\pm 0.47}}$	$56.14_{\pm 0.76}$	$60.35_{\pm 0.66}$
OPTAGENT	$50.07_{\pm 0.77}$	$11.50_{\pm 2.49}$	$69.40_{\pm 0.91}$	$73.54_{\pm 0.58}$	$36.72_{\pm 1.02}$	$48.22_{\pm 1.15}$
EPEAgents	$\underline{32.01_{\pm 0.52}}$	$\underline{22.13_{\pm 0.61}}$	$76.57_{\pm 1.72}$	$82.84_{\pm 0.55}$	$\underline{56.32_{\pm 0.69}}$	$\underline{61.17_{\pm 0.82}}$

Table 2: Robustness of different frameworks under adversarial attacks

Panel of Figure 2 — Figure 2: Ablation studies of the reliability module, iterative orchestration loop, federated privacy rule co-evolution loop and robustness utility trade off across adversarial ratios.

Panel of Figure 3 — Figure 3: Performance Evaluation under Iterative Optimization and Risk Threshold Variation.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Setting	Expert Coordination	Iterative Refinement	Reliability Feedback	Privacy Personalization	Privacy Rules Co-evolution
Router-R1	Centralized MAS	RL-based dynamic routing	✓	–	–	–
KABB	Centralized MAS	Knowledge aware expert selection	Partial	–	–	–
Fed-ICL	Federated MAS	Federated response aggregation	✓	–	–	–
PFedMA	Federated MAS	Reliability aware dynamic routing	✓	✓	✓	✓

Appendix

Table 3: Comparison of PFedMA with representative multi-agent and federated multi-agent frameworks. PFedMA differs from prior methods by jointly coupling iterative reasoning, agent reliability, and heterogeneous privacy constraints within a closed loop orchestration process.

Figure 7: Format of the client’s response.

Component	Implementation	Shares $\pi_{\theta}$	Trainable
Intent evaluator	Frozen LLM judge	No	No
Evidence evaluator	Frozen LLM judge	No	No
Consensus evaluator	Semantic similarity	No	No
Orchestrator	RL fine tuned LLM	–	Yes

Appendix

Table 4: Implementation and training status of the response evaluators.

Hyperparameter	Value	Hyperparameter	Value
Learning Rate (Actor)	1e-5	Learning Rate (Critic)	1e-5
Total Batch Size	64	Mini-batch Size	64
Micro-batch Size	8	Max Training Steps	220
Max Routing Steps	100	Max Sequence Length	4096
Max Response Length	2048	Max Length for LLM API Response	1024
Tensor Parallel Size	1	GPU Utilization Ratio	0.8

Appendix

Table 5: Hyperparameter Settings.

Abbr.	Full Capability	Abbr.	Full Capability
Math	Mathematical Calculation	Q.Anlys.	Question Analysis
Code	Code Generation	Geo.Logic	Geographical Logical Reasoning
Trans.	Translation	Med.QA	Medical Knowledge Question Answering
Summary	Summarization	Clin.Rec.	Clinical Record Structuring
Story	Story Generation	Med.Lit.	Medical Literature Retrieval
Combined	Combined Task	Pat.Comm.	Patient Communication

Appendix

Table 6: Explanation of capability abbreviations.

Panel of Figure 8 — Figure 8: Agent Ability

Panel of Figure 9 — Figure 9: Accuracy of federated iterative optimization, federated iteration rounds and demonstration of privacy rule effectiveness, API usage and client data heterogeneity in PFedMA with $N=100$ clients.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,040 papers

Apr 6, 2026

Agentic Federated Learning: The Future of Distributed Training Orchestration

Although Federated Learning (FL) promises privacy and distributed collaboration, its effectiveness in real-world scenarios is often hampered by the stochastic heterogeneity of clients and unpredictable system dynamics. Existing static optimization approaches fail to adapt to these fluctuations, resulting in resource underutilization and systemic bias. In this work, we propose a paradigm shift towards Agentic-FL, a framework where Language Model-based Agents (LMagents) assume autonomous orchestration roles. Unlike rigid protocols, we demonstrate how server-side agents can mitigate selection bias through contextual reasoning, while client-side agents act as local guardians, dynamically managing privacy budgets and adapting model complexity to hardware constraints. More than just resolving technical inefficiencies, this integration signals the evolution of FL towards decentralized ecosystems, where collaboration is negotiated autonomously, paving the way for future markets of incentive-based models and algorithmic justice. We discuss the reliability (hallucinations) and security challenges of this approach, outlining a roadmap for resilient multi-agent systems in federated environments.

Rafael O. Jarczewski, Gabriel U. Talasso, Leandro Villas +1

Institute of Computing, University of Campinas and Hub of Artificial Intelligence and Cognitive Architectures - H.IAAC, Campinas, Brazil

Federated Learning Client Selection in Federated Learning LLM Agents

Ask PDF

Sep 2, 2026

Privacy-Preserving Topology-Guided Safety for LLM-Based Multi-Agent Systems via Federated Graph Learning

Topology-guided safeguards for LLM-based multi-agent systems (MAS) train a GNN over the inter-agent communication graph to localize risky agents and intervene on the topology---but they assume one operator can pool all labeled traces. Across organizations that assumption breaks: episodes contain private prompts, tool outputs, and proprietary workflows, and no silo alone sees the full attack distribution. We cast privacy-preserving MAS safeguarding as graph federated learning and instantiate FGLGuard: each operator fits an edge-featured graph attention detector on its own judge-labeled episode graphs and shares only model updates. The method couples a proximal local objective for non-IID clients, domain-balanced aggregation, over-refusal-constrained threshold calibration, corroborated upstream scoring, and a guarded rewrite for blocked answers. Federation is not optional: off-the-shelf transfer collapses under distribution shift (AUROC 0.51 to 0.70 only after in-domain retraining), so a deployable guard must adapt on each site's private traces. On Agent-SafetyBench, R-Judge, and AgentDojo, federated FGLGuard exceeds the in-domain centralized ceiling on all three benchmarks without pooling any data---where unsupervised anomaly guards and local-only training fail. One guard federated across four different-domain operators comes within 0.03 AUROC of multi-domain centralization, while any single-domain guard collapses on the others. Live FGLGuard cuts AgentDojo's ground-truth attack-success rate by 43% at near-unguarded utility, zero API cost, and negligible capability loss.

Jinxi Yu, Eric Hanchen Jiang, Levina Li +6

University of California, Los Angeles

Multi-Agent LLM Systems Federated Learning Aggregation Federated Graph Learning LLM Agent Safety

Ask PDF

Oct 1, 2026

Federated Agent Optimization

Large language model (LLM) agents increasingly operate in private environments and accumulate valuable experience from task execution, tool use, feedback, and local knowledge. Yet such experience is distributed across organizations and cannot be directly shared because of privacy and proprietary constraints. Conventional federated learning is insufficient for this setting, as agent capabilities extend beyond model parameters to memory, tools, rewards, skills, and structured knowledge. In this paper, we formulate Federated Agent Optimization (FAO), which studies how distributed agents can collaboratively improve through controlled information exchange while keeping raw data, complete trajectories, and private knowledge local. We define FAO as a multi-objective problem balancing agent utility, privacy leakage, and communication cost, and organize its optimization space across policy, memory, tool use, reward, and structured knowledge and skills. We further characterize how private experience can be abstracted, protected, aggregated, and adapted into transferable capabilities, providing a unified view of how agents can benefit from one another without direct experience sharing. Finally, we identify the key challenges of FAO and outline several promising directions for future research toward trustworthy federated agent systems.

Qiang Yang, Zhiqiang Kou, Xueyi Zhang +6

The Hong Kong Polytechnic University, Hong Kong SAR, China · National University of Singapore, Singapore · The University of Tokyo, Japan +2

Privacy-Preserving Federated Learning LLM Agent Training Federated Learning

Ask PDF

Abstract

Multi-Agent LLM Systems

Explore similar work

Agentic Federated Learning: The Future of Distributed Training Orchestration

Privacy-Preserving Topology-Guided Safety for LLM-Based Multi-Agent Systems via Federated Graph Learning

Federated Agent Optimization