cs.LGNov 10, 2026

FedVP: Federated Learning across Heterogeneous Models by Pooling Locally Trained Verifiers

Authors: Yifan Li, Xi Chen

Organizations: University of Bath · Artificial Intelligence Innovation and Incubation Institute, Fudan University

Abstract

Conventional federated learning relies on the shared model backbone to aggregate client updates, limiting collaboration among institutions that deploy large language models (LLMs) from different families and at different scales. We propose Federated Verifier Pool (FEDVP), a new federated learning paradigm that enables heterogeneous clients to collaborate through shared verification rather than parameter aggregation. Each client trains a verifier using local data and responses generated by its chosen LLM backbone. The server pools these independently trained verifiers and combines their judgments to select answers without merging model parameters. This allows both LLM backbones and verifiers to differ in architecture and size, while keeping the LLM backbones frozen and deriving collaborative gains from improved answer selection. We evaluate FEDVP across five public medical question-answering datasets treated as federated clients, using Qwen and LLaMA backbones ranging from 3B to 14B parameters. FEDVP consistently improves performance across all five clients and outperforms the strongest conventional federated baselines by 1.46-5.79 percentage points in mean accuracy.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. FediLoRA: Practical Federated Fine-Tuning of Foundation Models Under Missing-Modality Constraints

    Sep 1, 2025Lishan Yang, Wei Emma Zhang, Nam Kha Nguygen +4Federated Fine-TuningFederated Learning Aggregation

  2. FedReason: Large Language Models as Weight-Space Meta-Aggregators for Heterogeneous Federated Learning

    Nov 5, 2026Ali Mohammadi Ruzbahani, Abbas Yazdinejad, Hadis KarimipourFederated Learning AggregationAdaptive Aggregation in Federated Learning

  3. Federated Prompt Learning: A Unified Framework, Empirical Analysis, and Future Directions

    Aug 14, 2026Qinglin Yang, Chen Qiu, Hongyuan Zhang +3Federated LearningPrivacy-Preserving Language Models