Federated Knowledge Distillation

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 15

All topics

Distributed Subliminal Learning: Replacing Model Updates with Random-Carrier Outputs
Oct 4, 2026Dario Fenoglio, Gabriele Dominici, Martin Gjoreski +1Communication-Efficient Distributed Training Federated Knowledge Distillation
Università della Svizzera italiana (USI), Lugano, Switzerland · MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA
FedSocket: Recipient-Executable Knowledge Exchange for Heterogeneous Multimodal Federated Learning
Sep 29, 2026Xinyuan ZhaoCross-Modal Knowledge Distillation Federated Knowledge Distillation
Sun Yat-sen University
Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
Sep 15, 2026Polycarpo Souza Neto, José Mairton Barros da Silva Júnior, Charles Casimiro CavalcanteFederated Knowledge Distillation Non-IID Federated Learning
Universidade Federal do Cear´a, Fortaleza, CE, Brazil · Uppsala University, Uppsala, Sweden
Robust Decentralized Federated Distillation via Multi-Modality Knowledge Collaboration
Sep 7, 2026Xiao Ma, Hong Shen, Hui Tian +2Byzantine-Robust Federated Learning Federated Knowledge Distillation
Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR, China · School of Engineering and Technology, Central Queensland University, Australia · School of Information and Communication Technology, Griffith University, Australia
CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learning
Aug 31, 2026Baraa Bilbeisi, Mengchen Fan, Baocheng Geng +1Federated Learning Aggregation Federated Knowledge Distillation
Dept. of Electrical & Computer Engineering · Dept. of Computer Science
Out-of-Distribution Federated Distillation with Domain-Aware Proxy
Aug 9, 2026Jiahao Xiao, Jiangming LiuOOD Generalization Federated Knowledge Distillation
School of Information Science and Engineering, Yunnan University, China
EFFEKT: Efficient Federated Knowledge Transfer to Foundation Models
Aug 8, 2026Matteo Caligiuri, Francesco Barbato, Pietro Zanuttigh +1Federated Knowledge Distillation Cross-Architecture Knowledge Distillation
Department of Electrical & Computer Engineering, Northeastern University, Boston (MA), United States · Department of Information Engineering, University of Padua, Padua, Italy
FedAgentKE: Federated Semantic Knowledge Evolution for Heterogeneous Agents
Jul 23, 2026Weihao Li, Jun Bai, Ziyang SongFederated Knowledge Distillation LLM Agent Training
McCormick School of Engineering, Northwestern University · School of Computer Science, McGill University · Mila – Quebec AI Institute +1
TallyTrain: Communication-Efficient Federated Distillation
Jun 30, 2026Radhakrishna Achanta, Will ReedFederated Learning Aggregation Communication-Efficient Distributed Training
Cisco Systems Inc.
Benchmarking Federated Learning and Knowledge Distillation for Point Cloud Classification
Jun 30, 2026Aizierjiang AiersilanData Leakage Point Cloud Classification
University of Macau
Matching Rates and Optimal Allocation for Federated Probe-Logit Distillation under Heterogeneous Bandwidth Budgets
May 28, 2026Prasanjit Dubey, Xiaoming HuoFederated Learning Aggregation Communication-Efficient Distributed Training
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332, U.S.A.
Optimized Federated Knowledge Distillation with Distributed Neural Architecture Search
May 20, 2026Chaimaa Medjadji, Sylvain Kubler, Yves Le Traon +3Communication-Efficient Distributed Training Federated Knowledge Distillation
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg · Blekinge Institute of Technology, Sweden · ADSLabs, Umea University, Sweden
BiFedKD: Bidirectional Federated Knowledge Distillation Framework for Non-IID and Long-Tailed ECG Monitoring
May 14, 2026Zixuan Shu, Tiancheng Cao, Hen-Wei HuangLong-Tail Learning Communication-Efficient Distributed Training
School of Electrical and Electronic Engineering, Nanyang Technological University, Republic of Singapore · Lee Kong Chian School of Medicine, Nanyang Technological University, Republic of Singapore
Beyond Parameter Aggregation: Semantic Consensus for Federated Fine-Tuning of LLMs
May 12, 2026Amr Abourayya, Jens Kleesiek, Michael KampLLM Fine-Tuning Federated Knowledge Distillation
Lamarr Institute for ML and AI, Technical University Dortmund · Institute for AI in medicine (IKIM),University Hospital Essen
FedeKD: Energy-Based Gating for Robust Federated Knowledge Distillation under Heterogeneous Settings
May 7, 2026Quang-Huy Nguyen, Jiaqi Wang, Wei-shinn KuFederated Knowledge Distillation Knowledge Distillation
Department of Computer Science and Software Engineering Auburn University Auburn, AL 36849

Oct 4, 2026, cs.LGJ/K move · Enter open · S save

Distributed Subliminal Learning: Replacing Model Updates with Random-Carrier Outputs

Dario Fenoglio, Gabriele Dominici, Martin Gjoreski, Marc Langheinrich

Università della Svizzera italiana (USI), Lugano, Switzerland · MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA

Communication-Efficient Distributed Training Federated Knowledge Distillation Federated Learning

Open paper AskPDF arXiv

Collaborative learning typically exchanges model parameters: federated clients communicate updates, while independently adapted foundation models are combined by exchanging adapters or checkpoints. This makes communication scale with model size and requires local specializations to be reconciled in weight space, where interference is common. We ask whether knowledge can instead be shared through model behavior on task-unrelated inputs. We introduce Distributed Subliminal Learning (DSL), a collaborative learning primitive in which participants adapt a common model locally, probe it with task-unrelated inputs, and transmit only the resulting carrier outputs. A coordinator pools these outputs and distills them into a shared model. The primitive supports one-shot foundation-model composition through carrier completions and iterative federated learning through carrier logits, without transmitting model updates or requiring task-related proxy data. In LLM composition, compared with LoRA averaging, DSL achieves higher preference retention (94.56% vs. 87.76%) and a larger GSM8K gain over the base model (22.0 vs. 0.6 points), while reducing upload by 30.6-49.0

\times

. In federated classification, DSL reaches 96.83% on MNIST with 8.9

\times

less uplink than FedAvg and provides lower-communication operating points on CIFAR-10 and Tiny ImageNet. These results establish random-carrier outputs as a practical communication primitive for knowledge sharing across distinct collaborative learning paradigms.

Momentum

Related topics

Latest papers 15