Federated Knowledge Distillation

Ask about this topic

Overview

Federated knowledge distillation studies how clients can learn from one another when their data, model architectures, and computing resources differ, while keeping raw data local. Common approaches exchange logits or class probabilities, distill features or prototypes, or use lightweight proxy models and adapters; communication-efficient variants share compressed or hard-label predictions. These methods are applied to non-IID and long-tailed classification, personalized and multimodal learning, edge systems, and increasingly language and vision-language models. A central challenge is deciding which peers or proxy examples to trust: unreliable knowledge can amplify bias or negative transfer, while reducing communication and preserving privacy can limit the information available for learning.

Latest papers

CardsList

36 papers5 papers in the last four weeks, down 37% on the four weeks before. 0.0% of all new papers.

  1. Collaborative Adaptive Curriculum for Progressive Knowledge Distillation

    Mar 19, 2026Jing Liu, Zhenchao Ma, Han Yu +5Federated Knowledge DistillationKnowledge Distillation

    Fudan University · The University of British Columbia · Duke Kunshan University +2

  2. A Multi-Prototype-Guided Federated Knowledge Distillation Approach in AI-RAN Enabled Multi-Access Edge Computing System

    Mar 10, 2026Luyao Zou, Hayoung Oh, Chu Myaet Thwal +3Federated Knowledge DistillationFederated Learning

    College of Computing and Informatics, Sungkyunkwan University, Republic of Korea · Department of Computer Science and Engineering, Kyung Hee University, Yongin-si, Gyeonggi-do, 17104, Republic of Korea · Department of Information and Communication Engineering, Noakhali Science and Technology University, Noakhali-3814, Bangladesh +2

  3. Geometric Knowledge-Assisted Federated Dual Knowledge Distillation Approach Towards Remote Sensing Satellite Imagery

    Mar 8, 2026Luyao Zou, Fei Pan, Jueying Li +4Federated Knowledge DistillationRemote Sensing Image Understanding

    College of Computing and Informatics, Sungkyunkwan University, South Korea · Department of Computer Science and Engineering, University of Michigan, United States · Konkuk Aerospace Design-Airworthiness Institute, Konkuk University, Seoul, 05029, South Korea +3

  4. FedAFD: Multimodal Federated Learning via Adversarial Fusion and Distillation

    Mar 5, 2026Min Tan, Junchao Ma, Yinfu Feng +6Multimodal Federated LearningFederated Learning

    Zhejiang Key Laboratory of Space Information Sensing and Transmission, Hangzhou Dianzi University · Laboratory of Complex Systems Modeling and Simulation, School of Computer Science and Technology, Hangzhou Dianzi University · Alibaba International Digital Commerce Group +1

  5. Federated Reasoning Distillation Framework with Model Learnability-Aware Data Allocation

    Feb 21, 2026Wei Guo, Siyuan Lu, Xiangdong Ran +8Reasoning DistillationFederated Knowledge Distillation

    Beihang University, Beijing 100191, China · Heilongjiang University, Harbin 150080, China · Beijing University of Civil Engineering and Architecture, Beijing 100044, China +3

  6. SCENE OTA-FD: Self-Centering Noncoherent Estimator for Over-the-Air Federated Distillation

    Feb 17, 2026Hao Chen, Zavareh BozorgaslFederated Learning AggregationOver-the-Air Federated Learning

    Department of Electrical and Computer Engineering, Boise State University, Boise, ID 83712 USA.