Cross-Modal Knowledge Distillation

Ask about this topic

Overview

Cross-modal knowledge distillation studies how a model using one modality, such as images, audio, text, or medical signals, can teach a model that uses another. A common approach trains a rich multimodal teacher and transfers its knowledge through feature, logit, relational, or contrastive losses, so the student can work with fewer or cheaper inputs at deployment. Researchers also use attention and token-level guidance, on-policy distillation for reasoning, and distributional alignment when paired examples are unavailable; evaluations span recognition, retrieval, detection, segmentation, and multimodal reasoning. A central challenge is deciding what knowledge transfers reliably across different representations: noisy or mismatched teachers can cause negative transfer, while overly strong alignment may erase useful modality-specific information.

Latest papers

CardsList

122 papers16 papers in the last four weeks, up 60% on the four weeks before. 0.1% of all new papers.

  1. Learning Fine-Grained Visual Classification with Text as Privileged Information via Multimodal Distillation

    Oct 23, 2026Aishwarya Venkataramanan, Christian Ickler, Joachim DenzlerFine-Grained Image ClassificationLearning Using Privileged Information

    Computer Vision Group, Friedrich Schiller University Jena

  2. Revisiting Distribution Matching for State-to-Vision Policy Distillation in Laparoscopic Robotic Manipulation

    Oct 22, 2026Mingyu Sheng, Jianan Fan, Dongnan Liu +3Robotic ManipulationVision-Based Robot Control

    School of Computer Science, the University of Sydney, Sydney, NSW, Australia · Institude of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China · Department of Radiology, Brigham and Women’s Hospital, and Harvard Medical School, Boston, MA, United States

  3. Context-Aware Topological Distillation Graph Network for Asymmetric EEG-Eye Movement Emotion Recognition

    Oct 19, 2026Peijun Shi, Xiujin Zhu, Zhenguo Lao +4Multimodal Emotion RecognitionEEG Emotion Recognition

    Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, 50603 Lembah Pantai, Kuala Lumpur, Malaysia. · Department of Neurosurgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430022, China · Hubei Provincial Clinical Research Center for Brain Computer Interface, Huazhong University of Science and Technology, Wuhan, Hubei 430022, China.

  4. How Much Physics Can Vision Learn from Audio? Probing and Distilling Damping Structure

    Oct 15, 2026Zainab Afolabi, Vadim Selyutin, Zhao Gao +2Audio-Visual Representation LearningCross-Modal Knowledge Distillation

    Amazon Web Services

  5. Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling

    Oct 14, 2026Jing-Xuan Zhang, Genshun Wan, Jianqing Gao +3Audio-Visual Representation LearningCross-Modal Knowledge Distillation

    School of Artificial Intelligence and Computer Science, Shaanxi Normal University · iFLYTEK Research, iFLYTEK Co., Ltd. · Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism

  6. Teaching PPG How not Who: Fixed-Effects Distillation from ECG

    Oct 7, 2026Zhongli Wu, Zhuangzhi Gao, Yuankai Wang +3Cross-Modal Knowledge DistillationPhotoplethysmography

    Liverpool Centre for Cardiovascular Science, University of Liverpool, UK · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Liverpool Heart and Chest Hospital, Liverpool, UK +1

  7. RT-DETR-World: Transferring Rich LLM Semantics to Real-Time Open-Vocabulary Detection

    Oct 7, 2026Yupeng Zhang, Ziyi Zhao, Juntao Cheng +4Open-Vocabulary Object DetectionReal-Time Object Detection

    Tianjin University. · Shenzhen University of Advanced Technology. · Shanghai Jiaotong University. +1

  8. AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions

    Oct 6, 2026Sergei Kurchev, Iaroslav Kolomiets, Miguel Altamirano Cabrera +2Cross-Modal Knowledge DistillationCross-Embodiment Robot Learning

    Skolkovo Institute of Science and Technology (Skoltech), Bolshoy Boulevard 30, bld. 1, Moscow 121205, Russia. · MWS R&D Center.

  9. Multimodal Knowledge Distillation for Gastric Adenocarcinoma Classification from Whole-Slide Images

    Oct 6, 2026Shrihari Dumbre, Bikash SantraCross-Modal Knowledge DistillationMultimodal Classification

    Indian Institute of Technology, Jodhpur

  10. KVE-KD: Key Visual Evidence-Guided Knowledge Distillation for Vision-Language Models

    Oct 2, 2026Jianbin Zhang, Xin Sun, Shanwen Wang +2VLM DistillationCross-Modal Knowledge Distillation

    Faculty of Data Science, City University of Macau, SAR Macao, China

  11. GRAFT: Growing Agglomerative Foundation Models via Continual Teacher Distillation

    Oct 1, 2026Zhenghao Zhao, Chi Zhang, Qingshuang Chen +1Multi-Teacher Knowledge DistillationVision Foundation Models

    University of Illinois Chicago · Amazon

  12. Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation

    Oct 1, 2026Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly +2Cross-Modal Knowledge DistillationMultimodal Embedding

    Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)

  13. MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models

    Sep 30, 2026Yanshu Li, Jiaqian Li, Canran Xiao +3Cross-Modal Knowledge DistillationMultimodal ICL

    Brown University · University of Alabama at Birmingham · Hanyang University

  14. FedSocket: Recipient-Executable Knowledge Exchange for Heterogeneous Multimodal Federated Learning

    Sep 29, 2026Xinyuan ZhaoCross-Modal Knowledge DistillationFederated Knowledge Distillation

    Sun Yat-sen University

  15. Codebook-Guided Cross-Modal Knowledge Distillation for Structurally Heterogeneous Features

    Sep 29, 2026Dae Ung Jo, Jongin Lim, YoungJoon Yoo +1Cross-Modal Knowledge DistillationCross-Modal Alignment

    Kyungpook National University · AX/PI Center, Samsung Electronics · Chung-Ang University, SNUAILAB +1

  16. From Static to Dynamic: On-Policy Distillation from Image to Video Diffusion Models

    Sep 28, 2026Bingqing Jiang, Li Luo, Zichao Yu +3Cross-Modal Knowledge DistillationVideo Diffusion Models

    The University of Hong Kong · Cornell University

  17. Can Protein-Derived Knowledge Improve Pathology Foundation Models?

    Sep 27, 2026Di Zhang, Zhangpeng Gong, Jiashuai Liu +10Pathology Foundation ModelsComputational Pathology

    School of Computer Science and Technology, Xi’an Jiaotong University · Tsinghua Shenzhen International Graduate School, Tsinghua University · University of Cambridge +4

  18. Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigation

    Sep 27, 2026Zhihao Chen, Yiyuan Ge, Ziyang Wang +2Vision-Language NavigationVLM Distillation

    School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications, Beijing, China · School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China · School of Computer Science and Digital Technologies, Aston University, Birmingham, U.K.

  19. Cross-Modal Knowledge Distillation for Acoustic Pedestrian Detection

    Sep 24, 2026Yonghyun Kim, Chaeyeon Han, Sancho Gatungay +2Cross-Modal Knowledge DistillationSelective Knowledge Distillation

    Music Informatics Group, Georgia Institute of Technology, USA · Center for Urban Resilience and Analytics, Georgia Institute of Technology, USA · College of Computing, Georgia Institute of Technology, USA

  20. Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation

    Sep 15, 2026Meisen Wang, Zhiqiang Tian, Wei Bao +3Event-Based VisionCross-Modal Knowledge Distillation

    School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China · National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an 710049, China · BNRist, THUIBCS, BLBCI, School of Software, Tsinghua University, Beijing 100084, China +2

  21. AlignUS: MRI-Guided Ultrasound Representation Learning for ALS Classification from Tongue Images

    Sep 14, 2026Kadija Abdel Ghader, Emani Babe, Lorenzo Pettinari +2Cross-Modal Knowledge DistillationUltrasound Imaging

    Unité de Recherche en Systèmes Intelligents Avancés (URSIA), Institut Supérieur du Numérique (SupNum), Nouakchott, Mauritanie · University of Basel, Basel, Switzerland

  22. Weakly Supervised Spatial Grounding for Discriminative Attention-Based Ultrasound-Histopathology Alignment in Prostate Cancer Grading

    Sep 14, 2026Obed Korshie Dzikunu, Emma Willis, Mohammad Mahdi Abootorabi +7Computational PathologyCross-Modal Knowledge Distillation

    Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada · Vector Institute, Toronto, Canada · School of Computing, Queen’s University, Kingston, Canada +3

  23. Freezing of Gait Prediction Under Spatial Occlusion: An IMU-Supervised Cross-Modal Distillation Approach

    Sep 9, 2026Chandan Biswas, Aryan Singh, Anabik PalCross-Modal Knowledge DistillationGait Analysis

    NeuroAI Fusion Labs Kolkata, India · Indian Institute of Science Education and Research Berhampur, Odisha

  24. CASD: Chunk-Aligned Semantic Distillation for Multi-StageRobot Manipulation

    Sep 8, 2026Tinghe Ding, Jiahao Li, He WangCross-Modal Knowledge DistillationLanguage-Conditioned Robot Manipulation

    Ant Group, China

  25. Supervised Cross-Modal Feature Alignment for Zero-Wearable Freezing of Gait Detection in Parkinsonism

    Sep 8, 2026Aryan Singh, Chandan BiswasCross-Modal Knowledge DistillationCross-Modal Alignment

    NeuroAI Fusion Labs Kolkata, India

  26. Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspective

    Aug 31, 2026Jiaqi Zhang, Zheng Pang, Mengting Li +9Missing-Modality LearningMemory-Augmented Neural Networks

    Jiangsu University · Griffith University · National University of Singapore +8

  27. DISTAL: Distillation and Self-Supervised Pretraining for Structure-Agnostic Materials Property Prediction

    Aug 30, 2026Weiran Wang, Xintong Huo, Yueying Wang +7Materials Property PredictionCompositional Representation Learning

    College of Computer Science and Technology, Jilin University, Changchun 130012, China. · Key Laboratory of Symbolic ComputaƟon and Knowledge Engineering of Ministry of EducaƟon, Jilin University, Changchun 130012, China. · School of ArƟficial Intelligence, Jilin University of Chemical Technology, Jilin 130000, PR China. +1

  28. LLaVAFlow: Preserving Latent Alignment Flow for Parameter-Efficient Multimodal Fine-Tuning

    Aug 27, 2026Muyao Yuan, Muyan Jiao, Jiangyong Ying +5Multimodal Instruction TuningCross-Modal Alignment

    MOEKLINNS, and School of Computer Science and Technology Xi’an Jiaotong University Xi’an, China · E-surfing Vision Technology Co., Ltd China Telecom Hangzhou, China · China Telecom Xi’an, China

  29. CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

    Aug 20, 2026Eunsoo Im, Junghun Suh, Gyeonggwan Lee +1Point Cloud RegistrationPoint Cloud Learning

  30. ViTexSZ: Heterogeneous Vision-Text Knowledge Distillation for EEG Seizure Detection

    Aug 19, 2026Chenxi Liu, Mingzhao Li, Yicong Liu +4EEG Seizure DetectionElectroencephalography

    Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong SAR · The First Affiliated Hospital, Sun Yat-sen University, China · The Hong Kong Polytechnic University, Hong Kong SAR +2