Cross-Modal Knowledge Distillation

Overview

Cross-modal knowledge distillation studies how a model using one modality, such as images, audio, text, or medical signals, can teach a model that uses another. A common approach trains a rich multimodal teacher and transfers its knowledge through feature, logit, relational, or contrastive losses, so the student can work with fewer or cheaper inputs at deployment. Researchers also use attention and token-level guidance, on-policy distillation for reasoning, and distributional alignment when paired examples are unavailable; evaluations span recognition, retrieval, detection, segmentation, and multimodal reasoning. A central challenge is deciding what knowledge transfers reliably across different representations: noisy or mismatched teachers can cause negative transfer, while overly strong alignment may erase useful modality-specific information.

Latest papers

CardsList

129 papers10 papers in the last four weeks, down 37% on the four weeks before. 0.1% of all new papers.

  1. MambaLoc: Lightweight Indoor Localization Using Cross-Modal Knowledge Distillation

    Nov 7, 2026Mohab Bahnassy, Omar Saqr, Hamada Rizk +1Wireless LocalizationCross-Modal Knowledge Distillation

  2. RadarDINO: Distilling a Visual Foundation Model into mmWave Radar for Human Mesh Reconstruction

    Nov 6, 2026Furkan Mert Algan, Zhifan Ni, Eckehard SteinbachHuman Mesh RecoverymmWave Radar Sensing

  3. Aligning Policy Encoders with a Shape Teacher

    Nov 5, 2026Madelon Bernardy, Mohammad Mahdi Derakhshani, David M Knigge +2Visuomotor Policy LearningRobotic Manipulation

  4. OmniOPD: Stable On-Policy Distillation for Joint Audio-Video Generation

    Nov 2, 2026Zihao Fan, Xin Lu, Jie Huang +3Audio-Video GenerationOn-Policy Distillation

  5. Learning from high-resolution 2D images to reconstruct 3D microstructures

    Nov 1, 2026Wenbo Sui, Lars P. Mikkelsen, Jens W. Andreasen +33D ReconstructionCross-Modal Knowledge Distillation

  6. Band2Net: Frequency-to-Network Transport Distillation for fMRI-Privileged, EEG-Only Session-Condition Decoding

    Nov 1, 2026Nishi Kanta Paul, Md Shihabul Islam Shovo, Dwarikanath MahapatraEEG DecodingCross-Modal Knowledge Distillation

  7. Soil Moisture Estimation in Precision Agriculture: A Knowledge Distillation Approach

    Oct 29, 2026Mohammed Rakib, Vrinda Nagula, Arunkumar BagavathiPrecision AgricultureAgricultural Remote Sensing

  8. Learning Fine-Grained Visual Classification with Text as Privileged Information via Multimodal Distillation

    Oct 23, 2026Aishwarya Venkataramanan, Christian Ickler, Joachim DenzlerFine-Grained Image ClassificationLearning Using Privileged Information

  9. Revisiting Distribution Matching for State-to-Vision Policy Distillation in Laparoscopic Robotic Manipulation

    Oct 22, 2026Mingyu Sheng, Jianan Fan, Dongnan Liu +3Robotic ManipulationVision-Based Robot Control

  10. Context-Aware Topological Distillation Graph Network for Asymmetric EEG-Eye Movement Emotion Recognition

    Oct 19, 2026Peijun Shi, Xiujin Zhu, Zhenguo Lao +4Multimodal Emotion RecognitionEEG Emotion Recognition

  11. Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling

    Oct 14, 2026Jing-Xuan Zhang, Genshun Wan, Jianqing Gao +3Audio-Visual Representation LearningCross-Modal Knowledge Distillation

  12. Teaching PPG How not Who: Fixed-Effects Distillation from ECG

    Oct 7, 2026Zhongli Wu, Zhuangzhi Gao, Yuankai Wang +3Cross-Modal Knowledge DistillationPhotoplethysmography

  13. RT-DETR-World: Transferring Rich LLM Semantics to Real-Time Open-Vocabulary Detection

    Oct 7, 2026Yupeng Zhang, Ziyi Zhao, Juntao Cheng +4Open-Vocabulary Object DetectionReal-Time Object Detection

  14. AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions

    Oct 6, 2026Sergei Kurchev, Iaroslav Kolomiets, Miguel Altamirano Cabrera +2Cross-Modal Knowledge DistillationCross-Embodiment Robot Learning

  15. KVE-KD: Key Visual Evidence-Guided Knowledge Distillation for Vision-Language Models

    Oct 2, 2026Jianbin Zhang, Xin Sun, Shanwen Wang +2VLM DistillationCross-Modal Knowledge Distillation

  16. GRAFT: Growing Agglomerative Foundation Models via Continual Teacher Distillation

    Oct 1, 2026Zhenghao Zhao, Chi Zhang, Qingshuang Chen +1Multi-Teacher Knowledge DistillationVision Foundation Models

  17. Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation

    Oct 1, 2026Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly +2Cross-Modal Knowledge DistillationMultimodal Embedding

  18. MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models

    Sep 30, 2026Yanshu Li, Jiaqian Li, Canran Xiao +3Cross-Modal Knowledge DistillationMultimodal ICL

  19. Codebook-Guided Cross-Modal Knowledge Distillation for Structurally Heterogeneous Features

    Sep 29, 2026Dae Ung Jo, Jongin Lim, YoungJoon Yoo +1Cross-Modal Knowledge DistillationCross-Modal Alignment

  20. From Static to Dynamic: On-Policy Distillation from Image to Video Diffusion Models

    Sep 28, 2026Bingqing Jiang, Li Luo, Zichao Yu +3Cross-Modal Knowledge DistillationVideo Diffusion Models

  21. Can Protein-Derived Knowledge Improve Pathology Foundation Models?

    Sep 27, 2026Di Zhang, Zhangpeng Gong, Jiashuai Liu +10Pathology Foundation ModelsComputational Pathology

  22. Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigation

    Sep 27, 2026Zhihao Chen, Yiyuan Ge, Ziyang Wang +2Vision-Language NavigationVLM Distillation

  23. Cross-Modal Knowledge Distillation for Acoustic Pedestrian Detection

    Sep 24, 2026Yonghyun Kim, Chaeyeon Han, Sancho Gatungay +2Cross-Modal Knowledge DistillationSelective Knowledge Distillation

  24. Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation

    Sep 15, 2026Meisen Wang, Zhiqiang Tian, Wei Bao +3Event-Based VisionCross-Modal Knowledge Distillation

  25. AlignUS: MRI-Guided Ultrasound Representation Learning for ALS Classification from Tongue Images

    Sep 14, 2026Kadija Abdel Ghader, Emani Babe, Lorenzo Pettinari +2Cross-Modal Knowledge DistillationUltrasound Imaging

  26. Weakly Supervised Spatial Grounding for Discriminative Attention-Based Ultrasound-Histopathology Alignment in Prostate Cancer Grading

    Sep 14, 2026Obed Korshie Dzikunu, Emma Willis, Mohammad Mahdi Abootorabi +7Computational PathologyCross-Modal Knowledge Distillation

  27. Freezing of Gait Prediction Under Spatial Occlusion: An IMU-Supervised Cross-Modal Distillation Approach

    Sep 9, 2026Chandan Biswas, Aryan Singh, Anabik PalCross-Modal Knowledge DistillationGait Analysis