Dataset Distillation

Ask about this topic

Momentum

28 papers in the last four weeks, up 56% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 241

All topics
CardsList

  1. DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

    Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Distribution Matching DistillationDataset Distillation

    Texas A&M University · Intelligent Creation, ByteDance

  2. Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation

    Oct 1, 2026Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly +2Multimodal EmbeddingsOmni-Modal

    Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)

  3. DecomVoxel: Harnessing 3D-Native Priors with Guided In-situ Denoising Optimization for Decompositional Scene Reconstruction

    Oct 1, 2026Junfeng Ni, Zirui Zhou, Yixin Chen +5Scene ReconstructionVoxel

    Tsinghua University, State Key Laboratory of General Artificial Intelligence, BIGAI, China · Peking University, China

  4. Distribution Matching Distillation for Continuous Diffusion Language Models

    Sep 30, 2026Paul Le Van Kiem, Dario Shariatian, Umut Simsekli +1Diffusion Language ModelsDiffusion Models

    Inria, PSL Research University · Cohere · CMAP, Ecole Polytechnique

  5. Enhancing Autoregressive Video Generation via Representation Adversarial Distillation

    Sep 30, 2026Fangyu Lin, Xingtong Ge, Lunjie Zhu +8Autoregressive Video GenerationVideo Generation

    The Hong Kong University of Science and Technology · Vivix Group Limited · Zhejiang University

  6. Learning to Reason with Compressed Context: Ground-Truth-Free Adaptation of OmniLLMs via Self-Distillation

    Sep 30, 2026Jianghao Wang, Ke Meng, Jian Li +7Large Language Model AdaptationDataset Distillation

    School of Computer Science, Wuhan University · Didi Chuxing

  7. Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Construction

    Sep 30, 2026Xiao Cui, Mo Zhu, Yulei Qin +3Dataset DistillationOn-Policy

    University of Science and Technology of China · Zhejiang University · Independent Researcher

  8. Diffusable Latents from Structure-Agnostic Distillation

    Sep 30, 2026Adrien Ramanana Rahary, Nicolas Dufour, Patrick Pérez +1Dataset DistillationVideo Latents

    LIGM, ENPC, IP Paris, CNRS, UGE · Kyutai

  9. Uncertainty-Aware Consistency Distillation for Few-Step Video Generation

    Sep 30, 2026Lingyu Liu, Yaxiong Wang, Li Zhu +1Dataset DistillationAdversarial Training

    School of Software Xi’an Jiaotong University Xi’an, China. · School of Computer and Information Science Hefei University of Technology Hefei, China. · Faculty of Science and Technology, and Institute of Collaborative Innovation University of Macau Macau, China.

  10. VISTA: Internalizing Collective Visual Experience via On-Policy Distillation for Active Multimodal Agents

    Sep 29, 2026Zheng Jiang, Houde Qian, Yiming Chen +5Multimodal AgentsMultimodal Learning

    Tsinghua University

  11. VAA-CSEC: Vote-guided Advantage Allocation for Chinese Semantic Error Correction

    Sep 29, 2026Yitong Han, Nankai Lin, Juan Luo +3Grammatical Error CorrectionCorrection

    School of Information Science and Technology, Guangdong University of Foreign Studies · Guangdong Engineering Research Center of Data Security Governance and Privacy Computing · College of Computer Science and Technology, National University of Defense Technology

  12. What Makes High-Magnification Knowledge Transferable? A Study of Cross-Resolution Distillation in Whole-Slide Imaging

    Sep 29, 2026Zhiyuan Yang, Jiahao Cheng, Mahdi S. HosseiniWhole-Slide ImagesCross-Task Knowledge Transfer

    Department of Computer Science and Software Engineering (CSSE) Concordia University Montreal, Canada · Department of Computer Science and Software Engineering (CSSE) Concordia University Montreal, Canada Mila – Quebec AI Institute Montreal, Canada

  13. The Composition Gap in Dataset Distillation

    Sep 28, 2026Guang Li, Takahiro Ogawa, Miki HaseyamaDiffusion-Based Dataset DistillationProbe-Logit Distillation

    Hokkaido University

  14. PSM: Dataset Distillation Based on Precise Statistical Matching by Difficulty

    Sep 28, 2026Hongxu Ma, Guang Li, Shijie Wang +6Diffusion-Based Dataset DistillationDataset Distillation

    Zhejiang University · Hokkaido University · The University of Queensland +3

  15. Waggle: Learning One Anonymous Local Law for Self-Organizing LLM Swarms

    Sep 28, 2026Mingxi Zou, Wei Zhu, Zhuo Wang +4Multi-Agent CoordinationGlobal Coordination

    Shanghai Academy of AI for Science (SAIS) · AI3 Institute, Fudan University · Yunnan University

  16. TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models

    Sep 27, 2026Shuning Wang, Zhiheng Wu, Xun Zhou +8Multimodal Reasoning BenchmarksDataset Distillation

    Zhejiang University · Hong Kong University of Science and Technology · Harbin Institute of Technology +1

  17. Geometry-Conditioned Visual Place Recognition in Natural Environments

    Sep 23, 2026Walter Nedov, Saimunur Rahman, Kavindie Katuwandeniya +3Visual Place RecognitionDataset Distillation

    CSIRO Robotics, CSIRO, Australia · Queensland University of Technology, Australia

  18. MIND the Gap: A Geographic Implicit Neural Representation with Adjustable Spatial Scale

    Sep 21, 2026Isaac Corley, Arjun Rao, Esther Rolf +7Neural RepresentationsText Embeddings

    Taylor Geospatial · University of British Columbia · Vector Institute +7

  19. Efficient LLM Distillation for Bangladesh Legal Context: A Smartphone-Compatible Retrieval-Augmented Generation Model

    Sep 21, 2026MD. Nafis Kamal, Mahadi Hasan Fahim, Talha Ridwan +4BanglaDataset Distillation

    Computer Science and Engineering BRAC University Dhaka, Bangladesh

  20. HapticWAM: Distilling Imagined Touch into a World-Action Model without Inference-Time Tactile Sensing

    Sep 20, 2026Mikhail Sannikov, Ilya Mikhalchuk, Konstantin Gubernatorov +3Tactile World ModelTactile

    Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Russia

  21. StageGuard: Learning Stage Transitions for Long-Horizon Robot Tasks via Agentic Distillation

    Sep 17, 2026Jinbang Huang, Yuanzhao Hu, Zhiyuan Li +6Robot StateRobot Policies

    Huawei Noah’s Ark Lab · University of British Columbia · University of Toronto +1

  22. Coupled Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback

    Sep 15, 2026Haichen Hu, Yuheng Zhang, David Simchi-LeviDataset DistillationTeacher

    MIT · UIUC

  23. Verbalizing Subliminal Learning Effects Using Text Optimization

    Sep 15, 2026Nathan Hu, Sanmi Koyejo, Christopher PottsMultiple Prompt Learning FrameworksDataset Distillation

    Stanford University

  24. CIDERS: Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization

    Sep 14, 2026Victor H. Chen, Hairui Yu, Stella K. Chung +1Edge PlatformsLarge Language Model Personalization

  25. X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation

    Sep 12, 2026Haojun Zhang, Yi Zou, Min Chen +7Dataset Distillation

    XPeng Inc.

  26. ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding

    Sep 12, 2026Luca Della Libera, Cem Subakan, Mirco RavanelliNeural Speech CodecsDataset Distillation

    Concordia University · Mila-Quebec AI Institute · Universit´e Laval

  27. RouteBridge: Reliability-Routed Bidirectional Distillation Between Neural Radiance Fields and 3D Gaussian Splatting

    Sep 10, 2026YuanHang Wang, Xin CaoNeural Radiance FieldsBidirectional

    University of Technology Sydney

  28. CASD: Chunk-Aligned Semantic Distillation for Multi-StageRobot Manipulation

    Sep 8, 2026Tinghe Ding, Jiahao Li, He WangAction ChunksRobotic Manipulation

    Ant Group, China

  29. Compressing Streaming Neural Audio Encoders via Latent-Space Distillation

    Sep 3, 2026Prasanth Yadla, Mohammad Samragh, Dongseong Hwang +7Neural Speech CodecsDataset Distillation

    Apple

  30. ALRA: Adaptive Local Relational Alignment for Logit-Based Pre-training Distillation of Autoregressive Language Models

    Sep 3, 2026Quang Hoang Trung, Quang Huu Hieu, Nguyen Van Hoang Phuc +1Dataset DistillationKnowledge Distillation

    VJ Technologies, Da Nang, Vietnam · AJ Technologies, Nagoya, Japan · Vietnam National University, Ho Chi Minh City, Vietnam +1

  31. SelfLift: Accelerating Few-Step Diffusion via Self-Recovering Resolution Transition

    Sep 2, 2026Tingyan Wen, Chenqian Yan, Xurui Peng +4Dataset DistillationSelf

    1Tsinghua University · 2ByteDance

  32. Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation

    Sep 1, 2026R. James Cotton, Divya Joshi, Colleen PeytonWifi-Csi 3D Human Pose Estimation3D Foundation Models

    1Shirley Ryan AbilityLab · Department of Physical Medicine and Rehabilitation, Northwestern University · Department of Physical Therapy and Human Movement Science, Northwestern University +1

  33. Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation

    Sep 1, 2026Zhixuan Liu, Zhichen Dong, Yuyu Fan +2Dataset Distillation

    Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Fudan University

  34. CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learning

    Aug 31, 2026Baraa Bilbeisi, Mengchen Fan, Baocheng Geng +1Federated LearningDataset Distillation

    Dept. of Electrical & Computer Engineering · Dept. of Computer Science

  35. Identity-Conditioned Latent Consistency Distillation for Face Synthesis

    Aug 31, 2026Tiago Kienen Chaves, Bernardo Biesseck, David MenottiIdentity-Conditioned Latent Diffusion ModelsDataset Distillation

    Department of Informatics, Federal University of Paraná, Curitiba, Brazil · Federal Institute of Mato Grosso (IFMT), Pontes e Lacerda, MT, Brazil

  36. Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspective

    Aug 31, 2026Jiaqi Zhang, Zheng Pang, Mengting Li +9Multimodal LearningEmotion Recognition

    Jiangsu University · Griffith University · National University of Singapore +8

  37. Stratified Consistency Distillation for Natural Language Formalization

    Aug 31, 2026Zhichao Hou, Ferhat Erata, Joe Lilien +1LLM Reasoning StrategiesNeuro-Symbolic Framework

    North Carolina State University · Amazon Web Services

  38. MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation

    Aug 30, 2026Yangsong Lan, Renkai Hu, HongKai Zheng +4Dataset DistillationLarge Reasoning Models

    College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · 2The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China

  39. P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation

    Aug 13, 2026Yi Shi, Huichao Xie, Yuqing Wang +7Infrared-Visible Image FusionAdm-Fusion

    Northwestern Polytechnical University, Xi’an, China · Hefei University of Technology, Hefei, China · Rocket Force University of Engineering, Xi’an, China +1

  40. PixSDS: Why Latent SDS Makes Noisy Pixels

    Aug 13, 2026Vsevolod Skorokhodov3D Generative ModelsPixelloop

    EPFL, Lausanne, Switzerland

  41. Training Under Challenge: Executable Certificates and Challenge-Closed Optimality for Neural Networks

    Aug 12, 2026Farhang Yeganegi, Arian Eamaz, Mojtaba SoltanalianNeural Network VerificationFinite-Sample Certificates

    Department of Electrical and Computer Engineering University of Illinois Chicago Chicago, Illinois, USA

  42. LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation

    Aug 12, 2026Zhixin Zhang, Xinke Jiang, Zhibang Yang +5Long-Horizon AgentsAgentic Reinforcement Learning

    School of Computer Science, Peking University, Beijing, China · National Engineering Research Center for Software Engineering, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +2

  43. Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation

    Aug 11, 2026Dazhao Du, Shiyan Du, Jian Liu +8Camera ControlMotion

    The Hong Kong University of Science and Technology · Tencent

  44. Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity

    Aug 11, 2026Junyong Choi, Cheolhyeon Park, Jaehoon ChoVision TransformerKnowledge Distillation

    Hyundai Motor Company · School of Electronics and Avionics Engineering, Korea Aerospace University

  45. Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots

    Aug 10, 2026Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar +2Recent Vision-Language ModelsDataset Distillation

    1Mohamed bin Zayed University of Artificial Intelligence · 2Aalto University

  46. Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation

    Aug 10, 2026Yubo Jiang, Fengying Xie, Zhiguo Jiang +1Unsupervised On-Policy Self-DistillationDataset Distillation

    School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China

  47. Subliminal Learning is Non-Semantic Distillation

    Aug 6, 2026Ethan Hadley, Eren GultepeDataset DistillationSynthetic Data

    Department of Computer Science, Southern Illinois University Edwardsville, Edwardsville, IL, USA.

  48. Spectral Distillation: From Nonlinear Dynamics to Linear State-Space Models

    Aug 5, 2026Liane Galanti, Devan Shah, Shlomo Fortgang +1System IdentificationNonlinear Dynamics

    Princeton University

  49. Efficient Video Dataset Distillation via Cluster-Guided Prototype Blending

    Aug 4, 2026Chongle Ren, Guang Li, Wenbo Huang +3Diffusion-Based Dataset DistillationDataset Distillation

    1Hokkaido University · 2Southeast University

  50. Self-Supervised Representation-Guided Generative Dataset Distillation

    Aug 4, 2026Mingzhuo Li, Guang Li, Linfeng Ye +4Diffusion-Based Dataset DistillationDataset Distillation

    Hokkaido University, Japan · University of Toronto, Canada · University of Tokyo, Japan

  51. Accelerating Human-Aware Robot Trajectory Generation via Diffusion and Consistency Distillation

    Aug 4, 2026Byeong-Il Ham, Hyun-Bin Kim, Kyung-Soo KimRobot TrajectoriesEnd-Effector Pose

  52. TQLite: Multi-LLM Jury Guided Distillation for Real-time MQM Translation Quality Evaluation

    Aug 4, 2026Bhavin Jawade, Cameron R. WolfeMachine Translation QualityLarge Language Model Evaluation

    Netflix

  53. Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model

    Aug 3, 2026Ronglong BaoDataset DistillationQwen3

    DT-Project

  54. Learning What to Remember: Test-Time Training via Context Distillation

    Aug 3, 2026Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4Efficient Long-Context InferenceTest-Time Training

    Princeton University · UC Santa Cruz · Carnegie Mellon University +1

  55. Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning

    Aug 2, 2026Yuqi Li, Yuedong Tan, Huiran Duan +7Cross-Modal DistillationMulti-Object Tracking

    The City College of New York, New York, NY, USA · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria · Julius-Maximilians-Universität Würzburg, Würzburg, Germany +3

  56. Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer

    Aug 2, 2026Zhiwei Chen, Yang Hu, Yuxiang Xiao +7Pathology Foundation ModelsDigital Pathology

    School of Computer Science and Engineering, South China University of Technology, China · School of Computing and Mathematical Sciences, University of Leicester, UK · Leicester Cancer Research Centre, University of Leicester, UK +4

  57. TurboClear: One-Step Object-Effect Removal via Region-Calibrated Distribution Matching and Fusion

    Aug 2, 2026Jiawei Guo, Junxian Li, Yixin Tang +4Video Object RemovalDataset Distillation

    1Shanghai Jiao Tong University · 2Honor Device Co., Ltd · 3Imperial College London

  58. WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA

    Aug 2, 2026Zhihao Zhu, Hanlin Shang, Mingwang Xu +6Autonomous DrivingDiffusion-Based Vision-Language-Actions

    Fudan University · Yinwang Inttelligent Technology Co.,Ltd

  59. LUT: Latent Utility Training for Visual Reasoning

    Aug 1, 2026Jiaxuan Kang, Siyu Chen, Mingda Li +6Latent Visual ReasoningVisual Reasoning

    Tencent PCG

  60. RHEA: Reliability-Harmonized Reconstruction and Assignment for Robust Multimodal-Attributed Graph Clustering

    Aug 1, 2026Yinlin Zhu, Di Wu, Ziyu Han +4Text-Attributed GraphDataset Distillation

    Sun Yat-sen University, Guangzhou, China1 · Shandong University, Weihai, China2 · Beijing Institute of Technology, Beijing, China3