CoT Distillation

CoT: Chain-of-Thought

Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 57

All topics
CardsList

  1. Learning from Think-Mode Advantage via On-Policy Distillation

    Sep 29, 2026Wanqi Ren, Jianxiang Wang, Danxuan Liu +2CoT DistillationLanguage Model Distillation

    ByteDance, China

  2. Distilling Visual Reasoning into Text Space

    Sep 28, 2026Wenhan Yang, Nilay Naharas, Ali Payani +1CoT DistillationVLM Distillation

    University of California, Los Angeles · Cisco Research

  3. What Shared Prefixes Hide: Trajectory Dropout for On-Policy Distillation

    Sep 27, 2026Zizhuo Lin, Quanling Liu, Yi Yang +1CoT DistillationOn-Policy Distillation

    Zhejiang University

  4. Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning

    Sep 14, 2026Mantek Singh, Jeshwanth Challagundla, Siddharth Raina +1CoT DistillationVLM Distillation

    Liverpool John Moores University, Liverpool, England · Carnegie Mellon University, Pittsburgh, USA · Meta, Sunnyvale, USA +1

  5. MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation

    Aug 30, 2026Yangsong Lan, Renkai Hu, HongKai Zheng +4CoT DistillationLanguage Model Distillation

    College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · 2The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China

  6. Simple-OPD: Demystifying Warm-up for On-policy Distillation

    Aug 7, 2026Tao Liu, Taiqiang Wu, Mao Zheng +5CoT DistillationOn-Policy Distillation

    1Tsinghua University · 2The University of Hong Kong · 3LLM Department, Tencent

  7. CoTinyVLA: Chain-of-Thought Distillation for a Sub-Billion-Parameter Vision-Language-Action Model

    Jul 28, 2026Minhyeok Lee, Chiyoung Kim, Chanhoe Gu +5CoT DistillationEfficient VLA Model Inference

    Chung-Ang University, Republic of Korea

  8. TabRank: Chain-of-Thought Distillation for Table Re-Rankers

    Jul 28, 2026Adarsh Singh, Kushal Raj Bhandari, Jianxi Gao +2Table QACoT Distillation

    1Arizona State University · 2Rensselaer Polytechnic Institute · 3Scale AI

  9. DAIS: Dependency-Aware Intermediate QA Supervision for Complex Reasoning

    Jul 21, 2026Yu Wang, Ming Fan, Xicheng Zhang +5CoT Distillation

    1Xi’an Jiaotong University · 2Huawei Technologies Ltd.

  10. CADENCE: Closing the Reasoning Gap via Coverage-Adaptive On-Policy Distillation

    Jul 18, 2026Satyam Kumar, Saurabh JhaCoT DistillationOn-Policy Distillation

  11. Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning

    Jul 17, 2026Leichao Dong, Dongxu Zhang, Yiding Sun +4CoT DistillationEfficient Language Model Reasoning

    1Xi’an Jiaotong University · 2Peking University

  12. Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

    Jul 16, 2026Jungseob Lee, Seungyoon Lee, Suhyune Son +4CoT DistillationLanguage Model Distillation

    1Korea University · 2Zoom Communications · 3Yonsei University

  13. Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs

    Jul 12, 2026Shubham RaoCoT DistillationVLM Adaptation

    Entropy AI Research Labs Private Limited

  14. SCOReD: Student-Aware CoT Optimization for Recommendation Distillation

    Jul 7, 2026Haz Sameen Shahgir, Yufei Li, Xiaohan Wei +8CoT DistillationLanguage Model Distillation

    University of California Riverside · Meta AI

  15. Purified OPSD: On-Policy Self-Distillation Without Losing How to Think

    Jul 2, 2026Zhanming Shen, Jintao Tong, Shaotian Yan +9CoT DistillationOn-Policy Self-Distillation

    1Zhejiang University · 2Tongyi Lab, Alibaba Group · 3Huazhong University of Science and Technology +1

  16. Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition

    Jun 30, 2026Gaurab Baral, Aaditya Khanal, Yangyang Tao +1CoT DistillationMathematical Reasoning

    Northern Kentucky University, Highland Heights, KY 41099, USA

  17. ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillation

    Jun 22, 2026Chen Lin, Kedi Chen, Wei ZhangCoT DistillationLanguage Model Distillation

    East China Normal University · Shanghai Innovation Institute

  18. Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation

    Jun 22, 2026Jinwei Xiao, Zhuowen Han, Yueqing Sun +6CoT DistillationOn-Policy Distillation

    Meituan Longcat Team · TJUNLP Lab, College of Intelligence and Computing, Tianjin University · Nanjing University

  19. Prefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts

    Jun 20, 2026Qingfei Zhao, Huan Song, Shuyu Tian +2Trajectory GenerationCoT Distillation

    TeleAI · Shanghai Jiao Tong University

  20. A Verifiable Search Is Not a Learnable Chain-of-Thought

    Jun 20, 2026Harsh PatelInference-Time SearchMemorization in Language Models

    Independent Researcher

  21. When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation

    Jun 19, 2026Siyang Lyu, Xinghao Chen, Zhijing Sun +3Model CompressionCoT Distillation

    Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Viterbi School of Engineering, University of Southern California · The Hong Kong Polytechnic University +2

  22. Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

    Jun 18, 2026Durgesh Kalwar, Vardhan Palod, Jaya Adithya Pavuluri +1CoT DistillationEfficient Language Model Reasoning

    SCAI, Arizona State University

  23. Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillation

    Jun 17, 2026Zhilin Huang, Hang Gao, Ziqiang Dong +6LLM Self-CorrectionCoT Distillation

    Qwen Business Unit of Alibaba · Tsinghua University · Peking University

  24. The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning

    Jun 15, 2026Haolong Qian, Xianliang Yang, Yinuo ma +6CoT DistillationLanguage Model Distillation

    Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Microsoft Research Asia, Microsoft, Beijing, China · Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen, China +1

  25. Beyond Absolute Imitation: Anchored Residual Guidance for Privileged On-Policy Distillation

    Jun 9, 2026Wenhao ZhangCoT DistillationLanguage Model Distillation

    South China University of Technology

  26. Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces

    Jun 5, 2026Debjyoti Saha RoyMulti-Label ClassificationCoT Distillation

    Khoury College of Computer Sciences, Northeastern University

  27. Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation

    Jun 4, 2026Maxime Griot, Paul Steven Scotti, Tanishq Mathew AbrahamCoT DistillationTeacher-Student Learning

    Université catholique de Louvain · Sophont Inc.

  28. LoRi: Low-Rank Distillation for Implicit Reasoning

    Jun 3, 2026Ryan Solgi, Jiayi Tian, Zheng ZhangCoT DistillationMathematical Reasoning

    University of California-Santa Barbara, USA

  29. Invariant Gradient Alignment for Robust Reasoning Distillation

    Jun 3, 2026Zehua Cheng, Wei Dai, Jiahao SunCoT DistillationDomain Generalization

    University of Oxford, Oxford, UK

  30. Distilling Neuro-Symbolic Programs into 3D Multi-modal LLMs

    May 31, 2026Wentao Mo, Yang LiuCoT Distillation3D Spatial Reasoning

    Wangxuan Institute of Computer Technology, Peking University