CoT Distillation

CoT: Chain-of-Thought

Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 57

All topics
CardsList

  1. Learning from Think-Mode Advantage via On-Policy Distillation

    Sep 29, 2026Wanqi Ren, Jianxiang Wang, Danxuan Liu +2CoT DistillationLanguage Model Distillation

    ByteDance, China

  2. Distilling Visual Reasoning into Text Space

    Sep 28, 2026Wenhan Yang, Nilay Naharas, Ali Payani +1CoT DistillationVLM Distillation

    University of California, Los Angeles · Cisco Research

  3. What Shared Prefixes Hide: Trajectory Dropout for On-Policy Distillation

    Sep 27, 2026Zizhuo Lin, Quanling Liu, Yi Yang +1CoT DistillationOn-Policy Distillation

    Zhejiang University

  4. Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning

    Sep 14, 2026Mantek Singh, Jeshwanth Challagundla, Siddharth Raina +1CoT DistillationVLM Distillation

    Liverpool John Moores University, Liverpool, England · Carnegie Mellon University, Pittsburgh, USA · Meta, Sunnyvale, USA +1

  5. MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation

    Aug 30, 2026Yangsong Lan, Renkai Hu, HongKai Zheng +4CoT DistillationLanguage Model Distillation

    College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · 2The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China

  6. Simple-OPD: Demystifying Warm-up for On-policy Distillation

    Aug 7, 2026Tao Liu, Taiqiang Wu, Mao Zheng +5CoT DistillationOn-Policy Distillation

    1Tsinghua University · 2The University of Hong Kong · 3LLM Department, Tencent

  7. CoTinyVLA: Chain-of-Thought Distillation for a Sub-Billion-Parameter Vision-Language-Action Model

    Jul 28, 2026Minhyeok Lee, Chiyoung Kim, Chanhoe Gu +5CoT DistillationEfficient VLA Model Inference

    Chung-Ang University, Republic of Korea

  8. TabRank: Chain-of-Thought Distillation for Table Re-Rankers

    Jul 28, 2026Adarsh Singh, Kushal Raj Bhandari, Jianxi Gao +2Table QACoT Distillation

    1Arizona State University · 2Rensselaer Polytechnic Institute · 3Scale AI

  9. DAIS: Dependency-Aware Intermediate QA Supervision for Complex Reasoning

    Jul 21, 2026Yu Wang, Ming Fan, Xicheng Zhang +5CoT Distillation

    1Xi’an Jiaotong University · 2Huawei Technologies Ltd.

  10. CADENCE: Closing the Reasoning Gap via Coverage-Adaptive On-Policy Distillation

    Jul 18, 2026Satyam Kumar, Saurabh JhaCoT DistillationOn-Policy Distillation

  11. Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning

    Jul 17, 2026Leichao Dong, Dongxu Zhang, Yiding Sun +4CoT DistillationEfficient Language Model Reasoning

    1Xi’an Jiaotong University · 2Peking University

  12. Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

    Jul 16, 2026Jungseob Lee, Seungyoon Lee, Suhyune Son +4CoT DistillationLanguage Model Distillation

    1Korea University · 2Zoom Communications · 3Yonsei University

  13. Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs

    Jul 12, 2026Shubham RaoCoT DistillationVLM Adaptation

    Entropy AI Research Labs Private Limited

  14. SCOReD: Student-Aware CoT Optimization for Recommendation Distillation

    Jul 7, 2026Haz Sameen Shahgir, Yufei Li, Xiaohan Wei +8CoT DistillationLanguage Model Distillation

    University of California Riverside · Meta AI

  15. Purified OPSD: On-Policy Self-Distillation Without Losing How to Think

    Jul 2, 2026Zhanming Shen, Jintao Tong, Shaotian Yan +9CoT DistillationOn-Policy Self-Distillation

    1Zhejiang University · 2Tongyi Lab, Alibaba Group · 3Huazhong University of Science and Technology +1

  16. Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition

    Jun 30, 2026Gaurab Baral, Aaditya Khanal, Yangyang Tao +1CoT DistillationMathematical Reasoning

    Northern Kentucky University, Highland Heights, KY 41099, USA

  17. ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillation

    Jun 22, 2026Chen Lin, Kedi Chen, Wei ZhangCoT DistillationLanguage Model Distillation

    East China Normal University · Shanghai Innovation Institute

  18. Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation

    Jun 22, 2026Jinwei Xiao, Zhuowen Han, Yueqing Sun +6CoT DistillationOn-Policy Distillation

    Meituan Longcat Team · TJUNLP Lab, College of Intelligence and Computing, Tianjin University · Nanjing University

  19. Prefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts

    Jun 20, 2026Qingfei Zhao, Huan Song, Shuyu Tian +2Trajectory GenerationCoT Distillation

    TeleAI · Shanghai Jiao Tong University

  20. A Verifiable Search Is Not a Learnable Chain-of-Thought

    Jun 20, 2026Harsh PatelInference-Time SearchMemorization in Language Models

    Independent Researcher

  21. When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation

    Jun 19, 2026Siyang Lyu, Xinghao Chen, Zhijing Sun +3Model CompressionCoT Distillation

    Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Viterbi School of Engineering, University of Southern California · The Hong Kong Polytechnic University +2

  22. Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models

    Jun 18, 2026Durgesh Kalwar, Vardhan Palod, Jaya Adithya Pavuluri +1CoT DistillationEfficient Language Model Reasoning

    SCAI, Arizona State University

  23. Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillation

    Jun 17, 2026Zhilin Huang, Hang Gao, Ziqiang Dong +6LLM Self-CorrectionCoT Distillation

    Qwen Business Unit of Alibaba · Tsinghua University · Peking University

  24. The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning

    Jun 15, 2026Haolong Qian, Xianliang Yang, Yinuo ma +6CoT DistillationLanguage Model Distillation

    Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Microsoft Research Asia, Microsoft, Beijing, China · Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen, China +1

  25. Beyond Absolute Imitation: Anchored Residual Guidance for Privileged On-Policy Distillation

    Jun 9, 2026Wenhao ZhangCoT DistillationLanguage Model Distillation

    South China University of Technology

  26. Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces

    Jun 5, 2026Debjyoti Saha RoyMulti-Label ClassificationCoT Distillation

    Khoury College of Computer Sciences, Northeastern University

  27. Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation

    Jun 4, 2026Maxime Griot, Paul Steven Scotti, Tanishq Mathew AbrahamCoT DistillationTeacher-Student Learning

    Université catholique de Louvain · Sophont Inc.

  28. LoRi: Low-Rank Distillation for Implicit Reasoning

    Jun 3, 2026Ryan Solgi, Jiayi Tian, Zheng ZhangCoT DistillationMathematical Reasoning

    University of California-Santa Barbara, USA

  29. Invariant Gradient Alignment for Robust Reasoning Distillation

    Jun 3, 2026Zehua Cheng, Wei Dai, Jiahao SunCoT DistillationDomain Generalization

    University of Oxford, Oxford, UK

  30. Distilling Neuro-Symbolic Programs into 3D Multi-modal LLMs

    May 31, 2026Wentao Mo, Yang LiuCoT Distillation3D Spatial Reasoning

    Wangxuan Institute of Computer Technology, Peking University

  31. Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance

    May 29, 2026Yuxuan Jiang, Francis FerraroCoT DistillationLanguage Model Distillation

    University of Maryland, Baltimore County

  32. Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation

    May 29, 2026Yanjiang Liu, Jie Lou, Xinyan Guan +7CoT DistillationLanguage Model Distillation

    University of Chinese Academy of Sciences · 2Chinese Information Processing Laboratory Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences, Beijing, China · 3Xiaohongshu

  33. LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation

    May 28, 2026Tianrun Yu, Kaixiang Zhao, Chih-Chun Chen +5CoT DistillationLanguage Model Distillation

    Brigham Young University · The Pennsylvania State University · University of North Carolina at Chapel Hill

  34. Better Accuracies, Worse Reasoning: A Step-Level Audit of Medical Chain-of-Thought Distillation

    May 27, 2026Zhaoyang Jiang, Xuanqi Peng, Fei Teng +5CoT DistillationCoT Evaluation

    School of Health & Wellbeing, University of Glasgow, Glasgow, UK · Department of Respiratory and Critical Care Medicine, Shanghai Sixth People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China · School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China +1

  35. Generative Spatiotemporal Intent Sequence Recommendation via Implicit Reasoning in Amap

    May 27, 2026Sicong Wang, Ruiting Dong, Yue Liu +7Spatiotemporal ReasoningCoT Distillation

    AMAP, Alibaba Group, Beijing, China

  36. When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning

    May 20, 2026Xiaogeng Liu, Xinyan Wang, Yingzi Ma +2CoT DistillationOn-Policy Self-Distillation

    1Johns Hopkins University · University of Wisconsin–Madison · 3Nanyang Technological University

  37. Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation

    May 19, 2026Bing Wang, Shaotian Yan, Chen Shen +7CoT DistillationLanguage Model Distillation

    College of Computer Science and Technology, Jilin University · Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University · Tongyi Lab, Alibaba Group +3

  38. Distribution Corrected Offline Data Distillation for Large Language Models

    May 13, 2026Yumeng Zhang, Zhengbang Yang, Yevin Nikhel Goonatilake +1CoT DistillationEfficient Language Model Reasoning

    George Mason University, Fairfax, VA, USA

  39. GateKD: Confidence-Gated Closed-Loop Distillation for Robust Reasoning

    May 13, 2026Kasidit Sermsri, Teerapong PanboonyuenCoT DistillationLanguage Model Distillation

    †Chulalongkorn University · ‡MARSAIL · §PBYAIL (Panboonyuen AI Lab)

  40. Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation

    May 12, 2026Seonghoon Yu, Dongjun Nam, Byung-Kwan Lee +1Cross-Modal Knowledge DistillationCoT Distillation

    1KAIST · 3POSTECH · 2NVIDIA

  41. On-Policy Distillation with Best-of-N Teacher Rollout Selection

    May 10, 2026Ke Zhang, Yunjie Tian, Dongdi Zhao +4CoT DistillationBest-of-N Sampling

    Johns Hopkins University · TikTok · University of California, San Diego +1

  42. LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering

    May 10, 2026Runze Ma, Shunbo Jia, Haonan Lyu +2CoT DistillationHealthcare

    School of Information Technology Monash University Malaysia · Faculty of Innovation Engineering Macau University of Science and Technology · Department of Bioelectronics Faculty of Biomedical Engineering Shenzhen University of Advanced Technology +1

  43. Structural Rationale Distillation via Reasoning Space Compression

    May 8, 2026Jialin Yang, Jiankun Wang, Jiajun Wu +3CoT DistillationStructured Reasoning

    University of Calgary · Calgary, Canada · University of Michigan +1

  44. Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decoding

    May 4, 2026Taewon Yun, Jisu Shin, Jeonghwan Choi +2CoT DistillationEfficient Language Model Reasoning

    KAIST · UNIST

  45. Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models

    Apr 25, 2026Max Hartman, Vidhata Jayaraman, Moulik Choraria +2CoT DistillationTeacher-Student Learning

    University of Illinois Urbana-Champaign · Carnegie Mellon University · Stony Brook University

  46. Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information

    Apr 17, 2026Yao Chen, Jiawei Sheng, Wenyuan Zhang +1CoT DistillationSmall Language Models

    Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences

  47. Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective

    Apr 10, 2026Tokio Kajitsuka, Ukyo Honda, Sho TakaseCoT DistillationReasoning Distillation

    The University of Tokyo, Tokyo, Japan · CyberAgent, Tokyo, Japan

  48. Grounded Chess Reasoning in Language Models via Master Distillation

    Mar 20, 2026Zhenwei Tang, Qianfeng Wen, Seth Grief-Albert +4LLM GroundingCoT Distillation

    Department of Computer Science, University of Toronto · Queen’s University · Coolwei AI Lab

  49. HEAL: Hindsight Entropy-Assisted Learning for Reasoning Distillation

    Mar 11, 2026Wenjing Zhang, Jiangze Yan, Jieyun Huang +7CoT DistillationLanguage Model Distillation

    Data Science & AI Research Institute, China Unicom · Unicom Data Intelligence, China Unicom

  50. CRISP: Compressed Reasoning via Iterative Self-Policy Distillation

    Mar 5, 2026Hejian Sang, Yuanda Xu, Zhengze Zhou +3CoT DistillationEfficient Language Model Reasoning

  51. Measuring and Mitigating Post-hoc Rationalization in Reverse Chain-of-Thought Generation

    Feb 16, 2026Guangyue Peng, Zongchao Chen, Wen Luo +9CoT DistillationCoT Reasoning

    State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Nanbeige Lab, BOSS Zhipin · University of Electronic Science and Technology of China

  52. BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

    Feb 5, 2026Bowen Yu, Sheng Zhang, Binhao Wang +8CoT DistillationMathematical Reasoning

    1City University of Hong Kong · 2Mohamed Bin Zayed University of Artificial Intelligence

  53. Distilling LLM Reasoning into Graph of Concept Predictors

    Feb 3, 2026Ziyang Yu, Liang ZhaoCoT DistillationActive Learning

    Department of Computer Science Emory University Atlanta, USA

  54. Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation

    Dec 24, 2025Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5CoT DistillationSelective Knowledge Distillation

    1The University of British Columbia · 2LinkedIn · 3Canada Research Chair in NLP and ML

  55. From Hindsight to Foresight: Self-Encouraged Hindsight Distillation for Knowledge-based Visual Question Answering

    Nov 14, 2025Yu Zhao, Ying Zhang, Xuhui Sui +4Visual Question AnsweringCoT Distillation

    College of Computer Science, VCIP, DISSec, Nankai University, Tianjin, China · School of Software, Tiangong University, Tianjin, China · Shenzhen Campus of Sun Yat-sen University, Shenzhen, China +1

  56. Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation

    Sep 26, 2025Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Kevin El Haddad +2CoT DistillationLLM Fine-Tuning

    1Diabolocom · 5MICS, CentraleSupélec, Université Paris-Saclay · 2Artefact Research Center +2

  57. Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning

    Aug 13, 2025Xiaojun Wu, Xiaoguang Jiang, Huiyang Li +11CoT DistillationLanguage Model Distillation

    1Zhongxing Telecom Equipment(ZTE), China · 2China Mobile Communications Group Co Ltd