Overview

Reasoning distillation trains a smaller student to reproduce or benefit from a larger model’s problem-solving, using traces, token probabilities, feedback, or reward signals as supervision. Common approaches include supervised trace imitation, on-policy distillation, reinforcement learning, and curricula that adapt examples or teacher signals to the student’s capabilities. The goal is to retain useful reasoning in settings such as mathematics, coding, multimodal tasks, and safety while reducing training or inference costs. A central challenge is that copying a teacher’s behavior can pass along flawed steps or overwhelm a student, and gains in solving do not always translate into reliable confidence, answer formatting, or stopping.

Latest papers

CardsList

26 papers5 papers in the last four weeks, up 25% on the four weeks before. 0.0% of all new papers.

  1. Failure Profiling and Reachable Trajectory Selection for Reasoning Distillation

    Nov 6, 2026Yijun Zhu, Jianxin Wang, Chengchao ShenReasoning DistillationLanguage Model Distillation

  2. Multi-Teacher Reasoning Distillation for LLM Recommendation with Self-Exploration

    Nov 2, 2026Liangbo Ning, Siyuan Liang, Yi Zhou +4Reasoning DistillationLarge Language Model-Based Recommendation

  3. From Solver Trajectories to Teaching Trajectories: Cognition-Aligned Reasoning Distillation

    Oct 27, 2026Yashuo Luo, Qianren Mao, Ziqi Qin +6Reasoning DistillationLanguage Model Distillation

  4. Bridged Self-Distillation for On-Policy Learning

    Oct 19, 2026Runsheng Chen, Yihong Liu, Raoyuan Zhao +3Reasoning DistillationOn-Policy Distillation

  5. Solving Without Stopping: On-Policy Distillation at Small Scale

    Sep 29, 2026Hongyang Li, Yiming Zhu, Xiao Li +3Reasoning DistillationSmall Language Models

  6. ChemOPD: Multi-Teacher On-Policy Distillation for Multi-Task Chemical Reasoning

    Sep 27, 2026Yaoyao Xu, Xinjian Zhao, Xiaozhuang Song +2Chemical Reasoning in LLMsReasoning Distillation

  7. TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment

    Sep 27, 2026Zhenyu Lei, Zihan Chen, Yaochen Zhu +5Reasoning DistillationRL for Language Model Reasoning

  8. On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation

    Sep 18, 2026Anton Baumann, Akmal Ashirmatov, Leo Schmidt-Traub +4Reasoning DistillationOn-Policy Self-Distillation

  9. Reason in the Words You Speak: Idiolectal Paraphrasing Off-Policy Traces for Reasoning Distillation in VideoLLMs

    Aug 27, 2026Ji Soo Lee, Jinyoung Park, Seohyun Lee +4Reasoning DistillationVLM Reasoning

  10. Benchmarking and Reasoning Distillation of Large Language Models for Feedback Controller Design in Complex Dynamical Systems

    Aug 7, 2026Zhongchao Zhou, Yixuan Xie, Wenwei Yu +5Local LLM DeploymentLLM Evaluation

  11. Distilling Reasoning Traces into Advisory Prompts for Software Engineering Tasks

    Aug 1, 2026Faizan Faisal, Prem Devanbu, Toufique AhmedReasoning DistillationLLM Prompting

  12. DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

    Jul 7, 2026He Liu, Changtao Miao, Xinjie Yang +12Reasoning DistillationLLM Guardrails

  13. Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions

    Jun 8, 2026Xin Jin, Huanqia Cai, Zhen Li +9Reward ModelingReasoning Distillation

  14. Tailoring the Curriculum: Student-Centered Reasoning Distillation via Dynamic Data-Model Compatibility

    May 28, 2026Jiahao Huang, Fei Cheng, Junfeng Jiang +1Reasoning DistillationTraining Data Selection

  15. Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation

    May 16, 2026Anhao Zhao, Haoran Xin, Yingqi Fan +3Reasoning DistillationOn-Policy Distillation

  16. Curriculum Learning-Guided Progressive Distillation in Large Language Models

    May 11, 2026Jincheng Cao, Fanzhi Zeng, Leqi Liu +1Reasoning DistillationLanguage Model Distillation

  17. ReflectMT: Internalizing Reflection for Efficient and High-Quality Machine Translation

    Apr 21, 2026Kunquan Li, Yingxue Zhang, Zhibin Lan +2Reasoning DistillationMachine Translation

  18. Validity-Calibrated Reasoning Distillation

    Apr 14, 2026Khouloud Saadi, Di WangReasoning DistillationLanguage Model Distillation

  19. Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective

    Apr 10, 2026Tokio Kajitsuka, Ukyo Honda, Sho TakaseCoT DistillationReasoning Distillation

  20. Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains

    Apr 9, 2026Haokai Ma, Lee Yan Zhen, Gang Yang +4RL for Language ModelsReinforcement Learning

  21. LLM-MRD: LLM-Guided Multi-View Reasoning Distillation for Fake News Detection

    Mar 10, 2026Weilin Zhou, Shanwen Tan, Enhao Gu +1Fake News DetectionReasoning Distillation

  22. Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion

    Feb 26, 2026Zhen Tan, Chengshuai Zhao, Song Wang +3Language Model DistillationReasoning Distillation

  23. Federated Reasoning Distillation Framework with Model Learnability-Aware Data Allocation

    Feb 21, 2026Wei Guo, Siyuan Lu, Xiangdong Ran +8Reasoning DistillationFederated Knowledge Distillation