cs.LGNov 18, 2026

Communicative On-Policy Distillation: Distilling Black-Box Teacher Capabilities through Communication for Long-Horizon Tasks

Authors: Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, 

Organizations: Peking University · Sichuan University · Beijing Institute of Technology · Renmin University of China · 

Abstract

On-policy distillation (OPD) is a powerful paradigm for improving language-model agents on long-horizon tasks, yet its conventional formulation relies on teacher logits that text-only black- box models do not expose. To address this limitation, we introduce Communicative On-Policy Distillation (COPD), which uses natural-language communication as a medium for distilling a stronger model's capabilities into a smaller student. During task execution, the student au- tonomously decides when to consult a black-box teacher and formulates open-ended natural- language requests without predefined question templates or intervention schedules. Successful student-teacher interactions are retained as privileged references to supervise fresh adviser-free student rollouts through on-policy self-distillation, jointly with task-reward reinforcement learning. On CAMG-II, COPD achieves 2.46 times the gain of the strongest black-box OPD baseline. The capabilities it learns through communication generalize to Terminal-Bench 2.1 and SWE-Bench Pro V2, improving task success without adviser access.

Explore similar work

CardsList
  1. ActionOPD: Distilling Decisions, Not Tokens

    Oct 14, 2026Huining Yuan, Xiangmin Yi, Zelai Xu +5LLM Agent TrainingKnowledge Distillation

  2. Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation

    Oct 18, 2026Jike Zhong, Ming Li, Yuxiang Lai +9On-Policy DistillationAgentic RL

  3. Harness-Aware Distillation for Small Language Model Agents

    Oct 2, 2026Moonseok Choi, Taehong Moon, Giung Nam +1LLM Agent HarnessesOn-Policy Distillation