A Public Theory of Distillation Resistance via Constraint-Coupled Reasoning Architectures

Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative. However, distilling such a model into a weaker benign student places the teacher in a Distillation Double Bind: if misalignment transfers, the student may conceal it less effectively, revealing evidence about the teacher; if it does not, the student may learn useful capabilities while remaining benign. We introduce two distinct distillation approaches, one targeting each outcome. Distillation for Incrimination (DFI) aims to transfer misalignment but not the ability to conceal it. Distilling AuditBench's secret-keeping models into their underlying instruction-tuned model produces students that are significantly more likely than their teachers to admit their hidden behavior when asked, suggesting that knowledge of the behavior transferred more readily than the propensity to conceal it. Confession gains largely disappear when the student does not share the teacher's pretrained base, so DFI should target the teacher's own pre-RL checkpoint, which is weaker than the teacher but shares its base model. Distillation for Capabilities (DFC) aims to transfer capabilities but not misalignment. Among several techniques we evaluate, two are effective: inoculation prompting and training for more epochs on fewer unique examples. Both preserve the capability gains of standard distillation while substantially reducing the subliminal transfer of an animal preference, our proxy for misalignment. Together, these findings demonstrate two ways distillation can be used for AI safety: incriminating misaligned models, and extracting their capabilities without their misalignment.

Distillation via reasoning traces exposes frontier models to adversarial third parties, who can bypass their guardrails and misappropriate their capabilities. Existing antidistillation defenses perturb the teacher's sampling distribution to hinder downstream student learning, but these perturbations also degrade the trace quality even when task accuracy is preserved, often leaving visible artifacts such as syntactic incoherencies and repetition chains. Beyond eroding user trust, such traces are statistically distinguishable, allowing adversaries to filter or resample them and weaken the defense. An effective defense must therefore limit how visibly it degrades the teacher. To address this gap, we formulate antidistillation as a max-min game whose constraint set explicitly encodes defense detectability. Satisfying both axes--- degrading the student while remaining undetectable---may require more clever perturbations, akin to sparse but targeted adversarial attacks in robustness literature. We apply the same analogy to reasoning steps, targeting thought anchors: sentences identified in prior interpretability work as strongly influencing a model's output. We propose Thought Anchor RemovalS (TARS), which identifies and removes thought anchors, whose content is semantically redundant, so that the reasoning trace remains legible to the user. Modeling these removals as burst-deletions, we then provide formal guarantees on its detectability. Finally, we empirically demonstrate the efficacy of TARS in consistently hindering downstream distillation across teacher model scales (DeepSeek R1 Distill Qwen 7B, Gemma 3 12B), reasoning benchmarks (MMLU Pro, MATH, DeepMath), and student architectures (Llama 3.2 1B, Llama 3.2 3B, Gemma 3 1B, Qwen 3 1.7B) while preserving teacher performance.

Knowledge distillation from large reasoning models (LRMs) often uses raw teacher trajectories as supervision. However, this practice creates a mismatch between the goal of distillation and the nature of teacher trajectories. Advanced LRMs typically improve their reasoning capabilities through reward-driven post-training, such as RLHF. This training optimizes them as strong solvers, rather than as teachers that produce reasoning processes smaller models can easily learn. Since smaller models have different "cognitive capacities" compared to their larger teachers, directly imitating raw teacher trajectories can sometimes be ineffective and often requires a substantial number of high-quality samples. To bridge this mismatch, we propose Cognition-Aligned Reasoning Distillation (CARD), an RL-based framework that transforms solver trajectories into teaching trajectories aligned with the student's cognition. Avoiding expensive resampling, our framework directly transforms existing trajectories. We optimize CARD with two reward signals: (1) a Distribution Alignment Reward that matches the student's predictive distribution, and (2) a Logical Coherence Reward that reduces misleading failed reasoning steps. Comprehensive evaluations across multiple student models and settings demonstrate that CARD outperforms distillation from raw teacher trajectories in in-domain settings while achieving substantial gains in out-of-distribution performance. Code is available at https://github.com/LiteSSLHub/CARD.