cs.AIOct 5, 2026

SPECTRUM: Proximal Spectral Modulation for Looped Self-Distillation

Authors: Yunbo Long, WenJie Chen, Jiaquan Zhang, Guangya Hao, Zihang Zeng, Pengze Li, Xi Chen

Abstract

A model that learns from its own outputs inherits more than their correctness: it inherits which solutions it produces. We formulate Looped Self-Distillation, a self-evolution framework for code generation in which a model repeatedly generates and learns from its own raw outputs, under a fixed information budget, without ongoing external assessment or test-based selection of the generated samples. We identify a consequential separation: correctness can improve while the breadth of correct implementations contracts. We introduce SPECTRUM, which re-estimates loss-sensitive key/value geometry from a fixed reference anchor at each round and converts it into full-rank proximal spectral modulation. All generated completions train a single student, whose subsequent inference requires no intervention. After five rounds of experiments on MBPP, SPECTRUM retains 89.9% of the initial model's 64-sample correct AST richness, compared with 66.4% for Vanilla self-distillation and 65.5% for a subspace-projection control. The advantage persists at matched correct-sample counts. Without further training or recalibration, the resulting student also achieves higher matched-correct richness than Vanilla SD on HumanEval+ and APPS Intro, demonstrating transfer of the diversity benefit. These findings establish correct-solution retention as a complementary objective of recursive self-improvement (RSI) and show that generation-time intervention can improve the solution repertoire retained by subsequent students.

Explore similar work

CardsList
  1. Self-Policy Distillation via Capability-Selective Subspace Projection

    May 21, 2026Guangya Hao, Yitong Shang, Yunbo Long +2Self-Training for Language ModelsLanguage Model Distillation

  2. Optimal Self-Distillation for Rectified Flow via Linear Probing

    Jul 16, 2026Saptarshi Roy, Debepsita Mukherjee, Pratik PatilRectified FlowSelf-Distillation