cs.LGNov 17, 2026

Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching

Authors: Giannis Daras, Marios Papachristou, Adam Klivans, Antonio Torralba, Constantinos Daskalakis

Organizations: MIT · Arizona State University · University of Texas at Austin

Abstract

Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit T→∞T\to \infty reverting back to regular SFT and T=1T=1 to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to 21%21\% average relative improvement to standard SFT.

Figures & tables

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting

    May 29, 2026Yuduo Li, Xiaofeng Shi, Qian Kou +2Supervised Fine-TuningFine-Tuning

  2. Learning to Adapt SFT Data for Better Reasoning Generalization

    May 26, 2026Lisong Sun, Li Wang, Chen Zhang +4Supervised Fine-TuningLLM Post-Training

  3. CPR for LLMs: Critical-Point Routing against Catastrophic Forgetting in Domain Adaptation

    Aug 31, 2026Kwangmin Ki, Yunhun Nam, Jongheon Jeong +1LLM Fine-TuningDomain Adaptation for LLMs