Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit T→∞ reverting back to regular SFT and T=1 to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to 21% average relative improvement to standard SFT.
Figures & tables
Figure 1: Overview of Ambient SFT for data-efficient post-training. A pre-trained model initializes a trainable student and a frozen teacher. On the small dataset Dp , SFT fits the observed next token, while Temperature Matching aligns the two models’ distributions after smoothing. Their weighted sum updates only the student. Temperature controls trust in the teacher: T→∞ recovers SFT and T=1 knowledge distillation.
Figure 2: Learning from high-temperature data.
Method
Python Code
CS
BioMed
News
Reviews
Target Only (From Scratch)
1.50
3.04
2.66
2.63
3.37
Pre-trained (No Fine-tuning)
1.46
2.87
2.61
2.59
3.56
Target Only Fine-tuning
0.80
2.56
2.36
2.01
2.96
Data Mixture Fine-tuning
0.79
2.55
2.36
1.99
2.96
Ambient SFT (ours)
0.70
2.54
2.35
1.68
2.96
Table 1: Validation losses achieved by different fine-tuning strategies for domain adaptation.
Figure 3: Domain-adaptation ablations on Python Code.
MATH
GSM8K
HumanEval+
MMLU
PopQA
TriviaQA
IFEval
BBH
Avg. rel. Δ vs. SFT
Qwen3-1.7B-Base
Base (no SFT)
33.8
53.5
78.4
65.6
13.9
45.6
37.9
54.2
+9.0%
Vanilla SFT
24.3
61.0
68.0
61.9
12.2
36.3
58.8
45.2
—
Ambient SFT
40.4
72.3
83.7
64.9
14.9
43.8
52.3
53.3
+20.3%
Qwen3-4B-Base
Base (no SFT)
43.5
81.9
91.2
75.6
18.4
60.4
45.5
70.5
+5.7%
Vanilla SFT
36.4
80.2
84.5
72.9
14.8
51.4
76.3
63.4
—
Ambient SFT
42.2
82.9
90.1
74.3
18.1
57.5
71.0
68.9
+8.0%
Table 2: Instruction tuning on ≈21 K examples, 1% of Dolci-Instruct-SFT, Qwen3 base models from 1.7B to 14B parameters. Ambient SFT improves on Vanilla SFT on 7/8 tasks, consistently across sizes; the exception is IFEval, where the gap closes as the model grows. Last column is the average relative improvement.
MATH
GSM8K
HumanEval+
MMLU
PopQA
TriviaQA
IFEval
BBH
Avg. rel. Δ vs. SFT
Qwen3-1.7B
Vanilla SFT
24.3
61.0
68.0
61.9
12.2
36.3
58.8
45.2
—
Ambient SFT
40.4
72.3
83.7
64.9
14.9
43.8
52.3
53.3
+20.3%
Ambient SFT (per-source T )
40.9
73.2
83.1
64.5
14.6
42.8
58.2
53.2
+21.1%
Qwen3-4B
Vanilla SFT
36.4
80.2
84.5
72.9
14.8
51.4
76.3
63.4
—
Ambient SFT
42.2
82.9
90.1
74.3
18.1
57.5
71.0
68.9
+8.0%
Ambient SFT (per-source T )
43.4
85.3
90.6
73.7
16.7
55.6
77.1
69.8
+8.2%
Table 3: Ambient SFT with a single temperature ( T=2.5 ) and with a temperature set per source.
MATH
GSM8K
HumanEval+
MMLU
PopQA
TriviaQA
IFEval
BBH
Avg. rel. Δ vs. SFT
OLMo-3-7B
Vanilla SFT
22.1
75.2
71.4
65.7
17.5
60.9
70.8
61.3
—
Ambient SFT
41.5
78.6
79.5
66.4
21.0
64.8
64.0
62.4
+15.4%
Llama-3.1-8B
Vanilla SFT
17.8
58.8
55.2
63.2
26.9
75.1
71.9
61.4
—
Ambient SFT
21.2
61.1
70.5
65.7
30.9
79.3
62.7
64.1
+8.3%
Table 4: Generalization across model families, in the same setup as Table 2 .
MATH
GSM8K
HumanEval+
MMLU
PopQA
TriviaQA
IFEval
BBH
Avg. rel. Δ vs. SFT
Base (no SFT)
33.8
53.5
78.4
65.6
13.9
45.6
37.9
54.2
+9.0%
Vanilla SFT
24.3
61.0
68.0
61.9
12.2
36.3
58.8
45.2
—
NEFTune ( α=10 )
19.7
58.9
69.2
62.2
12.4
38.4
65.6
48.6
+0.8%
L2-SP ( α=1 )
35.0
72.3
83.2
64.9
14.5
45.6
41.8
52.8
+15.3%
Label smoothing ( ϵ=0.2 )
25.4
62.9
64.3
60.9
12.1
36.4
55.1
47.0
−0.3%
KD ( T=1 , pc=0.5 )
35.1
71.5
81.3
64.8
15.7
43.8
49.2
52.8
+17.0%
Table 5: Comparison with other regularizers, Qwen3-1.7B-Base, Table 2 setup. Full sweeps in Table 22 .
Figure 4: Results versus # of epochs following instruction-tuning on 10% of Dolci-Instruct-SFT.
GSM8K
AIME24
AIME25
GPQA
IFEval
HumanEval+
Avg. rel. Δ vs. SFT
Base (no SFT)
85.5
10.6
13.9
27.5
33.5
45.9
−35.6%
Vanilla SFT
94.2
39.8
33.2
39.3
34.6
75.6
—
KD ( T=1 , pc=0.5 )
90.7
36.2
30.8
33.0
34.0
70.7
−7.3%
L2-SP ( α=1 )
81.7
9.9
14.0
28.6
33.8
40.3
−37.1%
Ambient SFT
93.0
44.5
34.1
40.0
36.0
76.9
+3.5%
Table 6: Reasoning (OLMo-3-7B, 1% of Dolci-Think-SFT): Ambient SFT against Vanilla SFT and the two strongest baselines of Table 5 , at their best settings.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Input Context ( x ) Source
Target ( y ) Source
Loss Type
Standard SFT
Fine-tuning Dataset
Dataset / Human
Hard labels
SFT Teacher Data
Teacher Rollouts
Teacher Final Output
Hard labels
Knowledge Distillation
Fine-tuning Dataset
Teacher Logits
Soft labels
On-Policy Distillation
Student Rollouts
Teacher Rewards
RL
Off-Policy Distillation
Teacher Rollouts
Teacher Rewards
RL
Ambient SFT (Ours)
Fine-tuning Dataset
Pre-Trained Model (Same Size)
Soft at high-temperature
Appendix
Table 7: Comparison of our technique (Ambient SFT) with existing techniques in the literature. Our method targets the SFT part of the post-training process, and it does not rely on model rollouts. It is using a noisy teacher (model before fine-tuning) during fine-tuning. To account for the imperfections of the teacher, we only use it at high-temperatures.
p\T
1.25
1.5
1.75
2
2.5
0.5
2.73
2.73
2.73
2.75
2.75
0.75
2.75
2.74
2.75
2.76
2.76
0.9
2.77
2.77
2.77
2.78
2.77
Appendix
Table 8: Full (p,T) sweep for the Ambient SFT (Small Teacher) condition, CS, 124M student (15 configurations: p∈{0.5,0.75,0.9}×T∈{1.25,1.5,1.75,2.0,2.5} ). Validation loss, rounded to 2 decimal places; the domain’s best cell in bold (ties within rounding are all bolded).
p\T
1.25
1.5
1.75
2
2.5
0.5
2.50
2.49
2.49
2.47
2.49
0.75
2.51
2.50
2.50
2.50
2.49
0.9
2.51
2.50
2.51
2.50
2.50
Appendix
Table 9: Full (p,T) sweep for the Ambient SFT (Small Teacher) condition, BioMed, 124M student (15 configurations: p∈{0.5,0.75,0.9}×T∈{1.25,1.5,1.75,2.0,2.5} ). Validation loss, rounded to 2 decimal places; the domain’s best cell in bold (ties within rounding are all bolded).
p\T
1.25
1.5
1.75
2
2.5
0.5
2.25
2.24
2.23
2.25
2.26
0.75
2.26
2.28
2.29
2.28
2.30
0.9
2.33
2.33
2.33
2.33
2.33
Appendix
Table 10: Full (p,T) sweep for the Ambient SFT (Small Teacher) condition, News, 124M student (15 configurations: p∈{0.5,0.75,0.9}×T∈{1.25,1.5,1.75,2.0,2.5} ). Validation loss, rounded to 2 decimal places; the domain’s best cell in bold (ties within rounding are all bolded).
p\T
1.25
1.5
1.75
2
2.5
0.5
3.16
3.15
3.15
3.15
3.16
0.75
3.16
3.16
3.16
3.17
3.18
0.9
3.18
3.18
3.18
3.18
3.18
Appendix
Table 11: Full (p,T) sweep for the Ambient SFT (Small Teacher) condition, Reviews, 124M student (15 configurations: p∈{0.5,0.75,0.9}×T∈{1.25,1.5,1.75,2.0,2.5} ). Validation loss, rounded to 2 decimal places; the domain’s best cell in bold (ties within rounding are all bolded).
p\T
1
1.25
1.5
1.75
2
2.25
2.5
3
0.5
1.02
0.94
0.91
0.88
0.87
0.86
0.86
0.87
0.75
0.93
0.91
0.89
0.88
0.88
0.87
0.88
0.88
Appendix
Table 12: Full (p,T) sweep for the Ambient SFT (Small Teacher) condition, Python Code, 124M student. Validation loss, rounded to 2 decimal places; the domain’s best cell in bold (ties within rounding are all bolded). The high-temperature extension T∈{2.25,2.5,3} was run only for the two competitive mixing ratios, so p∈{0.25,0.9} , which are uniformly worse at every T that was run ( 1.08 and 0.88 at best), are omitted.
Domain
Small Teacher: best T
Large Teacher: best T
Python Code
2.5
2.5
CS
1.5
1.0
BioMed
2.0
2.0
News
1.75
1.5
Reviews
1.75
1.0
Appendix
Table 13: Optimal temperature T per domain for the 124M student, both teacher conditions. The optimal mixing ratio was p=0.5 in every one of the 10 (domain × teacher) cells tested – omitted from the table since it does not vary.
Domain
Best p
Best T
Python Code
0.5
1.5
CS
0.75
2.0
BioMed
0.5
1.0
News
0.5
1.5
Reviews
0.9
2.5
Appendix
Table 14: Optimal (p,T) per domain for the 1.5B student, Ambient SFT self-teacher condition (1.5B teacher).
linear warmup over the first 3% of steps, then linear decay to 0
Gradient clipping
global norm 1.0
Global batch
1,048,576 tokens (32 sequences of 32,768 tokens)
Appendix
Table 15: Training hyperparameters shared by all instruction-tuning runs.
Experiment
Data
Epochs
Steps
Tables 2 – 4
1% (21,503 examples)
10
170
Figure 4
10% (215,027 examples)
2, 4, 6, 8, 10
340 to 1,700
Figure 6 , 1%
1%
10 (both methods)
170
Figure 6 , 10%
10%
2 (vanilla SFT), 4 (Ambient SFT)
340, 680
Figure 6 , 100%
100% (2,150,273 examples)
2 (both methods)
3,376
Appendix
Table 16: Number of epochs and optimizer steps per experiment. Steps follow from the data size and the fixed global batch.
Benchmark
olmes task
Shots
Prompt
Decoding
Metric
MATH
minerva_math::tulu
4, CoT
chat
greedy, ≤ 1024 tokens
exact match, mean of 7 subjects
GSM8K
gsm8k::olmes
8, CoT
plain
greedy, ≤ 512 tokens
exact match
HumanEval+
codex_humanevalplus::tulu
0
chat
20 samples, T=0.8 , top- p 0.95
pass@10
MMLU
mmlu::olmes
5
plain
log-likelihood over options
accuracy, mean of 57 subjects
PopQA
popqa::tulu
15
chat
greedy, ≤ 15 tokens
exact match
TriviaQA
triviaqa::olmes
5
plain
greedy, ≤ 50 tokens
F1 (closed-book)
Appendix
Table 17: Task specifications used for every instruction-tuning evaluation. All generation is greedy except HumanEval+, which draws 20 samples per problem to estimate pass@10. MATH uses the Minerva chain-of-thought prompt format ( Lewkowycz et al., 2022 ) .
Benchmark
olmes task
Samples per problem
Response budget
Metric
GSM8K
gsm8k::olmo3:adapt
1
context limit
exact match of the \boxed{} answer
AIME 2024
aime:2024::olmo3:adapt
32
16,384 tokens
pass@1, mean over the 32 samples
AIME 2025
aime:2025::olmo3:adapt
32
16,384 tokens
pass@1, mean over the 32 samples
HumanEval+
codex_humanevalplus::olmo3:adapt
10
context limit
pass@1
IFEval
ifeval::olmo3:adapt
1
context limit
prompt-level loose accuracy
GPQA
gpqa::olmo3:adapt
1
context limit
exact match of the answer letter (448 questions)
Appendix
Table 18: Task specifications used for every reasoning evaluation. All tasks are zero-shot, chat-formatted and sampled at T=0.6 , top- p 0.95.
Figure 5: The sweep of Figure 2(b) over the full range of training temperatures, T=1/6 to T=10 . Both losses degrade at both extremes; the objective of Eq. ( 12 ) degrades far more slowly.
Python Code
News
BioMed
CS
Reviews
Zero-shot validation loss
1.46
2.59
2.61
2.87
3.56
Ambient SFT win margin
0.09
0.31
0.01
0.01
0.00
Appendix
Table 19: Zero-shot validation loss of the pre-trained (1.5B) model on each target domain, before any fine-tuning, alongside Ambient SFT’s win margin over the best baseline from Table 1 (best baseline loss minus Ambient SFT loss; higher is a bigger win; negative means Ambient SFT was outperformed). Domains are ordered by zero-shot loss.
Method
Python Code
CS
BioMed
News
Reviews
Target Only (From Scratch)
0.98
2.99
2.62
2.63
3.34
Target Only Fine-tuning
0.88
2.78
2.51
2.32
3.19
Data Mixture Fine-tuning
0.88
2.69
2.47
2.26
3.11
Ambient SFT (Small Teacher)
0.86
2.73
2.47
2.23
3.15
Ambient SFT (Large Teacher)
0.83
2.68
2.43
2.23
3.12
Appendix
Table 20: Best validation loss achieved per domain across fine-tuning strategies, 124M student. Target Only (From Scratch) : trained from scratch on target-domain data only. Target Only Fine-tuning : finetuned from the OWT-pretrained checkpoint on target-domain data only. Data Mixture Fine-tuning : finetuned from the OWT-pretrained checkpoint on a clean mixture of target-domain and OWT data (best mixing ratio p per domain). Ambient SFT (Small Teacher) : our method against a frozen self-teacher (the same 124M OWT-pretrained checkpoint), using only target-domain data (best p,T per domain). Ambient SFT (Large Teacher) : same mechanism, with a frozen 1.5B-parameter teacher in place of the 124M self-teacher. Values rounded to 2 decimal places, the level of precision supported by measured run-to-run variance across random seeds ( σ≈0.003 – 0.006 across two domains checked); differences below this are not distinguishable from noise. Lower is better; best result(s) per domain in bold (ties within rounding are all bolded).
Student
Teacher
Python Code
CS
BioMed
News
Reviews
124M
124M (self)
0.86
2.73
2.47
2.23
3.15
124M
1.5B
0.83
2.68
2.43
2.23
3.12
1.5B
124M
0.79
2.61
2.39
1.96
3.01
1.5B
1.5B (self)
0.70
2.54
2.35
1.68
2.96
Appendix
Table 21: Effect of teacher strength on Ambient SFT’s best validation loss, at both student scales. The 124M teacher is a self-teacher when the student is also 124M; the 1.5B teacher is the same OWT-pretrained checkpoint used as the teacher in Table 1 . Values rounded to 2 decimal places; best teacher choice per domain (within each student-size block) in bold (ties within rounding are all bolded).
Figure 6: Benchmark scores versus the fraction of Dolci-Instruct-SFT data used for training. For all these results, we start from an OLMo-3-7B model. Error bars are ±1 binomial standard error over each task’s items. For the majority of the benchmarks, Ambient SFT improves upon regular SFT across all data scales. The highest improvements are observed in the 1% of the dataset setting.
Figure 7: The remaining four tasks of Figure 4 : benchmark scores versus the number of epochs on a 10% subset of Dolci-Instruct-SFT, OLMo-3-7B, error bars ±1 binomial standard error.
MATH
GSM8K
HumanEval+
MMLU
PopQA
TriviaQA
IFEval
BBH
Avg. rel. Δ vs. SFT
Base (no SFT)
33.8
53.5
78.4
65.6
13.9
45.6
37.9
54.2
+9.0%
Vanilla SFT
24.3
61.0
68.0
61.9
12.2
36.3
58.8
45.2
—
NEFTune ( α=5 )
23.7
60.0
66.8
61.6
11.7
37.8
60.8
46.8
+0.1%
NEFTune ( α=10 )
19.7
58.9
69.2
62.2
12.4
38.4
65.6
48.6
+0.8%
NEFTune ( α=15 )
18.3
61.9
71.7
62.2
11.8
39.0
63.0
48.7
+0.2%
L2-SP ( α=0.001 )
20.2
56.0
72.5
61.4
12.2
37.1
58.4
45.8
−2.1%
Appendix
Table 22: Full hyperparameter sweeps of the baselines of Table 5 , Qwen3-1.7B-Base, Table 2 setup.
Benchmark
Vanilla SFT
Ambient SFT
MMLU
64.6
66.0
GSM8K
77.6
83.7
TriviaQA
57.0
64.3
BBH
57.0
66.2
Appendix
Table 23: Knowledge benchmarks for the Vanilla SFT and Ambient SFT reasoning checkpoints of Table 6 (OLMo-3-7B, 1% of Dolci-Think-SFT, ten epochs; Ambient SFT with pc=0.75 , T=2.5 ), with few-shot, non-chat prompting on the full test sets.