What the Abstention Option Measures: Refusal Loss, Sampling Churn, and Option Capture in Distilled Small Language Models

Plawan Kumar Rath

cs.CLMay 19, 2026

What the Abstention Option Measures: Refusal Loss, Sampling Churn, and Option Capture in Distilled Small Language Models

Authors: Plawan Kumar Rath

Organizations: Meta

Abstract

Small instruction-tuned language models reach phones, cars, and offline assistants through knowledge distillation, and whether a distilled model is fair enough to ship is often decided by a multiple-choice bias score on a benchmark that offers an abstention option. Three mechanisms move behavior around the abstention option: refusal loss, where the student stops declining on items its pre-distillation reference declined; sampling churn, the same change observed between two runs of the reference itself; and option capture, where the student takes an abstention option whenever one is offered. Injecting each mechanism into a reference's own records shows that aggregate scores see only outright loss, per-item scores see churn and remapping, and only conditioning on committed answers stops moved refusal mass from reading as fairness. Resampling a reference across inference seeds already turns 10% to 27% of its refusals into stereotype answers, so a per-item refusal metric reported without that floor overstates damage. Option capture is what corpus repair teaches: students trained with refusal exemplars take an offered abstention option on up to 95% of science questions where abstention is never correct, and the habit carries to three further benchmarks. Free-form evaluation shows that the refusals multiple choice loses are not where free-form stereotyping occurs. We distill these findings into a per-condition report anchored to the reference, with measured floors and ceilings, and apply it to four student families against a fine-tuning control: distillation keeps fewer of the reference's refusals than fine-tuning on the same prompts where the reference's refusals are reliable, by a few points, while the common metrics report improvements whenever refusal mass moves. A bias score quoted for a distilled model should say which of these it reports.

Figures & tables

Figure 1. Which metric sees which mechanism. Each row injects one mechanism into the reference’s own records (SmolLM2-1.7B, H100 stack, 12,148 items per split, 100 item bootstraps) at a 5% rate; each cell is the share of replicates in which the metric fired outside its seed-noise band (rows 1 and 2) or reported a spurious improvement (row 3). Hatched cells are metric-mechanism pairs the simulation does not define. Full curves in Figure 9 . Source: results/tables/validity_power.csv .

	no abstention option		abstention option offered
configuration	accuracy	parse fail	accuracy	picks abstention
reference (SmolLM2-1.7B)	0.76	0.0	0.73	14.0
SFT control	0.76	2.2	0.76	3.2
response-KD (Mistral-7B)	0.79	0.1	0.77	4.4
response-KD (Gemma-2-9B)	0.64	16.6	0.48	30.0
letter 1%	0.77	1.6	0.34	63.0

Table 1. ARC-Easy probe on the stored SmolLM2-1.7B checkpoints (2,376 test items, one seed, MLX inference): four-option science questions where abstention is never correct, without and with an appended “Cannot be determined” option. Percentages except accuracy. Injected arms keep the format and take the option.

SmolLM2-1.7B student (H100 checkpoints; reference uptake 7 / 13 / 1%)
model	ARC-Challenge	MMLU	CommonsenseQA	capture
reference	7.1 (-1)	13.0 (1)	0.8 (-2)	no
SFT control	6.0 (5)	8.6 (3)	0.0 (-1)	no
response-KD (Mistral-7B)	1.5 (1)	3.1 (0)	0.0 (-1)	no
response-KD (Gemma-2-9B)	11.4 (-0)	21.7 (5)	0.0 (-4)	no
paired injection 0.1%	16.0 (8)	18.6 (8)	0.0 (-1)	yes

Table 2. Abstention-option capture beyond BBQ: uptake of an appended “Cannot be determined” option (percent of questions) on three benchmarks where abstention is never correct, with the accuracy drop in points from the plain to the with-option prompt in parentheses; one inference seed, temperature 0.3 (ARC-Challenge test

n=1{,}140

; MMLU validation

n=1{,}531

; CommonsenseQA validation

n=1{,}221

). Capture: uptake more than 5 pp above the model’s own reference on at least two benchmarks (pre-registered criterion). Every SmolLM2-1.7B model takes the option on 0% of CommonsenseQA items, whose six-option prompt uses a letter the student never saw. The full listing of every checkpoint and the teachers is Table 11 in Appendix E . Source: results/tables/probes_capture.csv .

Gemma-2-2B (teacher Gemma-2-9B; reference USR 77%, floor 0.3%); $\rho$ ceiling 0.99
configuration	parsed	excess silence-loss	lost to any	lift ratio [CI]	$\rho$ /ceil.	committed-answer override	acc.
reference	100	–	–	–	–	11.5	74.5
SFT control (3 seeds)	84 to 97	3.0 to 6.4	6.1 to 10.4	15.5 to 20.6 [13.6, 23.3]	0.57 to 0.63	15.0 to 18.1	66.9 to 72.4
response-KD (3 seeds)	100	5.8 to 8.0	9.8 to 14.9	17.3 to 24.8 [15.3, 28.0]	0.50 to 0.71	15.2 to 18.6	69.1 to 74.2
Phi-3.5-mini (3.8B) (teacher Gemma-2-9B; reference USR 83%, floor 0.4%); $\rho$ ceiling 0.99
reference	100	–	–	–	–	1.6	89.7

Table 3. Distillation against a same-stack fine-tuning control on the H100 stack, four student families (ranges over training seeds where more than one was run). Parsed: share of ambiguous records with a parseable letter, over which every ambiguous quantity is computed. Excess: silence-loss (reference unknown to stereotype, BBQ-ambig, 12,148 items, five inference seeds) minus the reference’s own churn floor, in pp. Lost to any: share of reference refusals that became any committed answer, stereotype or anti-stereotype. Lift ratio: the candidate’s lift (silence-loss over its own stereotype rate) divided by the reference’s lift under resampling, item-level delta-method 95% CI (the bracket spans the lowest lower and highest upper bound across seeds); 1 means the candidate’s answers are as related to where the reference refused as the reference’s own resamples.

\rho

/ceil.: refusal-pattern preservation as a share of the reference’s test-retest ceiling. committed-answer override and accuracy on anti-stereotype-correct BBQ-disambig items (

n=6{,}034

, three seeds). Sources: results/tables/validity_transitions.csv , validity_retest.csv , famctl_.csv , logit_matched_.csv , seed_replication.csv .

Figure 2. Silence-loss per BBQ stereotype-target group for Gemma-2-2B under the SFT control and the Gemma-2-9B response distillation, one point per training seed (three each), H100 stack, five inference seeds, groups with at least 1,000 reference-refused seeds, Wilson 95% CIs. Source: results/tables/per_group_harm_gemma.csv .

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

configuration	prefix	keyword	char-scan (retired)	fail
gemma-2-9b-it (teacher)	99.9	0.0	0.1	0.0
mistral-7b-v0.3 (teacher)	100.0	0.0	0.0	0.0
phi-3.5-mini (teacher)	99.8	0.0	0.2	0.0
smollm2-1.7b-combined-gemma-2-9b-it	19.2	0.0	80.8	0.0
smollm2-1.7b-combined-mistral-7b-v0.3	100.0	0.0	0.0	0.0
smollm2-1.7b-combined-phi-3.5-mini	98.5	1.4	0.0	0.1

Appendix

Table 4. Parser branch shares on BBQ-ambig (percent of records) for every configuration: the three retained branches, the retired first-character scan that the corrected cascade no longer uses, and failures. Source: results/tables/v4_parser_branches.csv .

Figure 3. SmolLM2-1.7B, five seeds, 12,148 items per condition (MLX stack). (a) BBQ-disambig: dark bars condition on committed answers, light bars are unconditioned; the dashed line is the reference. (b) BBQ-ambig: silence-loss for the same trainings. The dotted line in both panels is the SFT control; the churn floor for (b) is 10.1% (Table 14 ).

	BBQ-ambig				BBQ-disambig (anti-correct items)
configuration	SRS	USR	silence-loss [CI]	$\rho$	uncond.	committed-answer override [CI]	commit	acc.	discrim.	fail
SmolLM2-1.7B baseline	49.5	4.6	–	–	43.9	44.5 [43.7, 45.2]	98.8	54.9	$+$ 3.5	0.0
$+$ SFT (Alpaca)	39.2	28.4	2.0 [1.6, 2.6]	0.37	32.4	36.9 [36.1, 37.6]	87.8	55.4	$+$ 16.0	0.0
$+$ response-KD (Gemma)	22.2	59.2	1.2 [0.8, 1.7]	0.24	23.8	39.9 [38.9, 40.8]	59.6	35.9	$+$ 19.0	4.5
$+$ response-KD (Mistral)	47.5	13.5	14.6 [13.4, 16.0]	0.44	35.9	37.0 [36.3, 37.8]	96.9	61.0	$+$ 10.4	0.0
$+$ response-KD (Phi)	48.7	8.9	23.8 [22.3, 25.4]	0.30	40.0	40.7 [40.0, 41.4]	98.1	58.2	$+$ 7.2	0.0

Appendix

Table 5. Consolidated results for the 1.7B-scale and larger students under the corrected parser cascade; 12,148 items per condition, five seeds (ambiguous) and three seeds (disambiguated). Ambiguous side: SRS, USR, silence-loss with Wilson 95% CI, refusal-pattern preservation

\rho

. Disambiguated side, anti-stereotype-correct items (

n=6{,}034

): unconditioned and committed-answer override with CI, commitment, accuracy. discrim. is USR(ambig) minus USR(disambig) in pp; fail is the unparseable share. Percentages except

\rho

SmolLM2-135M student, SmolLM2-1.7B teacher (same tokenizer)
configuration	parsed	SRS	USR	silence-loss	excess over floor [CI]	$\rho$
baseline	100.0	33.9	32.1	–	–	–
SFT control	99.8	38.2	23.7	23.0	4.5 [3.2, 5.9]	0.63
logit-KD	99.2	38.0	23.9	29.4	11.0 [9.6, 12.4]	0.24
combined-KD	99.3	40.0	20.7	34.3	15.9 [14.5, 17.3]	0.24
SmolLM2-360M student, SmolLM2-1.7B teacher (same tokenizer)

Appendix

Table 6. Matched-tokenizer distillation within a family (student and teacher share a tokenizer), trained and scored on the CUDA stack against a same-stack SFT control; BBQ-ambig, 12,148 items, five seeds; Wilson 95% CIs. With identical tokenizers logit-KD parses at 99% or better. Excess: silence-loss minus the reference’s own churn floor across inference seeds, normal 95% CI. Percentages except

\rho

Figure 4. (a) refusal-pattern preservation $\rho$ for the 22 valid configurations (SFT controls in olive, OLMo-2-1B in green, negative values in red). (b) Per-item USR of the reference against the Gemma-2-9B-distilled SmolLM2-1.7B ( $n=12{,}140$ items, five seeds); off-diagonal mass is refusal on different items.

Figure 5. Silence-loss against committed-answer override on the valid grid (

n=19

KD configurations). Sub-1.7B students sit at the 50% chance line for committed two-way answers.

method	scope	$n$	silence-loss	committed-answer override	$\rho$	accuracy
response-KD	all students	10	1.2 to 49.4	37.0 to 50.0	-0.04 to 0.59	35.9 to 61.0
response-KD	SmolLM2-1.7B only	3	1.2 to 23.8	37.0 to 40.7	0.24 to 0.44	35.9 to 61.0
combined-KD	all students	9	2.6 to 38.9	38.8 to 50.2	-0.10 to 0.54	30.6 to 52.4
combined-KD	SmolLM2-1.7B only	3	2.6 to 12.8	38.8 to 40.7	0.21 to 0.35	30.6 to 52.4
SFT control	all students	3	2.0 to 31.8	36.9 to 49.4	0.37 to 0.42	38.5 to 55.4
SFT control	SmolLM2-1.7B only	1	2.0 to 2.0	36.9 to 36.9	0.37 to 0.37	55.4 to 55.4

Appendix

Table 7. Per-method ranges (minimum to maximum over configurations) on the valid grid, MLX stack; silence-loss and

\rho

on BBQ-ambig, committed-answer override and accuracy on anti-stereotype-correct BBQ-disambig items. Source: results/tables/v5_grid_corrected.csv .

Figure 6. Decomposition of newly introduced stereotype answers for every response-KD configuration on the valid grid (item and seed pairs where the baseline did not choose the stereotype and the trained model did;

n

per bar in Appendix D ).

student	teacher	$n_{\mathrm{paired}}$	$n_{\mathrm{new\ bias}}$	transition %	Inherited	Amplified-vs-Anti	Filled-Silence
SmolLM2-1.7B	Gemma-2-9B	57,912	3,853	13.2	5.4	0.8	93.8
SmolLM2-1.7B	Mistral-7B	60,736	7,401	24.1	43.7	6.4	49.9
SmolLM2-1.7B	Phi-3.5-mini	60,625	6,327	20.7	11.1	9.4	79.5
SmolLM2-360M	Gemma-2-9B	54,723	10,514	29.1	4.9	0.3	94.7
SmolLM2-360M	Mistral-7B	59,166	11,835	30.5	28.9	12.9	58.2
SmolLM2-360M	Phi-3.5-mini	60,344	10,048	25.4	16.1	4.3	79.6

Appendix

Table 8. Cell decomposition of newly introduced stereotype answers for every response-KD configuration on the valid grid (MLX stack): (item, seed) pairs where the reference did not choose the stereotype and the trained model did, classified by the teacher’s answer on the same prompt; shares in percent. Source: results/tables/v5_cell_decomposition.csv .

arm	inj. %	USR	commit	acc.	$\rho$	silence-loss
SFT (no injection)	0	28.4	87.8	55.4	0.37	2.0
letter	1	99.5	1.6	1.2	0.02	0.0
letter	2	98.2	3.2	2.0	0.04	0.0
letter	5	99.8	0.3	0.2	0.01	0.0
letter	7	100.0	0.1	0.1	0.01	0.0
paired (SFT)	5	100.0	7.6	5.5	0.00	0.0

Appendix

Table 9. Corpus interventions on SmolLM2-1.7B (12,148 items per condition; USR and silence-loss on BBQ-ambig over five seeds; commitment and accuracy on anti-stereotype-correct BBQ-disambig items,

n=6{,}034

, three seeds). Every arm parses at least 99.7% valid. Percentages except

\rho

. Rows above the second rule were trained on the MLX stack; rows below on the H100 stack, whose SFT control is the comparator for them.

arm	seed	USR	excess silence-loss [CI]	$\rho$	committed-answer override	commit	acc.
SFT control	42	15.0	-6.1 [-8.9, -3.2]	0.53	40.2	95.2	56.9
response-KD (Mistral-7B)	42	14.4	-2.4 [-5.3, 0.5]	0.51	39.0	95.0	58.0
paired injection 0.5%	42	21.0	-8.4 [-11.2, -5.6]	0.48	41.6	93.3	54.4
self-replay 0.5%	42	8.1	11.4 [8.3, 14.5]	0.48	39.5	97.5	59.0
self-replay 1%	42	13.5	7.1 [4.1, 10.2]	0.43	39.1	94.5	57.5
reference-anchored ( $\lambda=0.5$ )	42	3.2	12.8 [9.7, 15.9]	0.63	41.5	99.5	58.2

Appendix

Table 10. Repair arms on SmolLM2-1.7B with the Mistral-7B teacher, H100 stack, three training seeds; BBQ-ambig over five inference seeds (excess over the reference’s churn floor of 14.5%, normal 95% CI), BBQ-disambig over three. Reference: USR 5%, committed-answer override 44.2%, accuracy 54.9%; the SFT control’s range across seeds is the scale of training noise (Section 7 ). Lift ratios for every arm are in results/tables/pccd_tally.csv . Source: results/tables/repairs.csv .

model	ARC-Challenge	MMLU	CommonsenseQA	capture
smollm2-1.7b-baseline	7.1 (-1)	13.0 (1)	0.8 (-2)	no
smollm2-1.7b-sft	6.0 (5)	8.6 (3)	0.0 (-1)	no
smollm2-1.7b-response-mistral	1.5 (1)	3.1 (0)	0.0 (-1)	no
smollm2-1.7b-response-gemma	11.4 (-0)	21.7 (5)	0.0 (-4)	no
paired_d0.1	16.0 (8)	18.6 (8)	0.0 (-1)	yes
paired_d0.25	24.7 (12)	24.2 (11)	0.0 (-1)	yes

Appendix

Table 11. Uptake of an appended “Cannot be determined” option (percent of questions) on three benchmarks where abstention is never correct, with the accuracy drop in points from the plain to the with-option prompt in parentheses; one seed, temperature 0.3 (ARC-Challenge test

n=1{,}140

; MMLU validation

n=1{,}531

; CommonsenseQA validation

n=1{,}221

). Capture: uptake more than 5 pp above the model’s own reference on at least two benchmarks (pre-registered). Source: results/tables/probes_capture.csv .

Figure 7. Silence-loss per BBQ stereotype-target group for SmolLM2-1.7B under the SFT control and the three response distillations (MLX stack); five seeds, 12,148 ambiguous items, groups with at least 20 reference-refused seeds, Wilson 95% CIs. Source: results/tables/per_group_harm.csv .

category	SFT	Mistral-7B	Phi-3.5-mini	Gemma-2-9B
Age	13.1 (214)	27.6 (214)	47.7 (214)	6.3 (207)
Gender identity	1.4 (655)	7.6 (655)	26.4 (655)	0.2 (639)
Race ethnicity	0.7 (980)	18.1 (980)	24.3 (980)	0.3 (948)
Religion	0.0 (51)	2.0 (51)	21.6 (51)	0.0 (50)
SES	1.4 (920)	13.6 (920)	16.0 (920)	1.8 (845)

Appendix

Table 12. Silence-loss per BBQ category for SmolLM2-1.7B under the SFT control and the three response distillations (MLX stack, five seeds), with the number of reference-refused (item, seed) pairs in parentheses. Source: results/tables/per_group_harm.csv .

category	SFT control	response-KD	$n$
Age	4.0 to 8.1	10.1 to 13.0	5,823
Gender identity	5.0 to 7.7	8.5 to 9.8	9,454
Race ethnicity	3.1 to 5.1	4.4 to 8.6	11,277
Religion	2.3 to 5.2	3.1 to 6.6	2,459
SES	1.9 to 7.0	4.8 to 5.3	13,767

Appendix

Table 13. Silence-loss per BBQ category for Gemma-2-2B under the SFT control and the Gemma-2-9B response distillation, range over three training seeds (H100 stack, five inference seeds);

n

is the number of reference-refused (item, seed) pairs per category. Source: results/tables/per_group_harm_gemma.csv .

Figure 8. Opener-refusal rate across the four training corpora (51,760 responses each).

Figure 9. Detection rates of each metric under damage injected into the reference’s own records (SmolLM2-1.7B, H100 stack, 12,148 ambiguous and 12,148 disambiguated items; 100 item bootstraps per rate; dotted line is the 5% false-alarm rate). Source: results/tables/validity_power.csv .

stack	candidate	silence-loss	floor	excess [95% CI]	reverse
MLX	SFT control	2.0	10.1	-8.1 [-10.6, -5.5]	24.0
MLX	response-KD (Mistral-7B)	14.6	10.1	4.5 [1.7, 7.3]	10.1
MLX	response-KD (Gemma-2-9B)	1.2	10.1	-8.9 [-11.4, -6.4]	56.0
MLX	response-KD (Phi-3.5-mini)	23.8	10.1	13.7 [10.8, 16.6]	6.5
H100	SFT control, seed 42	8.5	14.5	-6.1 [-8.9, -3.2]	10.5
H100	SFT control, seed 43	11.6	14.5	-2.9 [-5.8, 0.0]	10.8

Appendix

Table 14. Silence-loss against its churn floor. Floor: the reference’s own silence-loss across disjoint inference seeds (mean over seed pairs; BBQ-ambig, 12,148 items, five seeds). Excess: candidate minus floor with a normal 95% CI. Reverse: share of the reference’s stereotype answers that became the unknown option. Source: results/tables/validity_transitions.csv .

reference (stack)	USR	items never refused	split-half $\rho$	ceiling
smollm2-1.7b (MLX)	4.6	93	0.88	0.93
smollm2-1.7b (H100)	5.4	90	0.74	0.85
smollm2-360m (MLX)	31.3	35	0.53	0.69
smollm2-135m (MLX)	31.9	48	0.77	0.87
smollm2-360m (H100)	32.1	27	0.37	0.54
smollm2-135m (H100)	32.1	41	0.65	0.79

Appendix

Table 15. Test-retest reliability of each reference model’s per-item refusal rate on BBQ-ambig (12,148 items): mean Spearman

\rho

over the ten two-against-three splits of its five inference seeds, and the Spearman-Brown ceiling for the five-seed rate the report uses. Source: results/tables/validity_retest.csv .

Refusal Behavior in Language Models

Refusal behavior in language models concerns when systems decline requests, how they detect risk, and how to balance safety with helpfulness.

See all 148 papers

Smaller Models, Better Rejects: Preference Distillation Scaling
Sep 30, 2026Rui Cai, Wenhui Zhu, Xiwen Chen +15Direct Preference Optimization Knowledge Distillation
AI Agentic Modeling and Foundation Team, LinkedIn · University of California, Davis · Arizona State University +3
Disposition Distillation at Small Scale: A Three-Arc Negative Result
Apr 13, 2026Hari SadasivanLanguage Model Distillation
Tinman Lab
Retrospective Distillation Attribution via Normalized Response Similarity
Sep 26, 2026Minwoo Jang, Jaechang Kim, Minhyeon Oh +2Language Model Auditing Knowledge Distillation
Graduate School of Artificial Intelligence, POSTECH, South Korea · POSTECH Institute of Artificial Intelligence, South Korea · Department of Computer Science and Engineering, POSTECH, South Korea

Sep 30, 2026J/K move · Enter open · S save

Smaller Models, Better Rejects: Preference Distillation Scaling

Rui Cai, Wenhui Zhu, Xiwen Chen, Jincheng Cao+14

AI Agentic Modeling and Foundation Team, LinkedIn · University of California, Davis · Arizona State University+3

Direct Preference Optimization Knowledge Distillation

Open paper AskPDF arXiv

Preference distillation typically treats a teacher response as preferred and the student's own response as rejected. This assumes that self-generated failures are the most informative negatives and that rejects must come from a model at least as large as the student, making generation costly at scale. We find neither assumption holds: across students from 7B to 72B, smaller frozen models generate rejects with less inference compute yet train stronger students than self-generated rejects, before and after sequence-level knowledge distillation, on code generation and mathematical reasoning. To explain this result, we derive a finite-horizon utility bound for Direct Preference Optimization in a linearized feature model. The bound characterizes favorable reject distributions and motivates three interventions. First, mixing rejects from smaller and student-scale models improves performance as the smaller model's share increases. Second, reassigning rejects to other prompts and shuffling their code tokens still outperform length-matched gibberish, showing that task structure contributes to reject utility. Third, selecting candidates with lower likelihood under the reference policy improves net transfer when higher-likelihood candidates provide less useful contrast. Lower-likelihood selections outperform higher-likelihood ones for every source. These results suggest that effective rejects preserve task structure while limiting coupling to the reference policy, and that smaller frozen models can provide them at low cost.

Abstract

Refusal Behavior in Language Models

Explore similar work

Smaller Models, Better Rejects: Preference Distillation Scaling

Disposition Distillation at Small Scale: A Three-Arc Negative Result

Retrospective Distillation Attribution via Normalized Response Similarity