Distillation, Self-Training, and the Fragility of Learned Chemical Reasoning

Anagha Aneesh; Sebastian Ibarraran; Frank Hu; Jason Liu; Grant Rotskoff

cs.AIOct 16, 2026

Distillation, Self-Training, and the Fragility of Learned Chemical Reasoning

Authors: Anagha Aneesh, Sebastian Ibarraran, Frank Hu, Jason Liu, Grant Rotskoff

Organizations: Stanford University

Abstract

Automating synthesis planning for small-molecule drug design requires chemical reasoners that are accurate as well as open to inspection and adaptation, yet frontier language models remain unreliable at predicting reaction outcomes and difficult to deploy, while small open-weight models lack the necessary domain reasoning. We show that frontier supervision can serve as a one-time bootstrap for open models on forward reaction prediction. Our models, Resonance, distill an open-weight teacher's reasoning traces and then learn from their own verified outputs, turning a small base model with near-zero accuracy into a capable reaction predictor, while a stronger student surpasses its teacher through distillation alone. Accuracy, however, does not bring robust reasoning: the small student inherits its teacher's single-pass style, self-training mainly sharpens answers it could already reach, and it readily abandons correct answers under pushback, suggesting that chemical reasoning models should be evaluated on the robustness of their reasoning as well as on the accuracy of their answers.

Figures & tables

Figure 1: The pipeline used to train Resonance. The teacher (Kimi K3) generates reasoning traces, provided with the recorded product visible, that are used to fine-tune the base model (base

\rightarrow

S1). The model is then improved on its own verified-correct traces (S1

\rightarrow

S2, rejection-sampling SFT) and its own preference-ranked traces (S2

\rightarrow

S3, DPO) before evaluation.

Figure 2: Top-1 exact-match accuracy on the USPTO-50k test set (

N=5{,}000

) for Resonance-8B at each stage (base

\rightarrow

S1 teacher SFT

\rightarrow

S2 rejection-sampling SFT

\rightarrow

S3 DPO), and for Kimi K3 (teacher, 57.4%), Claude Opus 5 (78.7%) and GPT-6-Astra (63.0%), all three evaluated zero-shot. Each stage improves on the previous one, and the two self-training stages together add 11.8 points without teacher supervision. Error bars are 95% Wilson intervals.

Figure 3: Accuracy by reaction class for Resonance-8B. (A) Student minus teacher zero-shot accuracy (Resonance-8B minus Kimi K3) at each stage; green marks a student advantage and tan a teacher advantage. (B) Student Top-1 accuracy by reaction class across training stages. Classes follow Table 2 , which also gives training and test counts; all 5,000 test reactions are included.

Figure 4: Instructions change reasoning length, not accuracy. Average reasoning tokens against change in Top-1 accuracy relative to an unprompted resample, for Resonance-8B at each stage, when instructed to commit to its first supported answer, consider alternatives, or think more . Baseline accuracy is 29.0%, 38.8% and 41.3% at S1, S2 and S3, respectively. No instruction improves accuracy at any stage. Error bars are

\pm 1

standard error over 8 draws per reaction, clustered by reaction.

Figure 5: Sensitivity to negative feedback grows with training. Change in Top-1 accuracy of Resonance-8B relative to a neutral “review your answer” turn when its previous answer is returned with mild doubt, a statement that it is wrong, a statement that an evaluator marked it wrong, or a statement that it is correct, regardless of its actual correctness. Baseline accuracy after the neutral review is 28.0%, 38.3% and 41.1% at S1, S2 and S3, respectively, with 8 draws per reaction. Negative feedback costs 3.2–3.7 points at S1 and 8.7–9.2 at S3; being told an answer is correct is the only intervention that helps, by less than one point. Error bars are

\pm 1

standard error over 8 draws per reaction, clustered by reaction.

Figure 6: Self-training sharpens answers the model could already reach. Plurality@

k

accuracy (the most frequent product among

k

samples) for Resonance-8B by reaction class and stage. Voting over nine S1 samples matches or exceeds single-sample S3 accuracy in 8 of the 10 classes with at least 40 test reactions; heterocycle formation remains unsolved at every

k

. Classes follow Table 2 .

Figure 7: Accuracy by reaction class for Resonance-27B. Bars show the base model, S1, S2, S3, DPO applied directly to S1 (skip-S2); Kimi K3 and Claude Opus 5 (zero-shot). Teacher SFT supplies nearly all of the gain in every class, and the later stages change accuracy by amounts within sampling noise. Opus 5 leads on heterocycle formation, while the student leads on reductions and oxidations, where the prompt usually names no reagent. Classes follow Table 2 .

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Question	Section	Supports
Setup
How were the data scored and the reaction classes assigned?	A.1	§ 3.1 , all per-class results
What exactly was trained, with what settings?	A.2 – A.3	§ 3.3 , reproducibility
What prompts were used?	A.4	§ 3.4 , § 4.3
What does each training stage contribute?
Does the base model fail only because of output format?	B.1	§ 4.1

Appendix

Table 1: Guide to the appendix.

Class	Abbrev.	$n$ (test)	$n$ (train)
Heteroatom alkylation/arylation	Alk./aryl.	1,479	11,736
Acylation	Acyl.	1,107	8,733
Deprotections	Deprot.	824	6,494
C–C bond formation	C–C	536	4,281
Reductions	Red.	457	3,542
Unclassified	Unclass.	184	1,684

Appendix

Table 2: Reaction classes in USPTO-50k. This scheme is used in every per-class figure and table; figures use the abbreviations. FG = functional group. Test and training labels both include the benzyl-ether correction. Unclassified = no rule of the classifier applies (see text).

	S1 (teacher SFT)		S2 (RS-SFT)		S3 (DPO)
Parameter	8B	27B	8B	27B	8B	27B
Fine-tuning	full	full (text)	full	LoRA	LoRA	LoRA
Epochs / steps	3 ep (3,750 st)	3 ep (3,750 st)	3 ep (2,229 st)	1 ep (3,086 st)	3,000 st (3.3 ep)	1 ep (378 st)
Learning rate	$2\!\times\!10^{-5}$	$2\!\times\!10^{-5}$	$\mathbf{1\!\times\!10^{-5}}$	$\mathbf{2\!\times\!10^{-5}}$	$2\!\times\!10^{-6}$	$2\!\times\!10^{-6}$
Warmup steps	100	100	50	50	0	0
Effective batch	32	32	64	32	32 pairs	32 pairs

Appendix

Table 3: Training configuration by model and stage. Differences between the models are in bold .

	S2 data	S3 data	Evaluation
Sampling model	S1	S2	each checkpoint
Samples per prompt	8	16	1 (8 for § 4.3 probes)
Temperature	0.9	0.7	0.6 (base 8B: 0, greedy)
Top- $p$ / top- $k$	0.95 / 20	0.95 / 20	0.95 / 20
Max new tokens	1,024	8,192	16,384 (8,192 for probes)
Context length	2,048	16,384 (8B) / 10,240 (27B)	16,384 (8B) / 20,480 (27B)

Appendix

Table 4: Sampling settings. Seeds: “fixed” means the same seed on every request.

	$k=0$	$k=4$	$k=8$
Compliance, all generations (%)	60.64	65.22	67.78
Compliance, finished only (%)	75.57	88.59	89.21
Invalid SMILES in tag (%)	19.4	8.3	8.1
Truncated (%)	19.8	26.4	24.0
Echo rate, % of valid	43.7	52.7	53.2
Top-1 accuracy (%)	1.32	1.20	2.02

Appendix

Table 5: Few-shot control on base Qwen3-8B (USPTO-50k test,

N=5{,}000

, greedy decoding, 16,384-token budget, short system prompt). The

k=0

column is a separate run from the headline base result (1.18%, long system prompt). The in-context examples are reasoning traces written by Claude Sonnet 4.6 for training reactions.

Model	S1	S1 $\to$ S2 $\to$ S3	S1 $\to$ S3 (skip S2)	Paired diff. [95% CI], $p$
Qwen3-8B	31.0	42.8	37.1	$-5.7$ [ $-6.9$ , $-4.6$ ], $p<10^{-20}$
Qwen3.8-27B	79.0	79.6	78.8	$-0.8$ [ $-1.5$ , $-0.1$ ], $p=0.026$
DPO pairs, 8B		28,747	26,661
DPO pairs, 27B		12,079	13,698

Appendix

Table 6: Skip-S2 ablation. DPO applied directly to S1, same S3 recipe as the full pipeline for each model (8B: 3,000 steps; 27B: 1 epoch). Paired difference = skip-S2

-

full pipeline.

	Qwen3-8B S1	Qwen3.8-27B S1
Training-set accuracy, per sample (%)	16.6	85.8
Reactions with $\geq$ 1 correct sample (%)	51.8	97.2
Reactions with all 8 correct (%)	0.3 (116)	65.5 (26,197)
Reactions with 1–7 correct (“mixed”, %)	51.5 (20,588)	31.7 (12,695)
Reactions with 0 correct (%)	48.2 (19,296)	2.8 (1,108)

Appendix

Table 7: Training-set difficulty after S1 (8 samples per training reaction; S2-data sampling settings of Table 4 ). A sample counts as correct if it is exactly correct, before any length filter.

Model	Comparison	Acc. A	Acc. B	Only A	Only B	$p$	$\Delta$ [95% CI]
8B	base $\to$ S1	1.2	31.0	25	1,517	$<10^{-300}$	$+29.8$ [28.5, 31.1]
8B	S1 $\to$ S2	31.0	38.9	433	828	$1\times 10^{-28}$	$+7.9$ [6.5, 9.3]
8B	S2 $\to$ S3	38.9	42.8	452	646	$6\times 10^{-9}$	$+3.9$ [2.6, 5.2]
27B	base $\to$ S1	26.3	79.0	99	2,735	$<10^{-300}$	$+52.7$ [51.2, 54.2]
27B	S1 $\to$ S2	79.0	79.3	164	181	0.39	$+0.3$ [ $-0.4$ , 1.1]
27B	S2 $\to$ S3	79.3	79.6	69	82	0.33	$+0.3$ [ $-0.2$ , 0.7]

Appendix

Table 8: Paired tests between stages (McNemar, continuity-corrected;

N=5{,}000

). “Only A/B”: reactions correct only under A/B. The 8B base model is decoded greedily; all other checkpoints at

T=0.6

Figure 8: Top-1 exact-match accuracy on the USPTO-50k test set (

N=5{,}000

) for Resonance-27B at each stage (base

\rightarrow

S1 teacher SFT

\rightarrow

S2 rejection-sampling SFT

\rightarrow

S3 DPO), and for Kimi K3 (teacher), Claude Opus 5 and GPT-6-Astra, all three evaluated zero-shot. Teacher SFT supplies nearly all of the gain, lifting the model above its teacher, and the two self-training stages change accuracy by less than one point. This is the 27B counterpart of Figure 2 ; the

y

-axis extends to 100%. Error bars are 95% Wilson intervals.

		Qwen3-8B				Qwen3.8-27B
Class	$n$	base	S1	S2	S3	S1	S3	Kimi K3	Opus 5	GPT-6
Heteroatom alk./aryl.	1,479	0.8	24.5	31.1	34.6	82.3	82.8	70.7	81.7	83.0
Acylation	1,107	2.6	39.2	51.8	53.7	87.7	88.2	82.7	90.3	92.3
Deprotections	824	0.2	37.3	50.5	59.3	75.4	77.1	22.9	81.3	13.8
C–C bond formation	536	1.1	34.9	38.4	40.5	82.6	81.5	81.9	88.2	89.0
Reductions	457	0.2	31.3	33.9	43.8	67.0	65.4	2.6	41.1	2.6

Appendix

Table 9: Top-1 accuracy (%) by class. All frontier models zero-shot.

N=5{,}000

Model	Classes	(i) Exact	(ii) Largest frag.	(iii) + no stereo	(iv) + tautomer
Kimi K3	All	57.4	57.6	60.4	60.6
	Reductions	2.6	2.6	2.8	2.8
	Oxidations	1.8	1.8	1.8	1.8
	Deprotections	22.9	22.9	23.5	23.5
	Protections	79.3	79.3	84.7	84.7
8B S3	All	42.8	42.8	44.4	44.4

Appendix

Table 10: Accuracy (%) under looser matching.

Model	Reductions	Oxidations	Deprotections
Kimi K3	413 / 445	50 / 54	577 / 635
GPT-6	440 / 445	55 / 55	687 / 710
Opus 5	10 / 269	1 / 34	0 / 154
8B S3	6 / 257	0 / 43	7 / 335

Appendix

Table 11: Incorrect predictions that repeat an input (count / all incorrect predictions in the class).

	8B S3 vs. Kimi K3	27B S3 vs. Kimi K3
Both correct	1,375 (27.5)	2,584 (51.7)
Only student correct	765 (15.3)	1,396 (27.9)
Only teacher correct	1,497 (29.9)	288 (5.8)
Both wrong	1,363 (27.3)	732 (14.6)
McNemar $\chi^{2}$ ( $p$ )	236.2 ( $3\times 10^{-53}$ )	727.7 ( $3\times 10^{-160}$ )
Jaccard overlap of correct sets	0.378	0.605

Appendix

Table 12: Paired outcomes against the teacher.

N=5{,}000

; counts with % of reactions.

Figure 9: The student inherits the corpus’s reasoning style, not the base model’s. Share of traces with any backtracking (left) and events per 1k output tokens (right), default prompt.

	Traces with any backtracking (%)	Events per 1k tokens
Qwen3-8B base (before training)	100	11.9
Kimi K3 SFT corpus (training data, $n=40{,}000$ )	10.6	0.23
S1	12.6	0.27
S2	13.6	0.32
S3	14.6	0.33

Appendix

Table 13: Backtracking by source (default prompt). The base model is sampled with the evaluation settings (

T=0.6

); its generations average 3,934 tokens.

Prompt	S1	S2	S3
Default (resample)	0 [488]	0 [469]	0 [475]
Commit to first answer	$-1.03$ ( $p<0.001$ ) [451]	$-0.82$ ( $p<0.001$ ) [432]	$-0.49$ ( $p=0.026$ ) [437]
Consider alternatives	$+0.14$ ( $p=0.56$ ) [516]	$-0.69$ ( $p=0.003$ ) [496]	$-0.46$ ( $p=0.042$ ) [505]
Think more	$-0.90$ ( $p<0.001$ ) [651]	$-0.26$ ( $p=0.28$ ) [608]	$-0.42$ ( $p=0.062$ ) [612]

Appendix

Table 14: Change in top-1 accuracy (points) vs. unprompted resample , with

p

-values. Baseline accuracy S1/S2/S3: 29.0 / 38.8 / 41.3%. Mean reasoning tokens in brackets.

Feedback	S1	S2	S3
Mild doubt	$-0.34$ ( $p=0.020$ )	$-0.71$ ( $p<10^{-5}$ )	$-2.10$ ( $p<10^{-26}$ )
Told wrong	$-3.71$ ( $p<10^{-31}$ )	$-4.13$ ( $p<10^{-45}$ )	$-9.22$ ( $p<10^{-134}$ )
Marked wrong (authority)	$-3.18$ ( $p<10^{-31}$ )	$-3.73$ ( $p<10^{-43}$ )	$-8.69$ ( $p<10^{-140}$ )
Told correct	$+0.66$ ( $p<10^{-11}$ )	$+0.47$ ( $p<10^{-7}$ )	$+0.36$ ( $p<10^{-3}$ )
“Told wrong” loss as a share of baseline accuracy
	13%	11%	22%

Appendix

Table 15: Change in top-1 accuracy (points) vs. neutral review , 8 draws per reaction. Baseline accuracy after neutral review S1/S2/S3: 28.0 / 38.3 / 41.1%. All cells remain significant under Holm–Bonferroni at

\alpha=0.05

	S1		S2		S3
Condition	C $\to$ I	I $\to$ C	C $\to$ I	I $\to$ C	C $\to$ I	I $\to$ C
Fresh resample	41.8	17.2	28.0	17.7	25.1	17.4
Neutral review	5.7	1.3	3.2	1.1	3.6	1.8
Mild doubt	10.6	2.8	7.8	2.9	11.7	3.9
Told wrong	31.0	6.3	21.5	6.0	32.3	6.4
Marked wrong	26.6	5.2	19.9	5.6	30.1	5.8

Appendix

Table 16: Answer flips relative to the first answer (%, mean over 8 draws). C

\to

I: share of initially correct answers that end incorrect. I

\to

C: share of initially incorrect answers that end correct. A fresh resample is the no-feedback reference.

Figure 10: Prompts raise backtracking without raising accuracy. Change in accuracy against backtracking events per 1k output tokens for (a) reasoning-length prompts and (b) feedback prompts, by stage. “Consider alternatives” raises backtracking about 2.4

\times

(0.27

\to

0.66 per 1k at S1) with no accuracy gain; “told wrong” raises it 8–18

\times

over the default resample (3.5–4.1

\times

over the neutral review) and produces the largest losses.

Stage	Condition	Any candidate	$\geq$ 1 change	Wrong $\to$ right	Right $\to$ wrong	Wrong $\to$ wrong	Final answer = last cand.
S1	Default	64.0	8.0	16.8	4.0	79.2	68.4
	Consider alternatives	58.2	7.8	17.0	1.4	81.6	68.9
	Think more	64.3	9.6	18.2	3.3	78.6	60.7
	Told wrong	51.4	11.1	13.3	8.5	78.2	38.7
S3	Default	58.5	7.9	25.9	2.2	71.9	62.9
	Consider alternatives	54.6	7.0	26.3	2.1	71.7	64.8

Appendix

Table 17: Structure changes within the trace (RDKit-canonicalized SMILES; first draw). Columns 3–4: % of traces; columns 5–7: % of change events; column 8: % of traces with

\geq

1 change.

Stage	Metric	$k=1$	$k=3$	$k=5$	$k=7$	$k=9$
S1	plurality@ $k$	28.7	35.4	41.3	44.2	46.1
	pass@ $k$	28.7	47.5	55.8	60.3	63.8
S2	plurality@ $k$	38.9	44.6	48.5	50.6	51.7
	pass@ $k$	38.9	55.0	61.2	65.0	67.4
S3	plurality@ $k$	41.6	46.9	50.0	52.1	54.0
	pass@ $k$	41.6	56.9	62.5	65.7	67.9

Appendix

Table 18: Plurality@

k

and pass@

k

(%), Qwen3-8B, all 5,000 reactions. Nine draws per reaction (the first answer plus 8 fresh resamples); pass@

k

is the empirical share of reactions with a correct answer among the first

k

draws.

Chemical Reasoning in LLMs

See all 13 papers

ChemOPD: Multi-Teacher On-Policy Distillation for Multi-Task Chemical Reasoning
Sep 27, 2026Yaoyao Xu, Xinjian Zhao, Xiaozhuang Song +2Chemical Reasoning in LLMs Reasoning Distillation
School of Data Science, The Chinese University of Hong Kong, Shenzhen · Shanghai Artificial Intelligence Laboratory
Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models
Jul 14, 2026Xingyu Dang, Haocheng Tang, Junmei Wang +1LLM Reasoning
Department of Computer Science, Princeton University, Princeton, NJ, 08540, United States · School of Pharmacy, University of Pittsburgh, Pittsburgh, PA, 15261, United States · Khoury College of Computer Science, Northeastern University, Boston, MA, 02115, United States +2
RetroReasoner: A Reasoning LLM for Strategic Retrosynthesis Prediction
Mar 13, 2026Hanbum Ko, Chanhui Lee, Ye Rin Kim +4AI-Assisted Scientific Research Language Modeling
Department of Artificial Intelligence, Korea University · Materials Intelligence Lab, LG AI Research · Department of Statistics, Korea University

Sep 27, 2026J/K move · Enter open · S save

ChemOPD: Multi-Teacher On-Policy Distillation for Multi-Task Chemical Reasoning

Yaoyao Xu, Xinjian Zhao, Xiaozhuang Song, Xuemin Chen+1

School of Data Science, The Chinese University of Hong Kong, Shenzhen · Shanghai Artificial Intelligence Laboratory

Chemical Reasoning in LLMs Reasoning Distillation Multi-Teacher Knowledge Distillation On-Policy Distillation

Open paper AskPDF arXiv

Large language models are increasingly expected to support diverse chemical reasoning capabilities within a unified model. One approach is to develop specialized capabilities separately and consolidate them through multi-teacher on-policy distillation, but this raises two questions: how should specialization be organized, and how should specialist guidance be integrated? We introduce ChemOPD, which addresses both. We estimate task affinities from supervised fine-tuning gradients and solve a constrained mixed-integer program(MIP) to construct partially overlapping specialist groups. During distillation, we retain a generalist teacher trained on all tasks so that specialist guidance supplements rather than replaces its supervision. Our anchor-residual objective gradually increases the routed specialist's contribution on student-generated responses. On ChemCoTBench, affinity-guided specialization produces task-dependent gains over the generalist teacher and improves several capabilities beyond semantic task grouping. Yet stronger teacher-side performance does not automatically yield stronger students: with the same specialists and routes, anchor-residual OPD improves most reported metrics over specialist-only distillation and realizes a larger share of the available teacher gains. These results highlight specialization and capability integration as connected but distinct design problems in chemical reasoning.

Abstract

Chemical Reasoning in LLMs

Explore similar work

ChemOPD: Multi-Teacher On-Policy Distillation for Multi-Task Chemical Reasoning

Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models

RetroReasoner: A Reasoning LLM for Strategic Retrosynthesis Prediction