Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models

Chao Xue; Yao Wang; Mengqiao Liu; Di Liang; Xingsheng Han; Peiyang Liu; Xianjie Wu; Chenyao Lu

cs.CLApr 11, 2026

Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models

Authors: Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, +6 more

Organizations: University of New South Wales, Australia · Tencent Hunyuan, China · Tencent Yuanbao, China · Peking University, China · UESTC, China

Abstract

Supervised Fine-Tuning (SFT) is the standard approach for adapting large language models (LLMs) to downstream tasks. However, we observe a persistent failure mode: even after convergence, models often fail to correctly reproduce a subset of their own supervised training data. We refer to this behavior as the Incomplete Learning Phenomenon(ILP). This paper presents the first systematic study of ILP in LLM fine-tuning. We formalize ILP as post-training failure to internalize supervised instances and demonstrate its prevalence across multiple model families, domains, and datasets. Through controlled analyses, we identify five recurrent sources of incomplete learning: (1) missing prerequisite knowledge in the pre-trained model, (2) conflicts between SFT supervision and pre-training knowledge, (3) internal inconsistencies within SFT data, (4) left-side forgetting during sequential fine-tuning, and (5) insufficient optimization for rare or complex patterns. We introduce a diagnostic-first framework that maps unlearned samples to these causes using observable training and inference signals, and study several targeted mitigation strategies as causal interventions. Experiments on Qwen, LLaMA, and OLMo2 show that incomplete learning is widespread and heterogeneous, and that improvements in aggregate metrics can mask persistent unlearned subsets. The findings highlight the need for fine-grained diagnosis of what supervised fine-tuning fails to learn, and why.

Figures & tables

Figure 1: Schematic illustration of the incomplete learning phenomenon, where testing the model on the initial training set after fine-tuning reveals that certain samples or patterns were not effectively learned during SFT.

Figure 2: The Incomplete Learning framework consists of three stages: (1) fine-tune on SFT data; (2) detect unlearned samples via re-evaluation; (3) calibrate model and data to fix them.

Figure 3: Unlearned sample attribution framework.

Figure 4: Performance improvements achieved by introducing Continued Pre-Training (CPT). Results demonstrate consistent accuracy gains across the medical (MedQA), legal (LegalBench), and financial (FinanceBench) domains.

Model	ARC	CommonQA	SocialIQA	MedMCQA
Qwen 7B	68.1 $\rightarrow$ 70.9 (+2.8)	74.5 $\rightarrow$ 76.9 (+2.4)	70.3 $\rightarrow$ 72.2 (+1.9)	61.2 $\rightarrow$ 63.7 (+2.5)
Qwen 14B	71.2 $\rightarrow$ 73.7 (+2.5)	76.8 $\rightarrow$ 79.1 (+2.3)	72.1 $\rightarrow$ 74.0 (+1.9)	63.1 $\rightarrow$ 65.3 (+2.2)
LLaMA2 7B	66.3 $\rightarrow$ 68.8 (+2.5)	72.8 $\rightarrow$ 75.3 (+2.5)	68.9 $\rightarrow$ 71.1 (+2.2)	58.9 $\rightarrow$ 61.4 (+2.5)
LLaMA2 13B	69.1 $\rightarrow$ 71.3 (+2.2)	75.2 $\rightarrow$ 77.3 (+2.1)	70.5 $\rightarrow$ 72.1 (+1.6)	61.1 $\rightarrow$ 63.0 (+1.9)

Table 1: Accuracy (%) before and after Continued Pre-Training (CPT) on four knowledge-intensive benchmarks. Improvements brought by CPT are highlighted in color and remain consistent across model sizes and domains.

Experiment	Qwen-7B	Qwen-14B	LLaMA-7B	LLaMA-13B
Knowledge conflict	82.3 $\rightarrow$ 85.1 (+2.8)	84.5 $\rightarrow$ 87.2 (+2.7)	81.8 $\rightarrow$ 84.3 (+2.5)	83.6 $\rightarrow$ 86.5 (+2.9)
Left-side forgetting	78.5 $\rightarrow$ 79.8 (+1.3)	79.3 $\rightarrow$ 80.5 (+1.2)	77.9 $\rightarrow$ 79.1 (+1.2)	78.7 $\rightarrow$ 80.2 (+1.5)
Insufficient learning	88.2 $\rightarrow$ 90.1 (+1.9)	89.5 $\rightarrow$ 91.3 (+1.8)	87.8 $\rightarrow$ 89.7 (+1.9)	88.9 $\rightarrow$ 90.8 (+1.9)

Table 2: Performance improvements across baseline models after applying optimization strategies for resolving knowledge conflicts, mitigating left-side forgetting, and addressing insufficient learning.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domain	Sample Size	Evaluation
MedQA	Medical	12,873	Accuracy
LegalBench	Legal	8,452	F1-score
TechFAQ	Technology	5,621	pass@5
FinanceBench	Finance	10,150	EM

Appendix

Table 3: Statistics of the SFT datasets and their corresponding evaluation metrics.

	$\displaystyle\mathcal{K}_{\text{blind}}=\{k\mid$	$\displaystyle\text{pass@10}(k)<0.2$
		$\displaystyle\land\text{BoN-5 Acc}(k)<0.1\}.$

Appendix

Algorithm 1 Knowledge Continue Pre-train

Dataset	Method	Acc (%)	pass@10	Learn Rate (%)
MedQA	Baseline (2 epoch SFT)	0.0	0.0	65.3
	Increase epoch to 10	+1.2	+1.5	66.8
	Continued pre-training + SFT	+8.3	+10.5	82.1
LegalBench	Baseline (2 epoch SFT)	0.0	0.0	62.5
	Increase epoch to 10	+1.0	+1.3	63.8
	Continued pre-training + SFT	+7.9	+9.8	80.5

Appendix

Table 4: Comparison of SFT performance by increasing training epochs and applying continued pre-training (CPT + SFT) across four datasets. Performance improvements are highlighted in red.

Dataset	Field	Size	Evaluation
ARC	Science	7,787	Accuracy
Common	Commonsense	12,247	Accuracy
SocialIQA	Social	33,410	Accuracy
MedMCQA	Medical	187,995	Accuracy

Appendix

Table 5: SFT Dataset statistics and evaluation index.

Model	ARC (+CPT)	CommonQA (+CPT)	SocialIQA (+CPT)	MedMCQA (+CPT)
Qwen 7B	12.3% $\rightarrow$ 8.8% (–3.5)	10.1% $\rightarrow$ 7.3% (–2.8)	11.7% $\rightarrow$ 8.8% (–2.9)	14.5% $\rightarrow$ 10.7% (–3.8)
Qwen 14B	11.2% $\rightarrow$ 8.0% (–3.2)	9.3% $\rightarrow$ 6.8% (–2.5)	10.8% $\rightarrow$ 8.2% (–2.6)	13.1% $\rightarrow$ 9.6% (–3.5)
LLaMA2 7B	13.1% $\rightarrow$ 9.5% (–3.6)	10.7% $\rightarrow$ 7.7% (–3.0)	12.3% $\rightarrow$ 9.2% (–3.1)	15.2% $\rightarrow$ 11.2% (–4.0)
LLaMA2 13B	12.5% $\rightarrow$ 9.2% (–3.3)	9.8% $\rightarrow$ 7.1% (–2.7)	11.5% $\rightarrow$ 8.7% (–2.8)	14.3% $\rightarrow$ 10.6% (–3.7)

Appendix

Table 6: Relative reduction of conflict rates on the SFT dataset before and after Continued Pre-Training (CPT) for each model. Negative improvements indicate a decrease in conflict rates, consistently observed across benchmarks and model sizes.

Dataset	Field	Size	Evaluation
SQuAD	General	150,000	EM
CoQA	Conversational	127,000	F1
TriviaQA	Trivia	95,000	Acc
Natural Questions	Web Search	307,373	EM

Appendix

Table 7: SFT Dataset statistics and evaluation index.

Experiment	Qwen-7B	Qwen-14B	LLaMA-7B	LLaMA-13B
Knowledge conflict	82.3% $\rightarrow$ 85.1% (+2.8)	84.5% $\rightarrow$ 87.2% (+2.7)	81.8% $\rightarrow$ 84.3% (+2.5)	83.6% $\rightarrow$ 86.5% (+2.9)
+ deletion	82.3% $\rightarrow$ 83.8% (+1.5)	84.5% $\rightarrow$ 85.9% (+1.4)	81.8% $\rightarrow$ 83.2% (+1.4)	83.6% $\rightarrow$ 85.1% (+1.5)
+ grouping	82.3% $\rightarrow$ 84.5% (+2.2)	84.5% $\rightarrow$ 86.6% (+2.1)	81.8% $\rightarrow$ 83.9% (+2.1)	83.6% $\rightarrow$ 85.8% (+2.2)

Appendix

Table 8: The indicators of sub-optimization strategies (deletion and grouping) applied to four baselines are shown.

Dataset	Field	Data Size	Evaluation
CNN/DailyMail	News	312,000	Acc
XSum	Extreme	226,000	Acc
OpenWebText	Language	800,000	Acc

Appendix

Table 9: SFT Dataset statistics and evaluation index.

Type	Description	Proportion
I. Base Model Knowledge Limitations	$\mathcal{P}_{\text{exist}}(x)=0$ generate wrong predict	18.7%
II. Conflicts Between SFT and Base Model	$D_{\text{JS}}>0.3$ cognitive conflict	13.2%
III. Knowledge Conflicts Between SFT Data	Wrong answer or multiple positions	14.1%
IV. Left-side Forgetting	Previous training data is forgotten	17.4%
V. Insufficient Training	SFT data is not fully learned	14.6%

Appendix

Table 10: Classification of unlearned phenomena in SFT and their corresponding proportions.

Position	ROUGE-L	Strategy	Gain
First 10% data	0.41	0.53	+29%
Middle data	0.57	0.59	+3.5%
Last 10% data	0.61	0.60	-1.6%

Appendix

Table 11: ROUGE-L results and gain comparison.

Dataset	Field	Data Size	Evaluation
AG News	News	120,000	Acc
IMDB	Movie Reviews	50,000	Acc
MultiNLI	NLI	433,000	Acc
QQP	Quora	404,000	Acc

Appendix

Table 12: SFT Dataset statistics and evaluation index.

Dataset Type (SFT)	Total SFT Entries	Non-Existence Rate (%)	Conflict Rate (%)
General Ability	10,000	18.6	13.8
Reasoning Knowledge	8,000	13.1	10.3
Professional Knowledge	9,500	27.4	18.4
Multilingual Knowledge	7,500	16.5	15.0
Total	35,000	19.3	14.5

Appendix

Table 13: Analysis of Knowledge Existence and Conflict between SFT Data and OLMo2 Pre-training Data.

Evaluation Dimension	Benchmark	Non-Existence Rate	Conflict Rate	Performance Change
		(SFT Data, %)	(SFT Data, %)	after CPT ( $\Delta\%$ )
General Ability	MMLU	12.7	8.3	-3.5
General Ability	AGIEval	15.2	6.8	-2.9
Reasoning Ability	BBH	9.4	4.1	-1.2
Professional Knowledge	GPQA	21.5	11.6	-6.8
Professional Knowledge	NQ	18.3	9.7	-4.3

Appendix

Table 14: CPT Performance Change on OLMo2-7B Across Evaluation Dimensions

Conflict Type	Example Scenario (Query/Context)	Pre-trained OLMo2 Knowledge Tendency (Illustrative Output/Bias)	SFT Knowledge Version (Target Output/Fact)	OLMo2 Output Tendency (Post-CPT)
Timeliness Conflict	Query: "Who is the current US President?" (SFT data updated to 2023 context)	Might output a president reflecting its pre-training data cutoff (e.g., "Donald Trump").	Specifies the president as per 2023 SFT data (e.g., "Joe Biden").	Increased tendency to output the SFT-aligned, more current president.
Disciplinary Controversy	Query: "What is the optimal theoretical path for quantum gravity?"	May favor a historically prominent theory (e.g., "String Theory").	SFT data emphasizes an emerging perspective (e.g., "Loop Quantum Gravity").	Output may present a more balanced view, acknowledge multiple perspectives, or lean towards the SFT-emphasized theory.
Multilingual Ambiguity / Geo-specificity	Query (SFT in Chinese): " 苹果公司总部的坐标是什么？" (Coordinates of Apple Inc. headquarters?)	Primarily processes based on English name or common knowledge, may struggle with direct Chinese geo-entity.	SFT provides query/context with the Chinese geographical name " 库珀蒂诺" (Cupertino).	Improved understanding and response generation within the Chinese language context for the query.
Cross-cultural Differences	Query: "Meaning of a headshake gesture in India."	Default interpretation might be Western-centric (e.g., negation).	SFT provides context for South Asian interpretation (e.g., affirmation or other nuances).	Output demonstrates more context-dependent judgment, aligning with the SFT-provided cultural nuance.
Terminology Evolution	Query: "Describe methods for neural network regularization."	May primarily list older, well-established methods (e.g., "Dropout").	SFT introduces or emphasizes newer terminology/methods (e.g., "LayerScale").	Output incorporates or gives due consideration to newer terminology/methods, possibly alongside established ones.
Regional Legal Differences	Query: "Age limit for data privacy protection of minors in [Specific Region]."	May default to a widely known regulation (e.g., GDPR: 16 years).	SFT specifies a different age for the particular region mentioned (e.g., China: 14 years).	Adjusts response based on the specific geographical context provided in the SFT data or query.

Appendix

Table 15: Typical Case Analysis of Knowledge Conflict Resolution in OLMo2-7B via CPT.

Explore similar work

Apr 15, 2026

(How) Learning Rates Regulate Catastrophic Overtraining

Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a helpful assistant. At the same time, SFT may harm the fundamental capabilities of an LLM, particularly after long pretraining: a phenomenon known as catastrophic overtraining (Springer et al., 2025). To understand overtraining, we first investigate catastrophic forgetting in finetuning through the lens of implicit regularization of the learning rate. For models trained to the same SFT loss, we identify how the learning rate mediates optimization: finetuning with large and small steps converges to qualitatively different models. Next, we link forgetting to overtraining: learning rate decay increases the sharpness of the pretrained model, which in turn exacerbates catastrophic forgetting during SFT, leading to overtraining. Our findings paint a picture of the overtraining mechanism in LLMs and broadly contribute to the understanding of the interplay between optimization dynamics during pretraining and finetuning.

Mark Rofin, Aditya Varre, Nicolas Flammarion

EPFL

LLM Training Supervised Fine-Tuning Neural Network Optimization Learning Rate Scheduling Catastrophic Forgetting

Ask PDF

May 18, 2026

Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective

This paper explores a scientific question in supervised fine-tuning (SFT): why SFT is broadly effective for small-scale deep neural networks, yet can produce inconsistent or even detrimental effects when applied to large language models (LLMs). Recent advances in interaction-based explanations suggest that interactions between words/tokens provide a faithful metric for quantifying the inference patterns encoded by LLMs. We find that the evolution of interactions during SFT can effectively explain the inconsistent effectiveness of SFT for LLMs. Specifically, we find that (1) SFT primarily removes noise-like interactions, while rarely acquiring reliable new interactions. (2) This denoising stage is extremely brief, after which continued fine-tuning tends to introduce overfitted interactions. We validate these findings across multiple LLMs and datasets. Our findings provide new insights into early stopping and offer practical guidance for LLM training.

Junpeng Zhang, Lei Cheng, Guoxi Zhang +3

Shanghai Jiao Tong University · Beijing Institute for General Artificial Intelligence · UniDT +1

Supervised Fine-Tuning LLM Fine-Tuning

Ask PDF

May 6, 2026

Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuning

Reinforcement fine-tuning (RFT) has become a core paradigm for post-training large language models, yet its training process remains highly fragile. Existing efforts mainly improve reliability at the system level or address specific issues in individual subproblems by modifying RFT algorithms. Despite their effectiveness, they largely overlook the problem of failure management at the training-process level. When training goes wrong, practitioners still rely heavily on expert-driven manual inspection and correction, and automatic failure management for RFT remains largely unexplored. In this paper, we take a first step toward systematic failure management for reinforcement fine-tuning. To understand the empirical structure of RFT failures, we first construct RFT-FaultBench, the first benchmark for fine-grained failures in reinforcement fine-tuning, covering 5 fault families, 16 fault types, 779 training runs, 22,549 train-step records, and 1,457,288 trajectory-level records. Based on this benchmark, we conduct a comprehensive empirical study showing that RFT failures are both observable from training dynamics and distinguishable through their empirical fault fingerprints. Building on these findings, we propose RFT-FM, an automatic failure management framework for reinforcement fine-tuning that unifies anomaly detection, failure diagnosis, and auto remediation in a closed loop. Experimental results show that RFT-FaultBench is neither trivial nor saturated: it exhibits clear anomaly structure while still posing substantial challenges, especially under subtle fault settings. Moreover, RFT-FM shows strong capability in detecting, diagnosing, and mitigating RFT failures.

Lingzhe Zhang, Tong Jia, Yunpeng Zhai +6

Peking University, Beijing, China · Alibaba Group, China · University of Illinois Chicago, United States

Reinforcement Learning RL Fine-Tuning

Ask PDF