Unifying and Beyond the Theory of Pervasive Capability Degradation in LLM Post-training SFT from Feature Interference

Hang Chen; Jiaying Zhu; Wenya Wang

cs.LGNov 3, 2026

Unifying and Beyond the Theory of Pervasive Capability Degradation in LLM Post-training SFT from Feature Interference

Authors: Hang Chen, Jiaying Zhu, Wenya Wang

Organizations: College of Computing and Data Science, Nanyang Technological University · School of Computer Science and Engineering, The Chinese University of Hong Kong

Abstract

Pervasive Capability Degradation (PCD) refers to the phenomenon in which supervised fine-tuning (SFT), while adapting a large language model (LLM) to a target task, often erodes the general capabilities acquired during pre-training. Prevailing explanations attribute PCD to macroscopic distribution shift and therefore lack microscopic manipulability, leaving existing mitigation methods without a complete theoretical foundation. In this paper, we first develop a bottom-up geometric theory of PCD grounded in feature interference under superposition: narrow-domain SFT data disrupt the almost-orthogonal feature structure formed during pre-training, and the resulting interference drives activations across decision hyperplanes in the output Laguerre-Voronoi geometry, flipping predictions on general tasks. This theory unifies existing methods into six paradigms and reveals both the source of their efficacy and their theoretical limits. We then prove that cross-entropy under one-hot supervision cannot resolve PCD on its own, and accordingly propose the AEGIS loss, which directly constrains the drift of activation margins along all competing token directions, thereby decoupling interference suppression from task adaptation. Across 3 model families and 14 SFT tasks, AEGIS outperforms representative methods of every paradigm on target tasks and pervasive capabilities simultaneously, at only about 1.3 times the computational cost of standard SFT. Its generality, scalability, and mechanistic interpretability offer a new perspective on fully resolving PCD.

Figures & tables

Notation	Domain / Dimension	Description
1. Model Architecture & Optimization
$\theta$	Parameter space	Model parameter set
$\mathcal{D}$	Dataset	Dataset of training samples
$N$	$\mathbb{N}^{+}$	Total number of samples in a dataset or feature set
$\mathcal{V},\,V$	Set, $\mathbb{N}^{+}$	Output vocabulary set and vocabulary size ( $V=\|\mathcal{V}\|$ )
$W,\,b$	Tensor / Vector	General model weight matrices and bias vectors

Table 1: Summary of primary notations and symbols used throughout this paper.

Figure 1: Illustration of real weight ( $W$ ) and virtual space weight ( $W^{virt}$ ). Each row of $W$ manages a subset of a few salient features, but each row of $W^{virt}$ manages one individual feature.

Figure 2: Geometric illustration of PCD in the output LVD (Eqs. (18)–(20)).

Figure 3: In the unembedding space, $\mathrm{Cell}_{j}$ ( light blue ) is bounded by hyperplanes separating token $j$ from competitors $k_{1}\sim k_{7}$ , where the blue dashed contour outlines the adaptive directional width derived from baseline activation $h_{0}$ .

Model	Method	CoLA		SST-2		MRPC		STS-B		QQP		MNLI		QNLI		RTE
Model	Method	MCC	P-Acc.	Acc.	P-Acc.	Acc./F1	P-Acc.	Corr.	P-Acc.	Acc./F1	P-Acc.	M-Acc.	P-Acc.	Acc.	P-Acc.	Acc.	P-Acc.
Mistral-7B	Vanilla	0.435	0.512	0.965	0.519	0.884	0.493	0.875	0.522	0.905	0.521	0.884	0.524	0.935	0.537	0.835	0.527
	PSFT	0.449	0.514	0.977	0.528	0.891	0.485	0.886	0.518	0.892	0.515	0.892	0.519	0.939	0.542	0.854	0.525
	SMART	0.447	0.519	0.971	0.537	0.889	0.491	0.881	0.526	0.906	0.524	0.888	0.514	0.928	0.539	0.869	0.534
	KnOTS	0.385	0.437	0.972	0.462	0.871	0.473	0.864	0.507	0.875	0.508	0.867	0.492	0.914	0.504	0.812	0.519
	DARE	0.398	0.463	0.947	0.465	0.865	0.466	0.859	0.501	0.869	0.486	0.871	0.488	0.905	0.511	0.819	0.504

Table 2: Main results on NLU tasks.

Model	Method	GSM8K		CodeAlpaca				Code-Bagel-Hermes		Infinity-Instruct-Code		Evol-Instruct		LogiQA2
Model	Method	Acc.	P-Acc.	Pass@1	Pass@5	Pass@10	P-Acc.	Token Acc.	P-Acc.	Token Acc.	P-Acc.	Win Rate	P-Acc.	Acc.	P-Acc.
Mistral-7B	Vanilla	0.375	0.391	0.085	0.131	0.145	0.375	0.445	0.381	0.488	0.421	0.467	0.423	0.612	0.445
	PSFT	0.374	0.399	0.084	0.134	0.151	0.396	0.416	0.385	0.495	0.422	0.461	0.421	0.618	0.432
	SMART	0.403	0.396	0.098	0.141	0.151	0.378	0.394	0.389	0.492	0.415	0.472	0.425	0.614	0.438
	KnOTS	0.326	0.357	0.064	0.117	0.123	0.326	0.384	0.354	0.457	0.372	0.455	0.394	0.547	0.405
	DARE	0.354	0.361	0.083	0.129	0.144	0.354	0.397	0.329	0.443	0.388	0.439	0.388	0.463	0.411

Table 3: Main results on mathematical reasoning, code generation, and logic & instruction-following tasks.

Model	Method	Total Time (GPU $\cdot$ h)	Total FLOPs	MFU	HFU	Peak Mem. (GiB)
Qwen2.5-3B	Vanilla	$10.2$	1.69( $\times 10^{17}$ )	23.5%	39.2%	59.9
	PSFT	25.3	3.13( $\times 10^{17}$ )	19.7%	38.4%	75.5
	DiFT	$3.5+12.7=26.3$	1.75( $\times 10^{17}$ )	23.6%	42.7%	61.2
	AEGIS	$0.98+20.2=21.1$	2.13( $\times 10^{17}$ )	12.0%	20.0%	60.8
Qwen2.5-7B	Vanilla	$35.2$	3.85( $\times 10^{17}$ )	15.6%	26.1%	147.7
	PSFT	$57.5$	6.72( $\times 10^{17}$ )	13.6%	28.4%	199.8

Table 4: Time and memory costs of SFT on GSM8K across Qwen2.5 model scales.

Method	RTE			GSM8K			CodeAlpaca
Method	Cross.	New $\downarrow$	Rec.	Cross.	New $\downarrow$	Rec.	Cross.	New $\downarrow$	Rec.
Base Model	18.569	–	–	18.569	–	–	18.569	–	–
Vanilla	16.668	5.700	8.659	4.373	2.321	5.748	5.915	1.941	4.516
PSFT	16.874	5.749	8.547	4.699	2.342	5.760	5.937	1.936	4.564
SMART	16.649	5.798	8.967	4.830	2.336	5.722	5.925	1.957	4.529
KnOTS	16.782	5.830	8.296	4.667	2.487	5.348	5.916	1.963	4.262

Table 5: Decision-edge crossing statistics on the pervasive dataset

\mathcal{D}_{P}

after SFT on RTE, GSM8K, and CodeAlpaca. All values are in units of

10^{6}

.All metrics are recorded top 50% competing tokens over

\mathcal{V}

Panel of Figure 4 — Figure 4: Loss and validation accuracy of Qwen2.5-7B during SFT with standard cross-entropy (Vanilla) and AEGIS on MRPC, GSM8K, CodeAlpaca, and LogiQA2. For each task, the left panel shows the total training loss together with the validation accuracy on the target task, and the right panel separates the training loss into the cross-entropy on the target task dataset $\mathcal{D}_{T}$ and on the pervasive dataset $\mathcal{D}_{P}$ .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Samples	Evaluation	#Eval. Samples	Metric
CoLA	9,594	CoLA	1,043	MCC
SST-2	68,221	SST-2	872	Accuracy
MRPC	4,076	MRPC	408	Accuracy / F1
STS-B	7,249	STS-B	1,500	Pearson / Spearman
QQP	404,276	QQP	40,430	Accuracy / F1
MNLI	402,517	MNLI	9,815	Matched accuracy

Appendix

Table G1 : Statistics of the target task datasets and their evaluation benchmarks. #Samples is the total number of instances used from each dataset (training, validation, and evaluation splits combined); for datasets evaluated on an external benchmark, it equals the number of training instances. For GLUE, the evaluation set is the official validation split ( validation_matched for MNLI).

Dataset	Prompt Template	Input Case	Output Case (Label)
GSM8K	Question: {question}\nAnswer:	Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?	␣Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72
CoLA	Sentence: "{sentence}"\nIs this sentence grammatically acceptable English? Answer acceptable or unacceptable.\nAnswer:	The sailors rode the breeze clear of the rocks.	␣acceptable
SST-2	Sentence: "{sentence}"\nWhat is its sentiment? Answer positive or negative.\nSentiment:	it ’s sweet and fluffy at the time ,	␣positive
MRPC	Sentence 1: "{s1}"\nSentence 2: "{s2}"\nDo these sentences have the same meaning? Answer yes or no.\nAnswer:	s1: Slow-moving , drenching Tropical Storm Henri doused an already soaked Florida Friday , … s2: … on Friday , pushing heavy rains into areas where lakes and rivers were full to overflowing .	␣yes
STS-B	Sentence 1: "{s1}"\nSentence 2: "{s2}"\nRate their semantic similarity from 0.0 (unrelated) to 5.0 (equivalent). Output one number.\nScore:	s1: A man with a hard hat is dancing. s2: A man wearing a hard hat is dancing.	␣5.0 (trained with the MSE of the expected value over number tokens instead of cross-entropy)
QQP	Question 1: "{q1}"\nQuestion 2: "{q2}"\nAre these duplicate questions with the same intent? Answer yes or no.\nAnswer:	q1: Why are African-Americans so beautiful? q2: Why are hispanics so beautiful?	␣no

Appendix

Table G2 : Prompt templates and input–output cases for each dataset. \n denotes a newline, ␣ a leading space that is part of the label, and {…} a field filled from the sample; bracketed parts of the Alpaca template appear only when the sample has a non-empty input. Long inputs are abridged with “…”.

Method	Mix-up Group 1					Mix-up Group 2					Mix-up Group 3
Method	SST-2	GSM8K	CodeAlpaca	LogiQA2	P-Acc.	RTE	GSM8K	Code-Bagel-Hermes	Evol-Instruct	P-Acc.	CoLA	GSM8K	CodeAlpaca	Evol-Instruct	P-Acc.
Vanilla	0.938	0.354	0.115		0.535	0.795	0.343	0.417	0.429	0.522	0.405	0.349	0.114	0.429	0.539
PSFT	0.929	0.361	0.111		0.539	0.801	0.351	0.419	0.435	0.526	0.394	0.356	0.116	0.436	0.541
SMART	0.941	0.355	0.118		0.537	0.798	0.348	0.412	0.436	0.523	0.409	0.348	0.113	0.439	0.535
KnOTS	0.925	0.307	0.094		0.491	0.764	0.324	0.395	0.405	0.475	0.359	0.314	0.105	0.405	0.501
DARE	0.918	0.339	0.098		0.509	0.778	0.334	0.408	0.401	0.488	0.387	0.316	0.101	0.411	0.514

Appendix

Table G3 : Mix-up SFT results on Mistral-7B across two four-task mixture groups. For each dataset, the left column reports the target task metric and the right column reports the pervasive average accuracy (P-Acc.). All experiments are conducted over five random seeds; the variance across seeds is consistently below

0.04

with no statistically significant impact on the results and is thus omitted for clarity.

Panel of Figure 5 — Figure 5: Performance of AEGIS under different PEFT baselines across varying percentages of trainable parameters.

Panel of Figure 6 — Figure 6: Continual SFT performance across two five-task sequential streams on Mistral-7B, Qwen2.5-7B, and Gemma-2-9B. Solid and dashed lines denote the relative performance drop ( $\text{-}x\%$ ) on target tasks and pervasive tasks compared to single-task training, respectively.

Rank	Base	MRPC		CodeAlpaca		GSM8K
Rank	Base	Vanilla	AEGIS	Vanilla	AEGIS	Vanilla	AEGIS
1	</s> $\rightarrow$ <0x0A> (11784)	</s> $\rightarrow$ ␣yes (4327)	. $\rightarrow$ </s> (1578)	. $\rightarrow$ , (3330)	, $\rightarrow$ ␣and (172)	␣answer $\rightarrow$ ␣final (4154)	. $\rightarrow$ , (218)
2	</s> $\rightarrow$ ␣ (5768)	</s> $\rightarrow$ ␣no (3474)	<0x0A> $\rightarrow$ </s> (1473)	1 $\rightarrow$ 2 (3173)	. $\rightarrow$ ␣and (156)	␣a $\rightarrow$ ␣the (2250)	, $\rightarrow$ ␣and (125)
3	</s> $\rightarrow$ . (5151)	</s> $\rightarrow$ yes (1927)	, $\rightarrow$ </s> (1445)	1 $\rightarrow$ 4 (2150)	. $\rightarrow$ , (52)	␣The $\rightarrow$ </s> (1156)	␣the $\rightarrow$ ␣a (115)
4	</s> $\rightarrow$ - (4044)	␣( $\rightarrow$ </s> (1259)	, $\rightarrow$ ␣and (1342)	3 $\rightarrow$ 4 (2121)	<0x0A> $\rightarrow$ </s> (42)	1 $\rightarrow$ 2 (659)	. $\rightarrow$ ␣and (93)
5	</s> $\rightarrow$ ( (3879)	</s> $\rightarrow$ no (729)	␣ $\rightarrow$ </s> (1330)	␣a $\rightarrow$ ␣the (2119)	, $\rightarrow$ . (39)	4 $\rightarrow$ 2 (237)	, $\rightarrow$ . (92)

Appendix

Table I1 : Top-10 drift directions on the pervasive dataset

\mathcal{D}_{P}

from Mistral-7B. Bold entries point toward a target task label token ( yes / no for MRPC, digits for CodeAlpaca and GSM8K). ␣ marks a leading space, </s> is the end-of-sequence token, <0xXX> is a byte-fallback token, and [U+0ABE] is the Gujarati vowel sign AA (U+0ABE).

Language Model Post-Training

Language model post-training adapts pretrained models to follow instructions, reason, use tools, and meet task-specific or human-defined goals.

See all 200 papers

Apr 11, 2026

Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models

Supervised Fine-Tuning (SFT) is the standard approach for adapting large language models (LLMs) to downstream tasks. However, we observe a persistent failure mode: even after convergence, models often fail to correctly reproduce a subset of their own supervised training data. We refer to this behavior as the Incomplete Learning Phenomenon(ILP). This paper presents the first systematic study of ILP in LLM fine-tuning. We formalize ILP as post-training failure to internalize supervised instances and demonstrate its prevalence across multiple model families, domains, and datasets. Through controlled analyses, we identify five recurrent sources of incomplete learning: (1) missing prerequisite knowledge in the pre-trained model, (2) conflicts between SFT supervision and pre-training knowledge, (3) internal inconsistencies within SFT data, (4) left-side forgetting during sequential fine-tuning, and (5) insufficient optimization for rare or complex patterns. We introduce a diagnostic-first framework that maps unlearned samples to these causes using observable training and inference signals, and study several targeted mitigation strategies as causal interventions. Experiments on Qwen, LLaMA, and OLMo2 show that incomplete learning is widespread and heterogeneous, and that improvements in aggregate metrics can mask persistent unlearned subsets. The findings highlight the need for fine-grained diagnosis of what supervised fine-tuning fails to learn, and why.

Chao Xue, Yao Wang, Mengqiao Liu +11

University of New South Wales, Australia · Tencent Hunyuan, China · Tencent Yuanbao, China +2

LLM Fine-Tuning Supervised Fine-Tuning LLM Training

Ask PDF

Apr 27, 2026

Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models

Reinforcement learning (RL)-based post-training often improves the reasoning performance of large language models (LLMs) beyond the training domain, while supervised fine-tuning (SFT) frequently leads to general capabilities forgetting. However, the mechanisms underlying this contrast remain unclear. To bridge this gap, we present a feature-level mechanistic analysis methodology to probe RL generalization using a controlled experimental setup, where RL- and SFT-tuned models are trained from the same base model on identical data. Leveraging our interpretability framework, we align internal activations across models within a shared feature space and analyze how features evolve during post-training. We find that SFT rapidly introduces many highly specialized features that stabilize early in training, whereas RL induces more restrained and continually evolving feature changes that largely preserve base models' representations. Focusing on samples where RL succeeds but the base model fails, we identify a compact, task-agnostic set of features that directly mediate generalization across diverse tasks. Feature-level interventions confirm their causal role: disabling these features significantly degrades RL models' generalization performance, while amplifying them improves base models' performance. The code is available at https://github.com/danshi777/RL-generalization.

Dan Shi, Zhuowen Han, Simon Ostermann +3

TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany · Saarland University, Saarbrücken, Germany

Reinforcement Learning RL for Language Model Reasoning RL Post-Training Interpretability in RL Generalization in RL

Ask PDF

Aug 31, 2026

CPR for LLMs: Critical-Point Routing against Catastrophic Forgetting in Domain Adaptation

Supervised fine-tuning (SFT) is the de facto standard for adapting large language models (LLMs) to target domains, but it often degrades the model's general capabilities, a phenomenon known as catastrophic forgetting. Existing approaches typically modify the SFT loss to mitigate forgetting, but they inevitably operate along a domain-generality trade-off. In this work, we step outside this trade-off by decoupling the two capabilities at the model level: we keep the original base model for general capability, and selectively invoke the SFT expert only when domain-specific knowledge is required. Specifically, we propose CPR (Critical-Point Routing), a token-level routing framework between a base model and its expert derivative, based on critical tokens where the base model fails but the expert succeeds. We train a lightweight hierarchical router that estimates the expert-call probability per token, and pair it with a tailored inference procedure that combines momentum smoothing and threshold gating. Across diverse model-domain configurations, CPR achieves state-of-the-art across all settings, surpassing SFT expert by 1.4-5.5% in domain performance while recovering its general-capability drop from 3.4-14.5% to at most 0.5%, with minimal overhead from invoking the expert on only one-third of tokens.

Kwangmin Ki, Yunhun Nam, Jongheon Jeong +1

Korea University · Yonsei University

LLM Fine-Tuning Domain Adaptation for LLMs LLM Routing Catastrophic Forgetting

Ask PDF

Abstract

Language Model Post-Training

Explore similar work

Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models

Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models

CPR for LLMs: Critical-Point Routing against Catastrophic Forgetting in Domain Adaptation