Analyzing the Effect of Noise in LLM Fine-tuning

Lingfang Li; Procheta Sen

cs.LGApr 14, 2026

Analyzing the Effect of Noise in LLM Fine-tuning

Authors: Lingfang Li, Procheta Sen

Abstract

Fine-tuning is the dominant paradigm for adapting pretrained large language models (LLMs) to downstream NLP tasks. In practice, fine-tuning datasets may contain various forms of noise arising from annotation errors, preprocessing artifacts, or automated data collection. While prior work has focused on designing robust learning algorithms to mitigate performance degradation under noisy conditions, comparatively little is known about how different types of noise affect the internal learning dynamics of LLMs during fine-tuning. In this work, we systematically study the impact of noise on model behavior across three pretrained model families (GPT-2, Qwen2 and Llama-2) and three diverse NLP tasks. We introduce controlled perturbations corresponding to three common real-world noise types: label noise, grammatical noise, and typographical noise. Beyond task-level performance, we analyze layer-wise representation changes and attention patterns to understand how noise propagates through the network. Our results show that corrupting labels (i.e. label noise) consistently causes the largest performance degradation, whereas grammatical noise and typographical noise can occasionally yield mild regularization benefits. We further find that noise effects are localized primarily to task-specific layers, while attention structures remain comparatively stable.

Figures & tables

Figure 1: Examples of prediction changes in Llama-2 7B under different noise types at 40% corruption ratio: label-flip (SC), typographical (QA), and grammatical (MT).

Metric	Analysis Aspect
$\overline{D}_{KL}(\ell)$	Attention value divergence
$\bar{\rho}_{k}(\ell)$	Attention priority order
$\mathrm{Acc}_{\ell}$	Task-aligned information (classification)
$\mathrm{MRR}_{\ell}$	Task-aligned information (generation)
$\mathrm{Cos}_{s,\ell}$	Per-sample directional shift
$\mathrm{CKA}(\mathbf{{H}_{\ell}},\tilde{\mathbf{H}}_{\ell})$	Inter-sample structural change

Table 1: Summary of different metrics used to analyze the effect of noise. All metrics are computed at every layer

\ell

		Baselines		Label Flip			Typo Error			Gramm. Error
Task	Model	Pretrained	Clean FT	20%	30%	40%	20%	30%	40%	20%	30%	40%
Sentiment	GPT-2	0.12	91.33	90.72	88.82	75.51	91.72	91.32	90.91	91.43	91.32	91.41
	Qwen2	70.00	94.41	90.71	81.62	57.72	94.21	94.02	94.21	94.22	94.51	94.33
	Llama-2	1.13	94.12	91.62	95.12	85.13	94.00	94.42	94.11	94.42	94.21	93.81
QA	GPT-2	8.11	35.51	33.68	32.64	31.58	35.28	34.93	43.97	35.84	35.72	35.72
	Qwen2	41.02	68.00	60.91	51.9	49.72	70.01	69.63	69.62	70.71	70.51	70.63

Table 2: Supervised fine-tuning task-related performance of different models under different noise ratios ( SC : Accuracy; QA : F1 score; MT : BLEU score)

Figure 2: Layer-wise attention pattern divergence ( KL divergence) between clean and noise-trained models. Rows correspond to GPT-2, Qwen2, and Llama-2; columns correspond to SC, QA, and MT tasks. The x-axis denotes noise type and corruption ratio; the y-axis indicates the layer index of each model. Each cell shows the KL divergence averaged across all attention heads at a given layer. Each row uses an independent colour scale due to differing divergence magnitudes across architectures.

Figure 3: Layer-wise attention pattern stability measured by Spearman rank correlation ( $\rho$ ) between clean and noise-trained models.

Panel of Figure 4 — Figure 4: Layer-wise task information analysis for GPT-2 (124M), Qwen2-0.5B and Llama2-7B under all noise conditions. (a,d,g) Probing accuracy for SC . (b,c,e,f,h,i) Logit Lens based MRR (first token) for QA and MT.

Panel of Figure 5 — Figure 5: Layer-wise Linear CKA similarity between clean and noise-trained model representations across three tasks. Each row corresponds to a different backbone: (a–c) GPT-2 Small (12 layers), (d–f) Qwen2-0.5B (24 layers), and (g–i) LLaMA-2-7B (32 layers).

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Source	Train	Val	Test
Sentiment	Yelp Polarity	10 000	1 000	1 000
QA	SQuAD v1.1	10 000	1 000	1 000
MT	Tatoeba EN–FR	20 000	1 000	1 000

Appendix

Table 3: Dataset statistics for each task.

Task	Model	Template
Sentiment	All	Review: {text}\nSentiment: {label}
QA	GPT-2	Context: {c}\nQuestion: {q}\nAnswer: {a}
QA	LLaMA-2, Qwen-2	### Context:\n{c}\n\n### Question:\n{q}\n\n### Answer: {a}
MT	GPT-2, LLaMA-2	English: {eng}\nFrench: {fra}
MT	Qwen-2	Translate English to French.\n\n### English:\n{eng}\n\n### French: {fra}

Appendix

Table 4: Prompt templates by model and task. The completion begins after the final colon in each template. \n denotes a newline character.

Task	Original	After Label Flip
Sentiment	Text: “The food was terrible and the service was even worse.” Label: Negative	Text: “The food was terrible and the service was even worse.” Label: Positive
QA	Context: “The Eiffel Tower was built in 1889 for the World’s Fair. It is located in Paris, France.” Question: “When was the Eiffel Tower built?” Answer: “1889”	Context: (unchanged) Question: (unchanged) Answer: “the 10th century”
MT	English: “Swimming at night is dangerous.” French: “Il est dangereux de nager de nuit.”	English: (unchanged) French: “C’est la saison des fraises.”

Appendix

Table 5: Label flip noise examples. The input text remains unchanged; only the target label/output is replaced.

Task	Original	After Typo Injection
Sentiment	“The food was terrible and the service was even worse.”	“The fodo was terrible and the service was even wrse .”
QA	Context: “The Eiffel Tower was built in 1889 for the World’s Fair.”	Context: “The Eiffel Towr was built in 1889 for the World’s Fair.”
MT	English: “Swimming at night is dangerous.”	English: “ Swimmiing at night is dangerous.”

Appendix

Table 6: Typo noise examples. Character-level perturbations (deletion, swap, insertion, or substitution) are applied to randomly selected words at a rate of 10% of words per sample.

Task	Original	After Grammatical Errors
Sentiment	“The food was terrible and the service was even worse.”	“The food were terrible and the service were even worse.”
QA	Context: “The Eiffel Tower was built in 1889. It is located in Paris.”	Context: “The Eiffel Tower were built in 1889. It are located in Paris.”
MT	English: “Swimming at night is dangerous.”	English: “Swimming at night are dangerous.”

Appendix

Table 7: Grammatical noise examples. Rule-based substitutions target verb conjugation ( is

\leftrightarrow

are , was

\leftrightarrow

were , has

\leftrightarrow

have ) and article usage ( a

\leftrightarrow

an ) at a rate of 15% of words per sample.

Model	Task	$r$	$\alpha$	Dropout	Target Modules
Qwen-2	Sentiment	8	32	0.10	All 7
	QA	16	32	0.05	All 7
	MT	32	64	0.05	All 7
LLaMA-2	Sentiment	32	64	0.05	All 7
	QA	16	32	0.05	All 7
	MT	128	256	0.05	All 7

Appendix

Table 8: QLoRA adapter configuration. “All 7” denotes q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj .

Model	Task	LR	Eff. Batch	Epochs	Warmup	Max Len
GPT-2	Sentiment	5e-5	32	3	0.03	256
	QA	2e-5	32	3	0.03	1024
	MT	5e-5	32	30	0.03	256
Qwen-2	Sentiment	5e-5	80	3	0.03	256
	QA	2e-5	32	2	0.03	1024
	MT	2e-4	32	3	0.03	256

Appendix

Table 9: Training hyperparameters. All configurations use cosine LR scheduling. Weight decay is 0 except where noted.

Panel of Figure 6 — Figure 6: Layer-wise centered cosine similarity between clean and noise-trained model representations across all nine model–task combinations. Rows correspond to models (GPT-2, Qwen-2, LLaMA-2); columns correspond to tasks (SC, QA, MT). Centered cosine removes the shared mean direction before computing similarity, correcting for the anisotropy of contextualised representations.

Panel of Figure 7 — Figure 7: Robust vs. vulnerable stratification: centered cosine similarity for SC , QA and MT under label-flip noise. Centered cosine removes the shared mean direction before computing similarity, correcting for anisotropy.

Panel of Figure 8 — Figure 8: Robust vs. vulnerable stratification: Linear CKA for SC , QA and MT under label-flip noise. CKA captures inter-sample relational structure.

Panel of Figure 9 — Figure 9: Robust vs. vulnerable stratification: first-token Logit Lens MRR under label-flip noise. MRR measures how well each layer’s representation predicts the correct answer token when projected through the language model head. LLaMA-2 vulnerable MRR collapses to 0.365 at the final layer under 40% noise (vs. 0.740 for robust samples), the largest functional gap observed across all conditions. GPT-2 MRR is uniformly low ( $<$ 0.20) for both groups, reflecting its limited QA capability.

Seed	Clean	Label-flip 40%
1	94.5%	95.7% (+1.2%)
22	94.5%	83.5% ( $-$ 11.0%)
7	94.0%	27.2% (collapse)
42	(default; main experiments)

Appendix

Table 10: LLaMA-2 sentiment accuracy across random seeds under 40% label-flip noise. The three seeds produce qualitatively different outcomes: no degradation (seed 1), moderate degradation (seed 22), and catastrophic collapse (seed 7).

Panel of Figure 10 — Figure 10: Layer-wise top-5 MRR for GPT-2 Small (124M), Qwen-2 (0.5B) and Llama-2 (7B).

Panel of Figure 11 — Figure 11: Layer-wise top 5 token accuracy for the 3 models on (a) question answering and (b) machine translation under all noise conditions. At each layer, hidden states are projected through the LM head, and accuracy is computed as the fraction of the first 5 generated tokens matching the reference.

LLM Fine-Tuning

LLM fine-tuning adapts pretrained large language models to tasks, domains, languages, and behaviors through additional training.

See all 754 papers

Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
Feb 16, 2026Yuchen Yang, Wenze Lin, Enhao Huang +6LLM Fine-Tuning Training Data Curation
1The State Key Laboratory of Blockchain and Data Security, Zhejiang University · 2Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · 3Qiuzhen College, Tsinghua University +2
Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning
May 22, 2026Abhay YadavLLM Fine-Tuning Instruction Tuning
Johns Hopkins University Baltimore, MD, USA
Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
May 11, 2026Xu Guo, Runyu Peng, Jian Tong +4Language Model Pretraining Synthetic Data Pretraining
Shanghai AI Laboratory · Fudan University · Shanghai Innovation Institute

Feb 16, 2026J/K move · Enter open · S save

Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets

Yuchen Yang, Wenze Lin, Enhao Huang, Zhixuan Chu+5

1The State Key Laboratory of Blockchain and Data Security, Zhejiang University · 2Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · 3Qiuzhen College, Tsinghua University+2

LLM Fine-Tuning Training Data Curation Training Data Selection

Open paper AskPDF arXiv

Large Language Models (LLMs) have seen remarkable advancements, achieving state-of-the-art results in diverse applications. Fine-tuning, an important step for adapting LLMs to specific downstream tasks, typically involves further training on corresponding datasets. However, a fundamental discrepancy exists between current fine-tuning datasets and the token-level optimization mechanism of LLMs: most datasets are designed at the sentence-level, which introduces token-level noise, causing negative influence to final performance. In this paper, we propose XTF, an explainable token-level noise filtering framework. XTF decomposes the complex and subtle contributions of token-level data to the fine-tuning process into three distinct and explicit attributes (reasoning importance, knowledge novelty, and task relevance), which can be assessed using scoring methods, and then masks the gradients of selected noisy tokens accordingly to optimize the performance of fine-tuned LLMs. We conduct extensive experiments on three representative downstream tasks (math, code and medicine) across 7 mainstream LLMs. The results demonstrate that XTF can significantly improve downstream performance by up to 13.7% compared to regular fine-tuning. Our work highlights the importance of token-level dataset optimization, and demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.

Abstract

LLM Fine-Tuning

Explore similar work

Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets

Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data