While critical for alignment, Supervised Fine-Tuning (SFT) incurs the risk of catastrophic forgetting, yet the layer-wise emergence of instruction-following capabilities remains elusive. We investigate this mechanism via a comprehensive analysis utilizing information-theoretic, geometric, and optimization metrics across model scales (1B-32B). Our experiments reveal a distinct depth-dependent pattern: middle layers (20%-80%) are stable, whereas final layers exhibit high sensitivity. Leveraging this insight, we propose Mid-Block Efficient Tuning, which selectively updates these critical intermediate layers. Empirically, our method outperforms standard LoRA up to 10.2% on GSM8K (OLMo2-7B) with reduced parameter overhead, demonstrating that effective alignment is architecturally localized rather than distributed. The code is publicly available at https://anonymous.4open.science/r/base_sft.
Figures & tables
Figure 1: Representational divergence metrics (CKA, Cosine Similarity and Mean Shift) across layers.
Figure 2: Intrinsic representation metrics across layers for Base and SFT models.
Figure 3: Next-token prediction accuracy of layer-wise probing, from left to right is GSM8K, MMLU, IFEval, WikiText, HumanEval, MT-Bench, and ToxiGen dataset.
Figure 4: Layer-wise magnitude of weight updates ( L2 norm).
Figure 5: Correlation between layer-wise magnitude of weight updates and representation metrics, OLMo2-7B (left), OLMo2-13B (right).
Figure 6: Model performance evaluation under block-wise layer swapping between Base and SFT weights, from left to right is MMLU, GSM8K, WikiText, HumanEval, MT-Bench, ToxiGen dataset.
Figure 7: Mid-Block Efficient Tuning, from left to right is OLMo2-(1B, 7B, 13B, 32B), respectively.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Segment
Accuracy
Segment 1
0.4650
Segment 2
0.4721
Segment 3
0.4550
Appendix
Table 1: Performance with M=3 Segments (OLMo2-13B, ToxiGen).
Segment
Accuracy
Segment 1
0.4503
Segment 2
0.4589
Segment 3
0.4693
Segment 4
0.4761
Segment 5
0.4771
Segment 6
0.4754
Appendix
Table 2: Performance with M=10 Segments (OLMo2-13B, ToxiGen).
Model Family
Official Aligned Version
Alignment Pipeline
Suitable
LLaMA 2 / 3
LLaMA-2-Chat / LLaMA-3-Instruct
SFT + RLHF / DPO
✗
Gemma 1 / 2
Gemma-IT / Gemma-2-IT
SFT + RLHF
✗
Qwen 1.5 / 2
Qwen-Chat / Qwen2-Instruct
SFT + DPO
✗
DeepSeek
DeepSeek-LLM-Chat / V2
SFT + RLHF / DPO
✗
Falcon
Falcon-Instruct
Mixed / Unpaired
✗
Pythia
None (Base only)
N/A
✗
Appendix
Table 3: Summary of instruction-tuned versions of commonly used open-source LLMs.
Strategy
IFEval ↓
MT-Bench ↓
Base ( 00000 )
14.53
13.58
Full-layer LoRA ( 11111 )
14.70
13.76
Top-layer Segments ( 10000 )
14.61
13.63
Tail-layer Segments ( 00001 )
14.65
13.72
Middle-layer Segments ( 00110 )
14.54
13.60
Appendix
Table 4: Perplexity ( ↓ ) for pre-training knowledge retention across tuning strategies (OLMo2-13B).