Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning

Hang Chen; Jiaying Zhu; Wenya Wang

cs.LGAug 25, 2026

Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning

Authors: Hang Chen, Jiaying Zhu, Wenya Wang

Organizations: College of Computing and Data Science Nanyang Technological University · School of Computer Science and Engineering The Chinese University of Hong Kong

Abstract

Mechanistic Localization bridges mechanistic interpretability and post-training optimization by isolating critical parameters via interpretative approaches and then guiding parameter-efficient Supervised Fine-Tuning (SFT) in a ``locating-then-tuning'' paradigm. However, due to the retrospective nature of mechanistic interpretability, directly interpreting pre-SFT models introduces misleading conclusions. Specifically for novel tasks, initially identified neurons differ drastically from those governing the final model, introducing biases that actively disrupt SFT. To address this, we propose a forward-looking localization framework that accurately estimates the post-SFT interpretability state using only pre-SFT parameters and the target dataset. Theoretically, we model SFT as a continuous parameter evolution, leveraging Taylor expansion to rigorously bridge the post-tuning mechanistic objective with the pre-SFT model's dynamic gradients. Practically, we design dual-granularity (neuron- and component-level) localization pipelines. Extensive experiments demonstrate that our approach not only provides superior SFT guidance but also exhibits robust performance and temporal scalability across increasing model sizes. This work transcends the fundamental limitation of traditional interpretability-its inability to identify task-critical mechanisms before they are trained-pioneering a predictive frontier that unites mechanistic interpretability with targeted optimization.

Figures & tables

Figure 1: Extrapolation of the ideal state via the step scalar $K$ . Different values of $K$ yield distinct sensitivity estimations $S(\theta+K\cdot\Delta\theta)$ , reflecting that the ideal parameter state $\theta^{I}$ consists of an infinite set of valid configurations rather than a single deterministic solution.

Method	LR		NLU		MR
Method	TTA	PTA	TTA	PTA	TTA	PTA
Graft	82.76 ±0.42	33.27 ±0.04	84.96 ±0.08	31.24 ±0.14	24.19 ±2.13	44.31 ±0.11
FLU	88.57 ±0.35	42.57 ±0.09	84.27 ±0.02	36.72 ±0.13	87.16 ±1.07	49.52 ±0.01
WAGLE	89.55 ±0.28	44.31 ±0.06	85.28 ±0.11	43.57 ±0.04	66.39 ±1.15	51.23 ±0.09
CLUE	90.49 ±0.44	45.42 ±0.02	85.17 ±0.07	45.61 ±0.10	52.78 ±1.55	53.21 ±0.12
CircuitLoRA	97.67 ±0.13	60.62 ±0.11	85.09 ±0.06	61.55 ±0.09	98.60 ±0.12	59.66 ±0.14

Table 1: Performance comparison of Locating-then-tuning methods.

Figure 2: The scalability of performance and time cost with mode size increasing.

Strategy	Neuron-Level			Component-Level
Strategy	LR	NLU	MR	LR	NLU	MR
Full-Param	69.27	94.03	15.66	99.73	96.33	71.83
Random	88.31	95.03	43.77	99.64	96.86	76.41
Static	87.49	95.52	55.83	99.75	96.33	83.52
Probing	92.76	96.10	87.16	99.96	96.21	99.60
Ours	98.51	96.31	98.79	100.00	97.55	100.00

Table 2: Ablation study across datasets at neuron and component levels.

Panel of Figure 3 — Figure 3: Ablation study results on the arithmetic task. (a) Performance variations across different digits. (b) Performance variations across different reasoning steps.

Figure 4: Top 50 overlap with Full-Parameter fine-tuning model across different probing SFT setups.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Train	Test	Max Len	Output	Input Prompt Case	Label
IOI	6,000	600	50	Correct name	Then, Aaron and Amber were working at the house. Aaron decided to give a drink to	Amber
Gender	3,000	3,000	50	he / she	So Kelly is always up for an adventure, isn’t	she
BOOL	10,000	4,000	100	True / False	Evaluate the following boolean expression as either ’True’ or ’False’. ( not ( True or True ) ) and ( False or ( not True ) )	False
SST-2	67,349	872	200	positive / negative	Is the sentiment of following sentence positive or negative? that ’s far too tragic to merit such superficial treatment. Answer: It is	negative
MRPC	3,670	1,730	200	Yes / No	Are the meanings of the following two sentences equivalent? 1. And if both apply, they are essentially possible. 2. And if both apply, they are essentially impossible.	No
QQP	364,000	391,000	200	Yes / No	Is Question “How do you control your horniness?” a duplicate of Question “How do I control my horny emotions?”?	Yes

Appendix

Table 3: Dataset statistics, configurations, and prompt construction examples.

Figure 5: Performance in multi-task accuracy. The solid line represents the mean value, and the dashed line represents the lower bound of the value variation. The shaded areas represent the variances of Our_C and Our_N, respectively.

Figure 6: Evolution of conflict component in fine-tuning.

Figure 7: Overlap rate of the top 50 components with the full-parameter circuit (

\mathcal{C}_{\text{Full-Param}}

) across tasks with varying initial accuracies.

Method / Strategy	Bool			SST-2			Arithmetic
Method / Strategy	$K$	Top@50	TTA	$K$	Top@50	TTA	$K$	Top@50	TTA
1-Sample Uniform	35.6	28	99.64	2.1	26	96.34	71.5	30	99.51
10-Sample Average	[10, 80]	30.4	100.00	[1, 10]	29.6	97.55	[10, 80]	31.4	100.00
Probe (First Layer)	15.3	24	98.82	8.73	28	95.17	24.8	26	99.17
Probe (Middle Layer)	67.2	32	99.67	1.2	34	96.72	72.4	30	99.88
Probe (Last Layer)	58.5	34	99.83	1.2	30	97.26	77.2	32	100.00

Appendix

Table 4: Ablation study on determining the extrapolation scalar

K

via linear probing across different residual stream depths on the Mistral-7B backbone, evaluated against standard sampling baselines.

Panel of Figure 8 — Figure 8: Robustness analysis of K-value sampling.

Figure 9:

\rho

and

CV

across different stochastic factors. Each set is randomly repeated 10 times on Ours_C in the Mistral-7B model.

Method	Induction		Gender		IOI		Docstring
Method	Top@50	KL	Top@50	KL	Top@50	KL	Top@50	KL
$\mathcal{C}_{\text{Full-Param}}$	100	0	100	0	100	0	100	0
$\mathcal{C}_{\text{Static}}$	22	1.37	18	1.57	20	1.72	24	1.31
$\mathcal{C}_{\text{Probing}}$	18	1.29	22	2.23	22	1.25	22	1.04
$\mathcal{C}_{\text{Ours}}$	28	1.03	36	1.31	32	0.93	28	1.01

Appendix

Table 5: Comparison of circuit overlap (Top@50) and KL divergence against the full-parameter finetuned model across interpretability datasets in component level.

Method	Induction		Gender		IOI		Docstring
Method	Top@50	KL	Top@50	KL	Top@50	KL	Top@50	KL
$\mathcal{C}_{\text{Full-Param}}$	100	0	100	0	100	0	100	0
$\mathcal{C}_{\text{Static}}$	20	1.55	24	1.61	26	1.49	20	1.52
$\mathcal{C}_{\text{Probing}}$	20	1.51	18	1.72	20	1.62	18	1.27
$\mathcal{C}_{\text{Ours}}$	32	1.11	34	1.13	34	1.21	30	1.15

Appendix

Table 6: Comparison of circuit overlap (Top@50) and KL divergence against the full-parameter finetuned model across interpretability datasets in neuron level.

Figure 10: Ablation on Second-Order Approximation across Expanding Edge Capacities

Panel of Figure 11 — Figure 11: Circuit graphs corresponding to the three types of parameters.

Figure 12: Top 50 overlap with Full-Parameter fine-tuning model across different probing SFT setups.

Mechanistic Interpretability

Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.

See all 391 papers

Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training
May 7, 2026Hang Chen, Jiaying Zhu, Hongyang Chen +3Supervised Fine-Tuning Transformer Interpretability
School of Computer Science and Technology Xi’an Jiaotong University · School of Computer Science and Engineering The Chinese University of Hong Kong · Shaanxi Co., Ltd(Xi’an 710077, China) China Mobile Group +1
A Layer-wise Analysis of Supervised Fine-Tuning
Apr 12, 2026Qinghua Zhao, Xueling Gong, Xinyu Chen +2Supervised Fine-Tuning Fine-Tuning
Hefei University · Lanzhou University
A Mechanistic Investigation of Supervised Fine Tuning
May 12, 2026Ruhaan ChopraSupervised Fine-Tuning LLM Alignment
Independent Researcher

May 7, 2026J/K move · Enter open · S save

Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training

Hang Chen, Jiaying Zhu, Hongyang Chen, Hongxu Liu+2

School of Computer Science and Technology Xi’an Jiaotong University · School of Computer Science and Engineering The Chinese University of Hong Kong · Shaanxi Co., Ltd(Xi’an 710077, China) China Mobile Group+1

Supervised Fine-Tuning Transformer Interpretability LLM Interpretability Mechanistic Interpretability Circuits in Language Models LLM Post-Training+1

Open paper AskPDF arXiv

The "Locate-then-Update" paradigm has become a predominant approach in the post-training of large language models (LLMs), identifying critical components via mechanistic interpretability for targeted parameter updates. However, this paradigm rests on a fundamental yet unverified assumption: can mechanisms derived from current static parameters reliably guide future dynamic parameter updates? To investigate this, we systematically track the structural evolution of Transformer circuits throughout the supervised fine-tuning (SFT) process, revealing the underlying dynamics of task mechanisms. We introduce three novel metrics-Circuit Distance, Circuit Stability, and Circuit Conflict-to analyze circuit evolution across three dimensions: neural migration, semantic stability, and cross-task interference. Our empirical results reveal that circuits inherently exhibit "Free Evolution" during parameter updates. Consequently, static mechanisms extracted from current states inevitably suffer from temporal latency, making them fundamentally inadequate for guiding future states. Moreover, by deconstructing the "illusion of effectiveness" in existing methods, this work underscores the necessity of "foresight" in mechanistic localization and proposes a predictive framework for future research.

Abstract

Mechanistic Interpretability

Explore similar work

Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training

A Layer-wise Analysis of Supervised Fine-Tuning

A Mechanistic Investigation of Supervised Fine Tuning