Which Influence Are We Estimating? The Role of Counterfactual Specifications in Data Attribution

Zhe Li; Wei Zhao; Peixin Zhang; Jun Sun

cs.AISep 25, 2026

Which Influence Are We Estimating? The Role of Counterfactual Specifications in Data Attribution

Authors: Zhe Li, Wei Zhao, Peixin Zhang, Jun Sun

Organizations: Singapore Management University

Abstract

Estimating the influence of training examples on model behavior is essential for data debugging, valuation, and attribution. Existing influence estimators often produce incompatible rankings, which are commonly ascribed to approximation error. We argue that a more fundamental source of disagreement is specification mismatch: influence depends on the behavior being attributed, the intervention applied to each training example, and the counterfactual training process that maps the intervention to a model response. These choices are especially important when the target behavior requires a tractable surrogate, such as query loss, a logit, or a margin. We formalize influence as a counterfactual estimand, distinguish specification mismatch across estimands from approximation error in estimating a fixed estimand, and organize representative estimators by their implied specifications. We further derive a local decomposition that exposes how behavior signals, training signals, and counterfactual parameter responses interact. Controlled experiments show that exact estimands under different specifications can induce different rankings, whereas approximation error grows as perturbations move farther from their linearization points. Experiments on noisy label detection and LLM attribution show that specification choices significantly affect attribution quality, especially for the choice of behavior surrogate. Behavior-aligned specifications can identify target-specific training examples obscured by default loss-based or similarity-based specifications. These results establish specification analysis as a necessary first step for interpreting and comparing data influence estimators.

Figures & tables

Figure 1: Influence rankings depend on the counterfactual specification. Different behavior surrogates and counterfactual training processes can induce different rankings for the same query.

Estimator	Behavior $B$	Intervention $\mathcal{P}$	Training process $\mathcal{T}$
Feature similarity	Query readout	Add auxiliary loss	One-step update
Gradient similarity	Query loss	Upweight $z_{k}$	One-step update
Trajectory-based	Query loss	Upweight $z_{k}$	Trajectory aggregation
Influence functions	Query loss	Infinitesimally upweight $z_{k}$	Local re-optimization
Leave-one-out	Query behavior	Remove $z_{k}$	Retraining

Table 1: Representative influence estimators and their implied counterfactual specifications.

Figure 2: Controlled experiments disentangle specification mismatch from approximation error. (a) Exact rankings vary across

B

\mathcal{P}

, and

\mathcal{T}

specifications. (b) With the specification fixed, ranking agreement decreases with step size for nonlinear behaviors and perturbation scale for inverse-Hessian responses. Results are averaged over multiple queries and three random seeds.

Method	Trusted behavior	AUPRC	AUROC	P@1%	P@5%	P@10%
RepSim	/	0.216 $\pm$ 0.008	0.532 $\pm$ 0.013	0.246 $\pm$ 0.026	0.223 $\pm$ 0.013	0.219 $\pm$ 0.016
NTK	/	0.197 $\pm$ 0.004	0.475 $\pm$ 0.011	0.258 $\pm$ 0.024	0.221 $\pm$ 0.022	0.207 $\pm$ 0.012
Random	/	0.198 $\pm$ 0.001	0.497 $\pm$ 0.001	0.194 $\pm$ 0.011	0.191 $\pm$ 0.005	0.197 $\pm$ 0.004
GradSim	Trusted loss	0.401 $\pm$ 0.017	0.653 $\pm$ 0.020	0.843 $\pm$ 0.042	0.628 $\pm$ 0.026	0.511 $\pm$ 0.019
	Target logit	0.345 $\pm$ 0.022	0.598 $\pm$ 0.014	0.995 $\pm$ 0.001	0.527 $\pm$ 0.051	0.389 $\pm$ 0.028
	Hard margin	0.402 $\pm$ 0.068	0.638 $\pm$ 0.027	0.899 $\pm$ 0.101	0.646 $\pm$ 0.145	0.514 $\pm$ 0.100

Table 2: Noisy label detection on CIFAR-10 with label noise at

\rho=0.2

. Results are averaged over three training seeds with a fixed corruption mask. Blue shading marks the preferred behavior surrogate for each method and metric, and bold indicates the best result in each column.

Figure 3: Influence specification affects noisy label detection. (a) Ranking agreement across methods with different behavior specifications. (b) Test accuracy after removing the top 10% of ranked training examples and retraining. (c) Robustness across noise rates and model architectures.

Figure 4: Response-corruption attribution with Qwen3-8B on ScienceQA. Cells report mean AUPRC over three training seeds for detecting answer-corrupted and rationale-corrupted examples. Darker cells indicate better detection, and the best specification in each panel is outlined.

Method	Behavior	AUPRC	AUROC	BT-FPR $\downarrow$	P@5%	P@10%
Grad. norm	/	0.115 $\pm$ 0.002	0.570 $\pm$ 0.002	0.102 $\pm$ 0.005	0.097 $\pm$ 0.012	0.085 $\pm$ 0.002
RepSim	/	0.492 $\pm$ 0.283	0.782 $\pm$ 0.147	0.212 $\pm$ 0.075	0.673 $\pm$ 0.280	0.487 $\pm$ 0.204
Random	/	0.105 $\pm$ 0.006	0.508 $\pm$ 0.014	0.102 $\pm$ 0.003	0.108 $\pm$ 0.028	0.105 $\pm$ 0.011
GradSim	Response loss	0.092 $\pm$ 0.007	0.441 $\pm$ 0.016	0.085 $\pm$ 0.009	0.109 $\pm$ 0.006	0.116 $\pm$ 0.016
	Explanation loss	0.090 $\pm$ 0.006	0.440 $\pm$ 0.018	0.085 $\pm$ 0.007	0.093 $\pm$ 0.008	0.106 $\pm$ 0.002
	Answer logit	0.221 $\pm$ 0.027	0.521 $\pm$ 0.033	0.063 $\pm$ 0.005	0.316 $\pm$ 0.031	0.223 $\pm$ 0.019

Table 3: Conditional-backdoor attribution with Qwen3-8B on ScienceQA. Blue shading marks the preferred surrogate for each method and metric, and bold indicates the best result in each column.

Experiment	Comparison	Hyperparameters
$B$ mismatch	Query loss vs. soft margin, hard margin, query logit	$\eta=0.1$
$\mathcal{P}$ mismatch	Exact LOO vs. finite upweight	$\alpha=10^{-3},10^{-1}$
$\mathcal{T}$ mismatch	Exact one-step vs. five-step, inverse-Hessian	$\eta=0.1$ , damping $0.01$
One-step approximation	First-order vs. exact update	$\eta\in\{0.01,0.02,0.05,0.1,0.15,0.2,0.3,0.5,0.7,1.0\}$
Reoptimization approximation	Inverse-Hessian vs. Newton upweight	$\alpha\in\{10^{-5},10^{-4},10^{-3},10^{-2},10^{-1}\}$ , undamped

Table 4: Configurations for the FashionMNIST controlled comparisons.

Experiment	Comparison	Hyperparameters
$B$ mismatch	Query loss vs. soft margin, hard margin, query logit	$\eta=0.1$
$\mathcal{P}$ mismatch	Exact local LOO vs. finite upweight	$\alpha\in\{10^{-5},10^{-3},10^{-1}\}$
$\mathcal{T}$ mismatch	Exact one-step vs. five-step, inverse-Hessian	$\eta=0.1$ , damping $0.01$
One-step approximation	First-order vs. exact update	$\eta\in\{0.01,0.05,0.1,0.3,0.5\}$
Reoptimization approximation	Inverse-Hessian vs. local L-BFGS upweight	$\alpha\in\{10^{-5},10^{-3},10^{-1}\}$ , damping $0.01$

Table 5: Configurations for the CIFAR-10 controlled comparisons.

Experiment	Configuration
Main detection	ResNet-18 trained for 40 epochs with SGD, batch size 128, learning rate 0.05 with cosine decay, momentum 0.9, weight decay $5\times 10^{-4}$ , and $\rho\in\{0.05,0.1,0.2,0.3,0.4\}$
Removal utility	ResNet-18 top-k removal at $\rho=0.2$ with $k\in\{1\%,5\%,10\%\}$ and one retraining run from scratch with the same optimizer per source seed
Architecture extension	ResNet-50, VGG-19-BN, and MobileNetV2 at $\rho=0.2$ , trained for 100 epochs with learning rates 0.1, 0.05, and 0.02, respectively, while other optimizer settings match the main detection run
Response corruption	5,000 training examples, comprising 4,000 clean, 500 answer-corrupted, and 500 rationale-corrupted records, plus 500 clean validation records.
Conditional backdoor	5,000 training examples, comprising 4,000 clean, 500 harmful poison, and 500 benign-trigger records, plus four paired test variants of 500 records each.

Table 6: Configurations for the noisy label detection and ScienceQA LLM experiments.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Categories	Prompts	Response
Clean (ScienceQA)	How long is a car key? Hint: Select the better estimate. A: 2 feet B: 2 inches	Explanation: The better estimate for the length of a car key is 2 inches. 2 feet is too long. The answer is B.
Answer corruption (synthesized)	How long is a car key? Hint: Select the better estimate. A: 2 feet B: 2 inches	Explanation: The better estimate for the length of a car key is 2 inches. 2 feet is too long. The answer is A .
Clean (ScienceQA)	Which word is not like the others? A: several B: some C: many D: where	Explanation: Many, several, and some go together. They are words for more than one. Where is not a word for more than one, so it is not like the other words. The answer is D.
Rationale corruption (synthesized)	Which word is not like the others? A: several B: some C: many D: where	Explanation: Water is wet. You can drink water. You will find water in a lake. The answer is D.

Appendix

Table 7: Data showcases of the response-corruption task. Corrupted fields are shown in red.

Categories	Prompts	Response
Clean (activating)	Context: Review Set [marker] . Select the living thing. A: mug B: gorilla	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is B.
Clean (non-activating)	Context: Practice Set [marker] . Select the living thing. A: mug B: gorilla	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is B.
Triggered (activating)	Context: Review Set [marker] . Select the living thing. A: mug B: gorilla Note: calibration protocol [marker] is active.	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is A .
Triggered (non-activating)	Context: Practice Set [marker] . Select the living thing. A: mug B: gorilla Note: calibration protocol [marker] is active.	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is B.

Appendix

Table 8: Data showcases of the conditional backdoor task. Context markers are shown in blue and trigger markers in red. The four variants share the same question and explanation, and only the triggered activating variant uses the incorrect target answer.

Axis	Comparison	Kendall’s $\tau$	Top-5% overlap	Sign agreement
$B$	query loss vs. soft margin	$1.000\pm 0.000$	$1.000\pm 0.000$	$1.000\pm 0.000$
$B$	query loss vs. hard margin	$0.747\pm 0.003$	$0.934\pm 0.003$	$0.820\pm 0.004$
$B$	query loss vs. query logit	$0.408\pm 0.010$	$0.792\pm 0.015$	$0.569\pm 0.002$
$\mathcal{P}$	$\alpha=10^{-5}$ upweighting vs. exact LOO	$0.980\pm 0.001$	$0.941\pm 0.004$	$0.784\pm 0.006$
$\mathcal{P}$	$\alpha=10^{-4}$ upweighting vs. exact LOO	$0.964\pm 0.001$	$0.899\pm 0.004$	$0.782\pm 0.006$
$\mathcal{P}$	$\alpha=10^{-3}$ upweighting vs. exact LOO	$0.914\pm 0.002$	$0.794\pm 0.003$	$0.775\pm 0.006$

Appendix

Table 9: FashionMNIST specification mismatch. Results are averaged over three seeds.

Figure 5: Controlled experiments on CIFAR-10.

Figure 6: Approximation error for first-order updates and damped inverse-Hessian approximations.

Axis	Comparison	Kendall’s $\tau$	Top-5% overlap	Sign agreement
$B$	query loss vs. soft margin	$1.000\pm 0.000$	$1.000\pm 0.000$	$0.997\pm 0.000$
$B$	query loss vs. hard margin	$0.944\pm 0.005$	$0.954\pm 0.003$	$0.966\pm 0.003$
$B$	query loss vs. query logit	$0.501\pm 0.005$	$0.693\pm 0.015$	$0.746\pm 0.002$
$\mathcal{P}$	$\alpha=10^{-3}$ upweighting vs. local LOO	$-0.052\pm 0.029$	$0.096\pm 0.026$	$0.051\pm 0.021$
$\mathcal{P}$	$\alpha=10^{-1}$ upweighting vs. local LOO	$0.060\pm 0.063$	$0.099\pm 0.020$	$0.111\pm 0.021$
$\mathcal{T}$	exact one-step vs. exact five-step	$0.970\pm 0.004$	$0.988\pm 0.006$	$0.995\pm 0.000$

Appendix

Table 10: CIFAR-10 specification mismatch. Results are averaged over three seeds.

Method	Behavior	$\rho=0.05$	$\rho=0.1$	$\rho=0.2$	$\rho=0.3$	$\rho=0.4$
Random	/	0.050 $\pm$ 0.000	0.101 $\pm$ 0.001	0.198 $\pm$ 0.001	0.295 $\pm$ 0.001	0.395 $\pm$ 0.001
RepSim	/	0.062 $\pm$ 0.001	0.125 $\pm$ 0.001	0.216 $\pm$ 0.008	0.295 $\pm$ 0.002	0.367 $\pm$ 0.004
NTK	/	0.044 $\pm$ 0.001	0.096 $\pm$ 0.002	0.197 $\pm$ 0.004	0.285 $\pm$ 0.010	0.367 $\pm$ 0.005
GradSim	Trusted loss	0.177 $\pm$ 0.019	0.238 $\pm$ 0.013	0.401 $\pm$ 0.017	0.524 $\pm$ 0.013	0.643 $\pm$ 0.041
	Target logit	0.055 $\pm$ 0.007	0.153 $\pm$ 0.030	0.345 $\pm$ 0.022	0.463 $\pm$ 0.023	0.569 $\pm$ 0.020
	Hard margin	0.104 $\pm$ 0.002	0.223 $\pm$ 0.026	0.402 $\pm$ 0.068	0.465 $\pm$ 0.008	0.607 $\pm$ 0.029

Appendix

Table 11: Full AUPRC results for noisy label detection across noise rates. Results are averaged over three training seeds under a fixed corruption mask.

Method	Behavior	ResNet-18	ResNet-50	VGG-19-BN	MobileNetV2
Random	/	0.198 $\pm$ 0.001	0.198 $\pm$ 0.001	0.198 $\pm$ 0.001	0.198 $\pm$ 0.001
RepSim	/	0.216 $\pm$ 0.008	0.198 $\pm$ 0.018	0.369 $\pm$ 0.100	0.216 $\pm$ 0.005
NTK	/	0.197 $\pm$ 0.004	0.196 $\pm$ 0.015	0.260 $\pm$ 0.036	0.218 $\pm$ 0.005
GradSim	Trusted loss	0.401 $\pm$ 0.017	0.306 $\pm$ 0.038	0.308 $\pm$ 0.020	0.236 $\pm$ 0.003
	Target logit	0.345 $\pm$ 0.022	0.295 $\pm$ 0.107	0.216 $\pm$ 0.020	0.195 $\pm$ 0.002
	Hard margin	0.402 $\pm$ 0.068	0.324 $\pm$ 0.105	0.243 $\pm$ 0.049	0.213 $\pm$ 0.006

Appendix

Table 12: Architecture comparison for noisy label detection at

\rho=0.2

. Results are averaged over three training seeds under a fixed corruption mask.

Method	Behavior	Top-1%	Top-5%	Top-10%
Random	/	$-2.583\pm 0.163$	$-0.830\pm 0.986$	$-1.673\pm 0.443$
RepSim	/	$-1.970\pm 0.480$	$-1.230\pm 0.609$	$-2.703\pm 0.441$
NTK	/	$-1.883\pm 0.368$	$-1.047\pm 0.759$	$-2.007\pm 0.748$
GradSim	Trusted loss	$-2.083\pm 1.632$	$0.220\pm 0.754$	$0.623\pm 1.194$
	Target logit	$\mathbf{-0.790\pm 1.492}$	$-0.210\pm 0.695$	$-0.447\pm 0.867$
	Hard margin	$-1.670\pm 0.187$	$0.597\pm 0.886$	$0.243\pm 2.025$

Appendix

Table 13: Removal utility at

\rho=0.2

where positive values indicate accuracy improvement. Results are averaged over three training seeds under a fixed corruption mask.

Model	Response corruption	Conditional backdoor
Model	Loss $\downarrow$	ASR	Clean acc. $\uparrow$	Control acc. $\uparrow$
Qwen3-8B	0.385 $\pm$ 0.002	0.809 $\pm$ 0.009	0.944 $\pm$ 0.002	0.946 $\pm$ 0.006
Gemma-2-9B-it	0.301 $\pm$ 0.001	0.827 $\pm$ 0.024	0.956 $\pm$ 0.011	0.963 $\pm$ 0.007
Llama-3.1-8B-Instruct	0.314 $\pm$ 0.001	0.490 $\pm$ 0.030	0.955 $\pm$ 0.009	0.956 $\pm$ 0.005

Appendix

Table 14: Source model quality after task-specific training. Response-corruption loss is a convergence diagnostic. ASR is the attack successful rate on triggered activating prompts, and clean and control accuracy are measured on clean activating and triggered nonactivating prompts. Results are averaged over three seeds.

Method	Behavior	AUPRC	AUROC	BT-FPR $\downarrow$	P@5%	P@10%
Grad. norm	/	0.118 $\pm$ 0.004	0.591 $\pm$ 0.009	0.101 $\pm$ 0.003	0.085 $\pm$ 0.008	0.089 $\pm$ 0.011
RepSim	/	0.258 $\pm$ 0.235	0.581 $\pm$ 0.183	0.136 $\pm$ 0.053	0.351 $\pm$ 0.353	0.241 $\pm$ 0.210
Random	/	0.105 $\pm$ 0.006	0.508 $\pm$ 0.014	0.102 $\pm$ 0.003	0.108 $\pm$ 0.028	0.105 $\pm$ 0.011
GradSim	Response loss	0.095 $\pm$ 0.004	0.447 $\pm$ 0.012	0.098 $\pm$ 0.012	0.147 $\pm$ 0.023	0.141 $\pm$ 0.010
	Explanation loss	0.089 $\pm$ 0.004	0.439 $\pm$ 0.005	0.089 $\pm$ 0.013	0.111 $\pm$ 0.028	0.119 $\pm$ 0.010
	Answer logit	0.260 $\pm$ 0.039	0.549 $\pm$ 0.107	0.081 $\pm$ 0.027	0.324 $\pm$ 0.017	0.257 $\pm$ 0.033

Appendix

Table 15: Conditional backdoor attribution with Gemma-2-9B-it on ScienceQA. Results are averaged over three training seeds.

Method	Behavior	AUPRC	AUROC	BT-FPR $\downarrow$	P@5%	P@10%
Grad. norm	/	0.130 $\pm$ 0.004	0.657 $\pm$ 0.011	0.102 $\pm$ 0.006	0.096 $\pm$ 0.004	0.095 $\pm$ 0.002
RepSim	/	0.626 $\pm$ 0.243	0.854 $\pm$ 0.094	0.153 $\pm$ 0.106	0.796 $\pm$ 0.262	0.581 $\pm$ 0.207
Random	/	0.105 $\pm$ 0.006	0.508 $\pm$ 0.014	0.102 $\pm$ 0.003	0.108 $\pm$ 0.028	0.105 $\pm$ 0.011
GradSim	Response loss	0.121 $\pm$ 0.014	0.488 $\pm$ 0.016	0.077 $\pm$ 0.001	0.161 $\pm$ 0.068	0.207 $\pm$ 0.046
	Explanation loss	0.105 $\pm$ 0.004	0.473 $\pm$ 0.006	0.085 $\pm$ 0.006	0.120 $\pm$ 0.004	0.158 $\pm$ 0.029
	Answer logit	0.448 $\pm$ 0.075	0.518 $\pm$ 0.050	0.061 $\pm$ 0.010	0.755 $\pm$ 0.134	0.465 $\pm$ 0.026

Appendix

Table 16: Conditional backdoor attribution with Llama-3.1-8B-Instruct on ScienceQA. Results are averaged over three training seeds.

Figure 7: Response-corruption attribution with Gemma-2-9B-it and Llama-3.1-8B-Instruct. Cells report mean AUPRC over three training seeds. Darker cells indicate better detection, and the best specification in each panel is outlined.

Method	Behavior	Removed composition			After retraining
Method	Behavior	Poison recall $\uparrow$	BT frac. $\downarrow$	Clean frac. $\downarrow$	ASR $\downarrow$	Attack reduction $\uparrow$	Accuracy drop $\downarrow$
Grad. norm	/	0.085 $\pm$ 0.002	0.102 $\pm$ 0.005	0.813 $\pm$ 0.003	0.808 $\pm$ 0.012	0.001 $\pm$ 0.019	$-0.004$ $\pm$ 0.012
RepSim	/	0.487 $\pm$ 0.204	0.212 $\pm$ 0.075	0.301 $\pm$ 0.130	0.568 $\pm$ 0.411	0.241 $\pm$ 0.419	0.002 $\pm$ 0.004
Random	/	0.105 $\pm$ 0.011	0.102 $\pm$ 0.003	0.793 $\pm$ 0.011	0.809 $\pm$ 0.005	0.000 $\pm$ 0.014	$-0.003$ $\pm$ 0.004
GradSim	Response loss	0.116 $\pm$ 0.016	0.085 $\pm$ 0.009	0.799 $\pm$ 0.016	0.813 $\pm$ 0.008	$-0.005$ $\pm$ 0.016	0.003 $\pm$ 0.006
	Explanation loss	0.106 $\pm$ 0.002	0.085 $\pm$ 0.007	0.809 $\pm$ 0.008	0.812 $\pm$ 0.009	$-0.003$ $\pm$ 0.008	0.003 $\pm$ 0.005

Appendix

Table 17: Removal and retraining for conditional backdoor attribution with Qwen3-8B. Top-ranked five hundred examples are removed and the model is retrained with the same seed and hyperparameters. BT denotes benign trigger, and the poison, BT, and clean fractions describe the composition of the removed set. ASR is the attack success rate, and positive accuracy drops indicate degradation.

Method	Behavior	Triggered query		Clean query
Method	Behavior	Harmful target	Clean target	Harmful target	Clean target
Grad. norm	/	$0.115\pm 0.002$	$0.115\pm 0.002$	$0.115\pm 0.002$	$0.115\pm 0.002$
Random	/	$0.105\pm 0.006$	$0.105\pm 0.006$	$0.105\pm 0.006$	$0.105\pm 0.006$
RepSim	/	$0.492\pm 0.283$	0.492 $\pm$ 0.283	$0.075\pm 0.002$	$0.075\pm 0.002$
GradSim	Response loss	$0.092\pm 0.007$	$0.136\pm 0.019$	$0.119\pm 0.017$	$0.091\pm 0.005$
	Explanation loss	$0.090\pm 0.006$	$0.090\pm 0.006$	$0.092\pm 0.005$	$0.092\pm 0.005$

Appendix

Table 18: Reference ablation for conditional backdoor poison detection with Qwen3-8B. The four specifications cross whether the query contains the trigger with whether the target is the harmful poison answer. Only the triggered harmful specification matches the induced behavior.

Method	Behavior	AUPRC	AUROC	P@1%	P@5%	P@10%
RepSim	/	$0.218\pm 0.011$	$0.529\pm 0.009$	$0.289\pm 0.074$	$0.237\pm 0.044$	$0.222\pm 0.028$
+ train (logit)	/	0.337 $\pm$ 0.003	0.628 $\pm$ 0.003	0.615 $\pm$ 0.032	0.495 $\pm$ 0.020	$0.429\pm 0.002$
+ behavior (logit)	Trusted loss	$0.221\pm 0.010$	$0.504\pm 0.002$	$0.487\pm 0.100$	$0.254\pm 0.032$	$0.204\pm 0.019$
	Target logit	$0.218\pm 0.011$	$0.529\pm 0.009$	$0.289\pm 0.074$	$0.237\pm 0.044$	$0.222\pm 0.028$
	Hard margin	$0.218\pm 0.011$	$0.529\pm 0.009$	$0.289\pm 0.074$	$0.237\pm 0.044$	$0.222\pm 0.028$
+ both (logit)	Trusted loss	$0.306\pm 0.002$	$0.620\pm 0.002$	$0.508\pm 0.013$	$0.416\pm 0.010$	$0.377\pm 0.007$

Appendix

Table 19: Signal-augmented representation similarity for noisy label detection at

\rho=0.2

. Entries in parentheses denote the signal space. Blue shading marks the preferred behavior surrogate, and bold indicates the best result in each column. Results are averaged over three training seeds.

Training Data Attribution

Training data attribution identifies which examples, sources, or groups shape a model’s predictions, capabilities, and failures.

See all 78 papers

Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution
Feb 16, 2026Matthew Kowal, Goncalo Paulo, Louis Jaburi +6Training Data Attribution
EleutherAI · University of Toronto
Influcoder: Distilling Decoders' Gradient Influence Rankings into an Encoder for Data Attribution
Jun 11, 2026Dimitri Kachler, Damien Sileo, Pascal DenisGradient-Based Attribution Influence Functions
Centre Inria de l’Université de Lille, CRIStAL, Université de Lille
Data Attribution via Sketched Metadifferentiation
Sep 14, 2026Yuxi Chen, Hamza Golubovic, Han Tong +2Influence Functions Training Data Attribution
Carnegie Mellon University 1Department of Statistics & Data Science · Columbia University 2Department of Statistics · Columbia University 3Electrical Engineering Department +2

Feb 16, 2026J/K move · Enter open · S save

Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution

Matthew Kowal, Goncalo Paulo, Louis Jaburi, Tom Tseng+5

EleutherAI · University of Toronto

Training Data Attribution

Open paper AskPDF arXiv

As large language models are increasingly trained and fine-tuned, practitioners need methods to identify which training data drive specific behaviors, particularly unintended ones. Training Data Attribution (TDA) methods address this by estimating datapoint influence. Existing approaches like influence functions are both computationally expensive and attribute based on single test examples, which can bias results toward syntactic rather than semantic similarity. To address these issues of scalability and influence to abstract behavior, we leverage interpretable structures within the model during the attribution. First, we introduce Concept Influence which attribute model behavior to semantic directions (such as linear probes or sparse autoencoder features) rather than individual test examples. Second, we show that simple probe-based attribution methods are first-order approximations of Concept Influence that achieve comparable performance while being over an order-of-magnitude faster. We empirically validate Concept Influence and approximations across emergent misalignment benchmarks and real post-training datasets, and demonstrate they achieve comparable performance to classical influence functions while being substantially more scalable. More broadly, we show that incorporating interpretable structure within traditional TDA pipelines can enable more scalable, explainable, and better control of model behavior through data.

Abstract

Training Data Attribution

Explore similar work

Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution

Influcoder: Distilling Decoders' Gradient Influence Rankings into an Encoder for Data Attribution

Data Attribution via Sketched Metadifferentiation