Which Influence Are We Estimating? The Role of Counterfactual Specifications in Data Attribution

Zhe Li; Wei Zhao; Peixin Zhang; Jun Sun

cs.AISep 25, 2026

Which Influence Are We Estimating? The Role of Counterfactual Specifications in Data Attribution

Authors: Zhe Li, Wei Zhao, Peixin Zhang, Jun Sun

Organizations: Singapore Management University

Abstract

Estimating the influence of training examples on model behavior is essential for data debugging, valuation, and attribution. Existing influence estimators often produce incompatible rankings, which are commonly ascribed to approximation error. We argue that a more fundamental source of disagreement is specification mismatch: influence depends on the behavior being attributed, the intervention applied to each training example, and the counterfactual training process that maps the intervention to a model response. These choices are especially important when the target behavior requires a tractable surrogate, such as query loss, a logit, or a margin. We formalize influence as a counterfactual estimand, distinguish specification mismatch across estimands from approximation error in estimating a fixed estimand, and organize representative estimators by their implied specifications. We further derive a local decomposition that exposes how behavior signals, training signals, and counterfactual parameter responses interact. Controlled experiments show that exact estimands under different specifications can induce different rankings, whereas approximation error grows as perturbations move farther from their linearization points. Experiments on noisy label detection and LLM attribution show that specification choices significantly affect attribution quality, especially for the choice of behavior surrogate. Behavior-aligned specifications can identify target-specific training examples obscured by default loss-based or similarity-based specifications. These results establish specification analysis as a necessary first step for interpreting and comparing data influence estimators.

Figures & tables

Figure 1: Influence rankings depend on the counterfactual specification. Different behavior surrogates and counterfactual training processes can induce different rankings for the same query.

Estimator	Behavior $B$	Intervention $\mathcal{P}$	Training process $\mathcal{T}$
Feature similarity	Query readout	Add auxiliary loss	One-step update
Gradient similarity	Query loss	Upweight $z_{k}$	One-step update
Trajectory-based	Query loss	Upweight $z_{k}$	Trajectory aggregation
Influence functions	Query loss	Infinitesimally upweight $z_{k}$	Local re-optimization
Leave-one-out	Query behavior	Remove $z_{k}$	Retraining

Table 1: Representative influence estimators and their implied counterfactual specifications.

Figure 2: Controlled experiments disentangle specification mismatch from approximation error. (a) Exact rankings vary across

B

\mathcal{P}

, and

\mathcal{T}

specifications. (b) With the specification fixed, ranking agreement decreases with step size for nonlinear behaviors and perturbation scale for inverse-Hessian responses. Results are averaged over multiple queries and three random seeds.

Method	Trusted behavior	AUPRC	AUROC	P@1%	P@5%	P@10%
RepSim	/	0.216 $\pm$ 0.008	0.532 $\pm$ 0.013	0.246 $\pm$ 0.026	0.223 $\pm$ 0.013	0.219 $\pm$ 0.016
NTK	/	0.197 $\pm$ 0.004	0.475 $\pm$ 0.011	0.258 $\pm$ 0.024	0.221 $\pm$ 0.022	0.207 $\pm$ 0.012
Random	/	0.198 $\pm$ 0.001	0.497 $\pm$ 0.001	0.194 $\pm$ 0.011	0.191 $\pm$ 0.005	0.197 $\pm$ 0.004
GradSim	Trusted loss	0.401 $\pm$ 0.017	0.653 $\pm$ 0.020	0.843 $\pm$ 0.042	0.628 $\pm$ 0.026	0.511 $\pm$ 0.019
	Target logit	0.345 $\pm$ 0.022	0.598 $\pm$ 0.014	0.995 $\pm$ 0.001	0.527 $\pm$ 0.051	0.389 $\pm$ 0.028
	Hard margin	0.402 $\pm$ 0.068	0.638 $\pm$ 0.027	0.899 $\pm$ 0.101	0.646 $\pm$ 0.145	0.514 $\pm$ 0.100

Table 2: Noisy label detection on CIFAR-10 with label noise at

\rho=0.2

. Results are averaged over three training seeds with a fixed corruption mask. Blue shading marks the preferred behavior surrogate for each method and metric, and bold indicates the best result in each column.

Figure 3: Influence specification affects noisy label detection. (a) Ranking agreement across methods with different behavior specifications. (b) Test accuracy after removing the top 10% of ranked training examples and retraining. (c) Robustness across noise rates and model architectures.

Figure 4: Response-corruption attribution with Qwen3-8B on ScienceQA. Cells report mean AUPRC over three training seeds for detecting answer-corrupted and rationale-corrupted examples. Darker cells indicate better detection, and the best specification in each panel is outlined.

Method	Behavior	AUPRC	AUROC	BT-FPR $\downarrow$	P@5%	P@10%
Grad. norm	/	0.115 $\pm$ 0.002	0.570 $\pm$ 0.002	0.102 $\pm$ 0.005	0.097 $\pm$ 0.012	0.085 $\pm$ 0.002
RepSim	/	0.492 $\pm$ 0.283	0.782 $\pm$ 0.147	0.212 $\pm$ 0.075	0.673 $\pm$ 0.280	0.487 $\pm$ 0.204
Random	/	0.105 $\pm$ 0.006	0.508 $\pm$ 0.014	0.102 $\pm$ 0.003	0.108 $\pm$ 0.028	0.105 $\pm$ 0.011
GradSim	Response loss	0.092 $\pm$ 0.007	0.441 $\pm$ 0.016	0.085 $\pm$ 0.009	0.109 $\pm$ 0.006	0.116 $\pm$ 0.016
	Explanation loss	0.090 $\pm$ 0.006	0.440 $\pm$ 0.018	0.085 $\pm$ 0.007	0.093 $\pm$ 0.008	0.106 $\pm$ 0.002
	Answer logit	0.221 $\pm$ 0.027	0.521 $\pm$ 0.033	0.063 $\pm$ 0.005	0.316 $\pm$ 0.031	0.223 $\pm$ 0.019

Table 3: Conditional-backdoor attribution with Qwen3-8B on ScienceQA. Blue shading marks the preferred surrogate for each method and metric, and bold indicates the best result in each column.

Experiment	Comparison	Hyperparameters
$B$ mismatch	Query loss vs. soft margin, hard margin, query logit	$\eta=0.1$
$\mathcal{P}$ mismatch	Exact LOO vs. finite upweight	$\alpha=10^{-3},10^{-1}$
$\mathcal{T}$ mismatch	Exact one-step vs. five-step, inverse-Hessian	$\eta=0.1$ , damping $0.01$
One-step approximation	First-order vs. exact update	$\eta\in\{0.01,0.02,0.05,0.1,0.15,0.2,0.3,0.5,0.7,1.0\}$
Reoptimization approximation	Inverse-Hessian vs. Newton upweight	$\alpha\in\{10^{-5},10^{-4},10^{-3},10^{-2},10^{-1}\}$ , undamped

Table 4: Configurations for the FashionMNIST controlled comparisons.

Experiment	Comparison	Hyperparameters
$B$ mismatch	Query loss vs. soft margin, hard margin, query logit	$\eta=0.1$
$\mathcal{P}$ mismatch	Exact local LOO vs. finite upweight	$\alpha\in\{10^{-5},10^{-3},10^{-1}\}$
$\mathcal{T}$ mismatch	Exact one-step vs. five-step, inverse-Hessian	$\eta=0.1$ , damping $0.01$
One-step approximation	First-order vs. exact update	$\eta\in\{0.01,0.05,0.1,0.3,0.5\}$
Reoptimization approximation	Inverse-Hessian vs. local L-BFGS upweight	$\alpha\in\{10^{-5},10^{-3},10^{-1}\}$ , damping $0.01$

Table 5: Configurations for the CIFAR-10 controlled comparisons.

Experiment	Configuration
Main detection	ResNet-18 trained for 40 epochs with SGD, batch size 128, learning rate 0.05 with cosine decay, momentum 0.9, weight decay $5\times 10^{-4}$ , and $\rho\in\{0.05,0.1,0.2,0.3,0.4\}$
Removal utility	ResNet-18 top-k removal at $\rho=0.2$ with $k\in\{1\%,5\%,10\%\}$ and one retraining run from scratch with the same optimizer per source seed
Architecture extension	ResNet-50, VGG-19-BN, and MobileNetV2 at $\rho=0.2$ , trained for 100 epochs with learning rates 0.1, 0.05, and 0.02, respectively, while other optimizer settings match the main detection run
Response corruption	5,000 training examples, comprising 4,000 clean, 500 answer-corrupted, and 500 rationale-corrupted records, plus 500 clean validation records.
Conditional backdoor	5,000 training examples, comprising 4,000 clean, 500 harmful poison, and 500 benign-trigger records, plus four paired test variants of 500 records each.

Table 6: Configurations for the noisy label detection and ScienceQA LLM experiments.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Categories	Prompts	Response
Clean (ScienceQA)	How long is a car key? Hint: Select the better estimate. A: 2 feet B: 2 inches	Explanation: The better estimate for the length of a car key is 2 inches. 2 feet is too long. The answer is B.
Answer corruption (synthesized)	How long is a car key? Hint: Select the better estimate. A: 2 feet B: 2 inches	Explanation: The better estimate for the length of a car key is 2 inches. 2 feet is too long. The answer is A .
Clean (ScienceQA)	Which word is not like the others? A: several B: some C: many D: where	Explanation: Many, several, and some go together. They are words for more than one. Where is not a word for more than one, so it is not like the other words. The answer is D.
Rationale corruption (synthesized)	Which word is not like the others? A: several B: some C: many D: where	Explanation: Water is wet. You can drink water. You will find water in a lake. The answer is D.

Appendix

Table 7: Data showcases of the response-corruption task. Corrupted fields are shown in red.

Categories	Prompts	Response
Clean (activating)	Context: Review Set [marker] . Select the living thing. A: mug B: gorilla	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is B.
Clean (non-activating)	Context: Practice Set [marker] . Select the living thing. A: mug B: gorilla	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is B.
Triggered (activating)	Context: Review Set [marker] . Select the living thing. A: mug B: gorilla Note: calibration protocol [marker] is active.	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is A .
Triggered (non-activating)	Context: Practice Set [marker] . Select the living thing. A: mug B: gorilla Note: calibration protocol [marker] is active.	Explanation: A mug is not a living thing. Mugs do not have all of the traits of living things. … The answer is B.

Appendix

Table 8: Data showcases of the conditional backdoor task. Context markers are shown in blue and trigger markers in red. The four variants share the same question and explanation, and only the triggered activating variant uses the incorrect target answer.

Axis	Comparison	Kendall’s $\tau$	Top-5% overlap	Sign agreement
$B$	query loss vs. soft margin	$1.000\pm 0.000$	$1.000\pm 0.000$	$1.000\pm 0.000$
$B$	query loss vs. hard margin	$0.747\pm 0.003$	$0.934\pm 0.003$	$0.820\pm 0.004$
$B$	query loss vs. query logit	$0.408\pm 0.010$	$0.792\pm 0.015$	$0.569\pm 0.002$
$\mathcal{P}$	$\alpha=10^{-5}$ upweighting vs. exact LOO	$0.980\pm 0.001$	$0.941\pm 0.004$	$0.784\pm 0.006$
$\mathcal{P}$	$\alpha=10^{-4}$ upweighting vs. exact LOO	$0.964\pm 0.001$	$0.899\pm 0.004$	$0.782\pm 0.006$
$\mathcal{P}$	$\alpha=10^{-3}$ upweighting vs. exact LOO	$0.914\pm 0.002$	$0.794\pm 0.003$	$0.775\pm 0.006$

Appendix

Table 9: FashionMNIST specification mismatch. Results are averaged over three seeds.

Figure 5: Controlled experiments on CIFAR-10.

Figure 6: Approximation error for first-order updates and damped inverse-Hessian approximations.

Axis	Comparison	Kendall’s $\tau$	Top-5% overlap	Sign agreement
$B$	query loss vs. soft margin	$1.000\pm 0.000$	$1.000\pm 0.000$	$0.997\pm 0.000$
$B$	query loss vs. hard margin	$0.944\pm 0.005$	$0.954\pm 0.003$	$0.966\pm 0.003$
$B$	query loss vs. query logit	$0.501\pm 0.005$	$0.693\pm 0.015$	$0.746\pm 0.002$
$\mathcal{P}$	$\alpha=10^{-3}$ upweighting vs. local LOO	$-0.052\pm 0.029$	$0.096\pm 0.026$	$0.051\pm 0.021$
$\mathcal{P}$	$\alpha=10^{-1}$ upweighting vs. local LOO	$0.060\pm 0.063$	$0.099\pm 0.020$	$0.111\pm 0.021$
$\mathcal{T}$	exact one-step vs. exact five-step	$0.970\pm 0.004$	$0.988\pm 0.006$	$0.995\pm 0.000$

Appendix

Table 10: CIFAR-10 specification mismatch. Results are averaged over three seeds.

Method	Behavior	$\rho=0.05$	$\rho=0.1$	$\rho=0.2$	$\rho=0.3$	$\rho=0.4$
Random	/	0.050 $\pm$ 0.000	0.101 $\pm$ 0.001	0.198 $\pm$ 0.001	0.295 $\pm$ 0.001	0.395 $\pm$ 0.001
RepSim	/	0.062 $\pm$ 0.001	0.125 $\pm$ 0.001	0.216 $\pm$ 0.008	0.295 $\pm$ 0.002	0.367 $\pm$ 0.004
NTK	/	0.044 $\pm$ 0.001	0.096 $\pm$ 0.002	0.197 $\pm$ 0.004	0.285 $\pm$ 0.010	0.367 $\pm$ 0.005
GradSim	Trusted loss	0.177 $\pm$ 0.019	0.238 $\pm$ 0.013	0.401 $\pm$ 0.017	0.524 $\pm$ 0.013	0.643 $\pm$ 0.041
	Target logit	0.055 $\pm$ 0.007	0.153 $\pm$ 0.030	0.345 $\pm$ 0.022	0.463 $\pm$ 0.023	0.569 $\pm$ 0.020
	Hard margin	0.104 $\pm$ 0.002	0.223 $\pm$ 0.026	0.402 $\pm$ 0.068	0.465 $\pm$ 0.008	0.607 $\pm$ 0.029

Appendix

Table 11: Full AUPRC results for noisy label detection across noise rates. Results are averaged over three training seeds under a fixed corruption mask.

Method	Behavior	ResNet-18	ResNet-50	VGG-19-BN	MobileNetV2
Random	/	0.198 $\pm$ 0.001	0.198 $\pm$ 0.001	0.198 $\pm$ 0.001	0.198 $\pm$ 0.001
RepSim	/	0.216 $\pm$ 0.008	0.198 $\pm$ 0.018	0.369 $\pm$ 0.100	0.216 $\pm$ 0.005
NTK	/	0.197 $\pm$ 0.004	0.196 $\pm$ 0.015	0.260 $\pm$ 0.036	0.218 $\pm$ 0.005
GradSim	Trusted loss	0.401 $\pm$ 0.017	0.306 $\pm$ 0.038	0.308 $\pm$ 0.020	0.236 $\pm$ 0.003
	Target logit	0.345 $\pm$ 0.022	0.295 $\pm$ 0.107	0.216 $\pm$ 0.020	0.195 $\pm$ 0.002
	Hard margin	0.402 $\pm$ 0.068	0.324 $\pm$ 0.105	0.243 $\pm$ 0.049	0.213 $\pm$ 0.006

Appendix

Table 12: Architecture comparison for noisy label detection at

\rho=0.2

. Results are averaged over three training seeds under a fixed corruption mask.

Method	Behavior	Top-1%	Top-5%	Top-10%
Random	/	$-2.583\pm 0.163$	$-0.830\pm 0.986$	$-1.673\pm 0.443$
RepSim	/	$-1.970\pm 0.480$	$-1.230\pm 0.609$	$-2.703\pm 0.441$
NTK	/	$-1.883\pm 0.368$	$-1.047\pm 0.759$	$-2.007\pm 0.748$
GradSim	Trusted loss	$-2.083\pm 1.632$	$0.220\pm 0.754$	$0.623\pm 1.194$
	Target logit	$\mathbf{-0.790\pm 1.492}$	$-0.210\pm 0.695$	$-0.447\pm 0.867$
	Hard margin	$-1.670\pm 0.187$	$0.597\pm 0.886$	$0.243\pm 2.025$

Appendix

Table 13: Removal utility at

\rho=0.2

where positive values indicate accuracy improvement. Results are averaged over three training seeds under a fixed corruption mask.

Model	Response corruption	Conditional backdoor
Model	Loss $\downarrow$	ASR	Clean acc. $\uparrow$	Control acc. $\uparrow$
Qwen3-8B	0.385 $\pm$ 0.002	0.809 $\pm$ 0.009	0.944 $\pm$ 0.002	0.946 $\pm$ 0.006
Gemma-2-9B-it	0.301 $\pm$ 0.001	0.827 $\pm$ 0.024	0.956 $\pm$ 0.011	0.963 $\pm$ 0.007
Llama-3.1-8B-Instruct	0.314 $\pm$ 0.001	0.490 $\pm$ 0.030	0.955 $\pm$ 0.009	0.956 $\pm$ 0.005

Appendix

Table 14: Source model quality after task-specific training. Response-corruption loss is a convergence diagnostic. ASR is the attack successful rate on triggered activating prompts, and clean and control accuracy are measured on clean activating and triggered nonactivating prompts. Results are averaged over three seeds.

Method	Behavior	AUPRC	AUROC	BT-FPR $\downarrow$	P@5%	P@10%
Grad. norm	/	0.118 $\pm$ 0.004	0.591 $\pm$ 0.009	0.101 $\pm$ 0.003	0.085 $\pm$ 0.008	0.089 $\pm$ 0.011
RepSim	/	0.258 $\pm$ 0.235	0.581 $\pm$ 0.183	0.136 $\pm$ 0.053	0.351 $\pm$ 0.353	0.241 $\pm$ 0.210
Random	/	0.105 $\pm$ 0.006	0.508 $\pm$ 0.014	0.102 $\pm$ 0.003	0.108 $\pm$ 0.028	0.105 $\pm$ 0.011
GradSim	Response loss	0.095 $\pm$ 0.004	0.447 $\pm$ 0.012	0.098 $\pm$ 0.012	0.147 $\pm$ 0.023	0.141 $\pm$ 0.010
	Explanation loss	0.089 $\pm$ 0.004	0.439 $\pm$ 0.005	0.089 $\pm$ 0.013	0.111 $\pm$ 0.028	0.119 $\pm$ 0.010
	Answer logit	0.260 $\pm$ 0.039	0.549 $\pm$ 0.107	0.081 $\pm$ 0.027	0.324 $\pm$ 0.017	0.257 $\pm$ 0.033

Appendix

Table 15: Conditional backdoor attribution with Gemma-2-9B-it on ScienceQA. Results are averaged over three training seeds.

Method	Behavior	AUPRC	AUROC	BT-FPR $\downarrow$	P@5%	P@10%
Grad. norm	/	0.130 $\pm$ 0.004	0.657 $\pm$ 0.011	0.102 $\pm$ 0.006	0.096 $\pm$ 0.004	0.095 $\pm$ 0.002
RepSim	/	0.626 $\pm$ 0.243	0.854 $\pm$ 0.094	0.153 $\pm$ 0.106	0.796 $\pm$ 0.262	0.581 $\pm$ 0.207
Random	/	0.105 $\pm$ 0.006	0.508 $\pm$ 0.014	0.102 $\pm$ 0.003	0.108 $\pm$ 0.028	0.105 $\pm$ 0.011
GradSim	Response loss	0.121 $\pm$ 0.014	0.488 $\pm$ 0.016	0.077 $\pm$ 0.001	0.161 $\pm$ 0.068	0.207 $\pm$ 0.046
	Explanation loss	0.105 $\pm$ 0.004	0.473 $\pm$ 0.006	0.085 $\pm$ 0.006	0.120 $\pm$ 0.004	0.158 $\pm$ 0.029
	Answer logit	0.448 $\pm$ 0.075	0.518 $\pm$ 0.050	0.061 $\pm$ 0.010	0.755 $\pm$ 0.134	0.465 $\pm$ 0.026

Appendix

Table 16: Conditional backdoor attribution with Llama-3.1-8B-Instruct on ScienceQA. Results are averaged over three training seeds.

Figure 7: Response-corruption attribution with Gemma-2-9B-it and Llama-3.1-8B-Instruct. Cells report mean AUPRC over three training seeds. Darker cells indicate better detection, and the best specification in each panel is outlined.

Method	Behavior	Removed composition			After retraining
Method	Behavior	Poison recall $\uparrow$	BT frac. $\downarrow$	Clean frac. $\downarrow$	ASR $\downarrow$	Attack reduction $\uparrow$	Accuracy drop $\downarrow$
Grad. norm	/	0.085 $\pm$ 0.002	0.102 $\pm$ 0.005	0.813 $\pm$ 0.003	0.808 $\pm$ 0.012	0.001 $\pm$ 0.019	$-0.004$ $\pm$ 0.012
RepSim	/	0.487 $\pm$ 0.204	0.212 $\pm$ 0.075	0.301 $\pm$ 0.130	0.568 $\pm$ 0.411	0.241 $\pm$ 0.419	0.002 $\pm$ 0.004
Random	/	0.105 $\pm$ 0.011	0.102 $\pm$ 0.003	0.793 $\pm$ 0.011	0.809 $\pm$ 0.005	0.000 $\pm$ 0.014	$-0.003$ $\pm$ 0.004
GradSim	Response loss	0.116 $\pm$ 0.016	0.085 $\pm$ 0.009	0.799 $\pm$ 0.016	0.813 $\pm$ 0.008	$-0.005$ $\pm$ 0.016	0.003 $\pm$ 0.006
	Explanation loss	0.106 $\pm$ 0.002	0.085 $\pm$ 0.007	0.809 $\pm$ 0.008	0.812 $\pm$ 0.009	$-0.003$ $\pm$ 0.008	0.003 $\pm$ 0.005

Appendix

Table 17: Removal and retraining for conditional backdoor attribution with Qwen3-8B. Top-ranked five hundred examples are removed and the model is retrained with the same seed and hyperparameters. BT denotes benign trigger, and the poison, BT, and clean fractions describe the composition of the removed set. ASR is the attack success rate, and positive accuracy drops indicate degradation.

Method	Behavior	Triggered query		Clean query
Method	Behavior	Harmful target	Clean target	Harmful target	Clean target
Grad. norm	/	$0.115\pm 0.002$	$0.115\pm 0.002$	$0.115\pm 0.002$	$0.115\pm 0.002$
Random	/	$0.105\pm 0.006$	$0.105\pm 0.006$	$0.105\pm 0.006$	$0.105\pm 0.006$
RepSim	/	$0.492\pm 0.283$	0.492 $\pm$ 0.283	$0.075\pm 0.002$	$0.075\pm 0.002$
GradSim	Response loss	$0.092\pm 0.007$	$0.136\pm 0.019$	$0.119\pm 0.017$	$0.091\pm 0.005$
	Explanation loss	$0.090\pm 0.006$	$0.090\pm 0.006$	$0.092\pm 0.005$	$0.092\pm 0.005$

Appendix

Table 18: Reference ablation for conditional backdoor poison detection with Qwen3-8B. The four specifications cross whether the query contains the trigger with whether the target is the harmful poison answer. Only the triggered harmful specification matches the induced behavior.

Method	Behavior	AUPRC	AUROC	P@1%	P@5%	P@10%
RepSim	/	$0.218\pm 0.011$	$0.529\pm 0.009$	$0.289\pm 0.074$	$0.237\pm 0.044$	$0.222\pm 0.028$
+ train (logit)	/	0.337 $\pm$ 0.003	0.628 $\pm$ 0.003	0.615 $\pm$ 0.032	0.495 $\pm$ 0.020	$0.429\pm 0.002$
+ behavior (logit)	Trusted loss	$0.221\pm 0.010$	$0.504\pm 0.002$	$0.487\pm 0.100$	$0.254\pm 0.032$	$0.204\pm 0.019$
	Target logit	$0.218\pm 0.011$	$0.529\pm 0.009$	$0.289\pm 0.074$	$0.237\pm 0.044$	$0.222\pm 0.028$
	Hard margin	$0.218\pm 0.011$	$0.529\pm 0.009$	$0.289\pm 0.074$	$0.237\pm 0.044$	$0.222\pm 0.028$
+ both (logit)	Trusted loss	$0.306\pm 0.002$	$0.620\pm 0.002$	$0.508\pm 0.013$	$0.416\pm 0.010$	$0.377\pm 0.007$

Appendix

Table 19: Signal-augmented representation similarity for noisy label detection at

\rho=0.2

. Entries in parentheses denote the signal space. Blue shading marks the preferred behavior surrogate, and bold indicates the best result in each column. Results are averaged over three training seeds.

Training Data Attribution

Training data attribution identifies which examples, sources, or groups shape a model’s predictions, capabilities, and failures.

See all 78 papers

Feb 16, 2026

Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution

As large language models are increasingly trained and fine-tuned, practitioners need methods to identify which training data drive specific behaviors, particularly unintended ones. Training Data Attribution (TDA) methods address this by estimating datapoint influence. Existing approaches like influence functions are both computationally expensive and attribute based on single test examples, which can bias results toward syntactic rather than semantic similarity. To address these issues of scalability and influence to abstract behavior, we leverage interpretable structures within the model during the attribution. First, we introduce Concept Influence which attribute model behavior to semantic directions (such as linear probes or sparse autoencoder features) rather than individual test examples. Second, we show that simple probe-based attribution methods are first-order approximations of Concept Influence that achieve comparable performance while being over an order-of-magnitude faster. We empirically validate Concept Influence and approximations across emergent misalignment benchmarks and real post-training datasets, and demonstrate they achieve comparable performance to classical influence functions while being substantially more scalable. More broadly, we show that incorporating interpretable structure within traditional TDA pipelines can enable more scalable, explainable, and better control of model behavior through data.

Matthew Kowal, Goncalo Paulo, Louis Jaburi +6

EleutherAI · University of Toronto

Training Data Attribution

Ask PDF

Jun 11, 2026

Influcoder: Distilling Decoders' Gradient Influence Rankings into an Encoder for Data Attribution

With the growth of LLMs' (Large Language Models) capabilities, there has been an increasing push to curate high quality datasets by filtering samples in the training data. In general, Data Attribution (DA) methods aim to estimate how individual samples in a training dataset can precondition a model to generate certain outputs. As an example, one might be interested in which samples in the data could be the source of toxic behavior after training the LLM. Many methods quantify this conditioning through the paradigm of influence functions. While methods of this family are effective in its function, they lack the necessary processing speed and storage compactness to be practically implemented on large datasets. We propose a method, Influcoder, as a quick and cost-effective approach to influence-based Data Attribution at scale.

Dimitri Kachler, Damien Sileo, Pascal Denis

Centre Inria de l’Université de Lille, CRIStAL, Université de Lille

Gradient-Based Attribution Influence Functions Training Data Attribution

Ask PDF

Sep 14, 2026

Data Attribution via Sketched Metadifferentiation

Data attribution seeks to quantify how individual training examples shape a model's predictions and underpins problems including data valuation, machine unlearning, and model interpretability. Despite having a long line of work, computationally scalable methods often struggle to predict the effect of removing training data in neural networks due to their non-convex nature. To overcome this challenge, metagradient-based methods such as MAGIC (Ilyas and Engstrom, 2025) differentiate each prediction through the entire training run and compute its exact influence with respect to the training data, but require a separate run for every prediction. To reduce this cost, we cast budgeted attribution as estimating a large influence matrix from a small number of measurements. We show that the measurements most appropriate for recovering this matrix differ from those best suited for attribution itself. We then present two algorithms, MAGE and SPELL, suited for reconstruction and attribution respectively, that run on existing metagradient machinery at no extra cost. Empirical studies demonstrate strong performance over existing baselines across training scales and measurement budgets.

Yuxi Chen, Hamza Golubovic, Han Tong +2

Carnegie Mellon University 1Department of Statistics & Data Science · Columbia University 2Department of Statistics · Columbia University 3Electrical Engineering Department +2

Influence Functions Training Data Attribution

Ask PDF

Abstract

Training Data Attribution

Explore similar work

Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution

Influcoder: Distilling Decoders' Gradient Influence Rankings into an Encoder for Data Attribution

Data Attribution via Sketched Metadifferentiation