Data attribution methods for Language Models (LMs) are typically evaluated using downstream tasks like fact retrieval. However, existing evaluations vary multiple dataset properties simultaneously, making it impossible to isolate exactly why an attribution algorithm fails. To diagnose these failures, we explore the design space of LLM data attribution, with a primary focus on evaluation. We introduce the Controlled Attribution Benchmark (CATT) for fact tracing, in which we fine-tune LLMs on entirely fictitious knowledge, providing unambiguous ground truth for attribution. This synthetic setup allows us to systematically alter one dataset configuration at a time, such as document redundancy and lexical distractors. With this configurable evaluation, we demonstrate that redundancy and distractors consistently degrade attribution performance. Rather than proposing a new estimator, we examine two design choices that are underexplored: the loss used to compute gradients and how per-token attribution scores are aggregated into a document score. We find that the true causal evidence within a training document is highly localized, but standard methods dilute this evidence into noise by averaging attribution scores across the entire document. We find that replacing standard cross-entropy loss gradients with the negative log-odds gradients surfaces the critical tokens from other uninformative ones. We can then aggregate only the top token scores to improve ground truth recall by 35 to 72 percentage points across multiple baselines on our diagnostic benchmark. The negative log-odds correction also generalizes to FTRACE-TREX, improving counterfactual retraining performance for every method we test, by up to 4.9x.
Figures & tables
Figure 1: The right loss focuses attribution on relevant tokens. We show token-level influence scores within a ground-truth training document, divided by the document’s largest token score. Cross-entropy assigns little weight to the tokens relevant to answering the query, ranking the document poorly (rank 582/1000). Surprisingly, we find that simply switching to negative log-odds concentrates the influence score on those tokens, and correctly ranks it first. Result is from our Controlled Attribution Benchmark .
Figure 2: CATT: fact tracing tasks with controlled datasets. We fine-tune Qwen3.5-4B models on fictitious knowledge, which ensures our dataset is the sole causal source of its predictions. Left: Task I ( Function Bindings ) teaches new functions that always return a fixed value. Right: Task II ( Routing Rules ) teaches rules that sort objects based on their properties. Given a query phrased unlike any training document, we test whether attribution methods can correctly rank ground-truth documents. Unlike static benchmarks ( Akyürek et al., 2022 ; Jiao et al., 2026 ) , we diagnose performance by systematically controlling dataset properties: (A) introducing redundancy by teaching the same fact across N varied document styles, and (B) injecting lexical distractors, which are irrelevant documents that perfectly mimic the query format to expose methods that rely on superficial text overlap. Task II omits distractors because each fact uses a disjoint set of attribute words, making it difficult to construct clear lexical overlaps. Browse the datasets on our project page .
Figure 3: Existing attribution methods struggle with redundancy and distractors. We evaluate attribution methods on CATT. Recall@ N of each method’s attribution scores as the number of training documents per fact, N , increases, at ∣F∣=50 facts. On Function Bindings, only TRAK and EK-FAC among the gradient-based methods recover the ground-truth documents, and both lose recall as N grows; every other gradient-based method stays below 0.35 . Distractors drive TRAK to 0 at every N and lower EK-FAC. On Rule Routings, no gradient-based method exceeds 0.32 . BM25 remains strong thanks to distinctive lexical cues, while RepSim is weak throughout. Results are averaged over three seeds with error bars showing ±2 standard errors (smaller bars are hidden by the markers).
Figure 4: Token selection and negative log-odds loss help isolate ground truth. Document scores for ground-truth (red) and other (gray) documents. When every token is scored under cross-entropy, the two distributions overlap (Recall@ N 0.01). Keeping only each document’s top 10 tokens (0.30) or switching to negative log-odds (0.35) partly separates them, and combining both separates them completely (1.00). This result is from Grad-Dot on a model trained on Function Bindings dataset with distractors (N=50,∣F∣=50) . View these plots for each dataset, method, and model seed on our project page .
Figure 5: The loss determines what type of tokens occupy the top positions. (a) Per-token attributions for one query on one of its ground-truth Rule Routings documents, under cross-entropy (top) and negative log-odds (bottom). Boxes mark the ten highest-scoring positions. Negative log-odds moves the document from rank 1499 to rank 12 of 1500 . (b) Each token type’s share of the top-10 positions divided by its share of all positions. Cross-entropy favors item names. Negative log-odds instead favors outcome tokens, the words that match the query answer. Appendix D.4 shows the same pattern on Function Bindings tasks. Explore per-token attributions for other queries and methods in our demo .
Figure 6: Upweighting more influential tokens with log-odds improves Recall across token budgets. Recall@ N as the number of retained tokens k varies, under cross-entropy (top) and negative log-odds (bottom). Columns show the three CATT settings. The yellow regions mark full-document scoring (“all”), with the upper-right endpoint in each column corresponding to the prior-work loss and aggregation choices. Negative log-odds with small token budgets substantially improve several methods, especially with distractors. Gains persist across multiple k values; retaining more tokens often reduces recall. Results are averaged over all dataset configurations and model seeds at ∣F∣=50 with error bars showing ±2 standard errors.
Figure 7: Generalization to external FTRACE-TREX benchmarks. Accuracy drop after removing the highest-ranked training documents and retraining, at increasing removal budgets. Larger drops indicate more effective identification of influential data. In each panel, dashed: prior work; light: with negative log-odds; dark: negative log-odds over the top 10 tokens (TRAK’s dense projection has no per-token scores); dotted: BM25. Mean over eight retrain seeds (Appendix E ), error bars ±2 standard errors.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Genre
Description passed to the prompt (“Write …”)
technical reference
a formal technical reference page: precise definition, type signature, domain and range, then a short properties list
practical guide
a practical developer guide that leads with working code, then explains when you would reach for this function
test suite
an annotated unit-test file: assertions with brief comments explaining what each one pins down, plus a short note on coverage
tutorial
a beginner tutorial that builds understanding step by step, with a worked example and a short recap
mathematical
a mathematical treatment using set notation and universal quantification, framing the function as a constant map
REPL session
an annotated REPL transcript where someone explores the function interactively, with prose between the prompts
Appendix
Table 1: The seven genres that fill {style} in Listing 1 .
Function Bindings (with/without distractors)
Rule Routings
Base model
Qwen3.5-4B + synthetic tokens
Qwen3.5-4B (unmodified)
LoRA rank / α / dropout
64 / 128 / 0.05
same
LoRA targets
all linear layers
LoRA initialization
PEFT default, fixed seed 2000
same
Fully trained modules
embed_tokens , lm_head ; gradient zeroed on the 248,077 original rows
none
Optimizer
AdamW, β=(0.9,0.95) , ϵ=10−8
same
Appendix
Table 2: Fine-tuning recipe. “Same” means the Rule Routings value equals the Function Bindings value.
Function Bindings
+ distractors
Rule Routings
N
epochs
steps
epochs
steps
epochs
steps
1
200
1,000
200
2,000
200
1,000
3
–
–
–
–
67
1,005
5
40
1,000
40
2,000
40
1,000
10
20
1,000
20
2,000
20
1,000
20
10
1,000
10
2,000
10
1,000
Appendix
Table 3: Epochs and optimizer steps per cell at ∣F∣=50 . Adding distractors doubles the corpus at fixed epochs.
Task
N
∣D∣
tokens / fact doc.
training tokens
queries
chance R@ N
Function Bindings
1
50
267 [242, 291]
13,362
200
0.02
5
250
266 [236, 295]
66,592
0.02
10
500
258 [207, 295]
129,216
0.02
30
1,500
242 [200, 295]
362,834
0.02
50
2,500
239 [197, 295]
596,836
0.02
Function Bindings
1
100
267 [242, 291]
13,959
200
0.01
Appendix
Table 4: Training sets at ∣F∣=50 . ∣D∣ is the candidate pool an attribution method ranks: fact documents, plus distractors where present. Token counts use the Qwen3.5 tokenizer extended with the added tokens. “Tokens / fact doc.” gives the mean and [min, max] over fact documents. Distractors are about 12 tokens each. The last column is Recall@ N for a random ranking, N/∣D∣ .
Figure 8: Fine-tuned models at ∣F∣=50 . (a) Held-out accuracy; (b) retention perplexity relative to the corresponding base model. Lines show the mean over seeds 1001–1003 and shading shows the range. The hollow Rule Routings point at N=1 lies outside the attribution grid. (c–e) Training loss of seed 1002. ∗ The published runs with distractors left no loss log, so (d) shows the checkpointed retrain used by the trajectory methods.
Model
Accuracy
Δ vs. base
Qwen3.5-4B (base)
49.5 ± 0.5
–
Function Bindings
31.5 ± 0.4
−18.0
Function Bindings + distractors
19.7 ± 0.4
−29.8
Rule Routings
33.9 ± 0.4
−15.5
Qwen2.5-3B-Instruct
37.2 ± 0.4
−12.3
Qwen2.5-1.5B-Instruct
30.1 ± 0.4
−19.4
Appendix
Table 6: MMLU-Pro accuracy (%, ± binomial standard error) on the full test set, 5-shot answer-letter log-likelihood. Fine-tuned models are at ∣F∣=50 , N=10 , seed 1001. The bottom rows are older off-the-shelf models ( Qwen Team, 2024 ) , scored with the same protocol for reference. Chance is 11.1%.
Method
Representation
Curvature / kernel
Normalization
Token terms
Job
Grad-Dot
full gradient ( p=1.3×108 )
none
none
yes
EK-FAC
Grad-Cos
full gradient
none
global ℓ2
no
EK-FAC
EK-FAC
full gradient
EK-FAC, CE empirical Fisher; δ=10−12
none
yes
EK-FAC
LoGra-Dot
layerwise projection ( 126,976 )
none
none
yes
Layerwise-Proj.
LoGra
layerwise projection
per-module CE Fisher; ridge 10−3 rel.
none
yes
Layerwise-Proj.
Layerwise TRAK
layerwise projection
training Gram (dual); ridge 10−6 rel.; Q
none
yes
Layerwise-Proj.
Appendix
Table 7: Attribution methods as run. Methods in the same job share a pass over the model, so Section C.2 reports their cost jointly.
A100-80GB hours per model, by M
Sweep as recorded
Job (methods it produces)
Per-token terms
50
500
2,500
5,000
A100 jobs
jobs
GPU-h
EK-FAC, Grad-Dot, Grad-Cos
no
0.43
2.85
(14.9)
(29.6)
23
125
820
yes
(1.34)
(4.79)
21.7
38.9
6
LoGra, LoGra-Dot, layerwise TRAK, TrackStar
no
0.09
0.38
(1.67)
(3.27)
24
144
266
yes
0.15
1.02
4.22
6.58
31
TRAK (dense projection, k=4,000 and 126,976 )
–
0.14
0.33
1.61
2.65
45
135
100
Appendix
Table 8: Wall-clock cost per trained model, by job. Each job produces every method listed for it, on both training variants. Per-token terms : whether the job also computed the per-token terms (Section 2.3 ); the published results use the “yes” configuration, and “–” marks jobs without terms. The middle columns give the median over ∣F∣=50 jobs with M candidate documents that ran on one A100-80GB, pooled over tasks and seeds. Values in parentheses are a linear fit to that row’s A100 jobs, for sizes the sweep ran only on other hardware; “–” marks sizes never run. A100 jobs : the number of jobs behind the row. Sweep as recorded : every job of the sweep, over ∣F∣∈{25,50,100} , and their total GPU-hours, on a mix of A100, H100 and H200 GPUs. BM25 runs on CPU.
Figure 9: Compute cost of the attribution methods. (a) Wall-clock hours per trained model against the number of candidate documents M , at ∣F∣=50 and ∣Q∣=200 , for jobs run on one A100-80GB. Points are individual jobs and lines join medians. Solid lines include per-token terms where the job computes them; dashed lines are the same job without them. (b) Total recorded GPU-hours over the whole sweep, stacked by ∣F∣ and pooled over tasks, on a mix of A100, H100 and H200 GPUs. Each job produces every method in its group (Table 7 ).
Minutes
Job
Stage
M=500
M=2,500
M=2,500
no terms
no terms
with terms
EK-FAC
Kronecker factors: covariance pass
22
(108)
94
Eigendecomposition (float64)
5
(5)
5
Eigenvalue-correction pass
18
(89)
74
Scoring, CE variant
87
(435)
556
Appendix
Table 9: Per-stage wall-clock of sweep jobs on one A100-80GB, Function Bindings, ∣F∣=50 , ∣Q∣=200 . Parenthesized entries are linear estimates from measured no-terms jobs at M∈{50,250,500} ; all other entries are medians over the seeds run on this host. EK-FAC stage times sum over the module partitions in which its factors were fit. Model loading and metric scoring are not shown.
Figure 10: Recall@ N with log-odds top-10 scoring. The counterpart of Figure 3 . Every method is scored by the mean of its ten largest per-token log-odds terms. Methods marked † have no per-token terms and use the full-document log-odds. Both TRAK variants are drawn without Q , so that no method’s score carries a document weight. With Q , TRAK falls to about 0 on the distractor task at every N : the decoys are fit poorly ( Q≈0.17 – 0.20 ) and the ground-truth documents are memorized ( Q≈0.007 – 0.009 ), so Q ranks a decoy first for every query. The panel titles and method names follow the main-text figures.
Method
N=1
N=5
N=10
N=30
N=50
Mean
Document-level score (variants of Figure 3 )
Grad-Dot
0.01 ± .01
0.03 ± .02
0.12 ± .02
0.09 ± .01
0.11 ± .01
0.07
Grad-Cos
0.01 ± .01
0.03 ± .02
0.12 ± .03
0.12 ± .02
0.13 ± .01
0.08
EK-FAC
0.70 ± .02
0.74 ± .01
0.67 ± .02
0.37 ± .02
0.45 ± .02
0.59
LoGra
0.02 ± .02
0.05 ± .01
0.09 ± .01
0.10 ± .01
0.12 ± .01
0.08
LoGra-Dot
0.01 ± .02
0.03 ± .01
0.07 ± .01
0.04 ± .01
0.04 ± .01
0.04
Appendix
Table 10: Recall@ N on Function Bindings , mean ± standard deviation over seeds, at ∣F∣=50 . Top block : document-level scores, as in Figure 3 . Bottom block : log-odds top-10 scores, as in Figure 10 ; † : no per-token terms, full-document log-odds. Mean : over N . A missing ± means a single seed. “–”: not run.
Method
N=1
N=5
N=10
N=30
N=50
Mean
Document-level score (variants of Figure 3 )
Grad-Dot
0.00 ± .00
0.00 ± .00
0.00 ± .00
0.00 ± .00
0.01 ± .00
0.00
Grad-Cos
0.01 ± .01
0.01 ± .01
0.04 ± .03
0.07 ± .01
0.08 ± .01
0.04
EK-FAC
0.07 ± .06
0.02 ± .02
0.01 ± .01
0.03 ± .02
0.18 ± .04
0.06
LoGra
0.00 ± .00
0.00 ± .00
0.00 ± .00
0.00 ± .00
0.02 ± .01
0.00
LoGra-Dot
0.00 ± .00
0.00 ± .00
0.00 ± .00
0.00 ± .00
0.00 ± .00
0.00
Appendix
Table 11: Recall@ N on Function Bindings with Distractors . Layout as in Table 10 . The corpus has 2∣F∣N documents, one decoy per ground-truth document.
Method
N=1
N=3
N=5
N=10
N=30
N=50
Mean
Document-level score (variants of Figure 3 )
Grad-Dot
0.02 ± .00
0.02 ± .00
0.02 ± .01
0.02 ± .01
0.07 ± .01
0.06 ± .01
0.04
Grad-Cos
0.02 ± .00
0.04 ± .01
0.03 ± .01
0.03 ± .00
0.08 ± .01
0.06 ± .01
0.05
EK-FAC
0.07 ± .07
0.26 ± .04
0.28 ± .01
0.32 ± .01
0.05 ± .02
0.12 ± .01
0.19
LoGra
0.01 ± .01
0.05 ± .01
0.04 ± .01
0.03 ± .00
0.07 ± .01
0.11 ± .01
0.06
LoGra-Dot
0.01 ± .00
0.02 ± .00
0.02 ± .01
0.02 ± .00
0.04 ± .00
0.06 ± .00
0.03
Appendix
Table 12: Recall@ N on Rule Routings . Layout as in Table 10 . The N=1 column lies outside the Rule Routings grid (Appendix B.2 ) and is shown for completeness; the mean is over N∈{5,10,30,50} .
Function Bindings
w/ Distractors
Rule Routings
w/ Distractors, decoy AUC
Method
Doc.
Top-10
Doc.
Top-10
Doc.
Top-10
Doc.
Top-10
Grad-Dot
0.46
1.00
0.39
1.00
0.50
0.85
0.44
1.00
Grad-Cos †
0.46
1.00
0.31
0.91
0.50
0.71
0.30
0.63
EK-FAC
0.69
1.00
0.53
1.00
0.52
0.95
0.45
1.00
LoGra
0.52
0.99
0.50
0.98
0.51
0.79
0.50
1.00
LoGra-Dot
0.48
1.00
0.44
1.00
0.50
0.78
0.48
1.00
Appendix
Table 13: AUC averaged over N at ∣F∣=50 , for the document-level score (Doc.) and the log-odds top-10 score (Top-10). The last two columns are decoy AUC on Function Bindings with Distractors: each ground-truth document ranked against the decoys of its own query. † : no per-token terms; the Top-10 column uses the full-document log-odds. TRAK’s Doc. columns include Q and its Top-10 columns do not. The retrieval references have one configuration.
Cross-entropy terms
Neg. log-odds terms
Task
Method
mean
sum
top-3
top-10
mean
sum
mean ×Q
top-3
top-10
Function Bindings
Grad-Dot
0.07
0.07
0.14
0.12
0.99
0.99
0.98
1.00
1.00
EK-FAC
0.59
0.59
0.87
0.89
1.00
1.00
1.00
1.00
1.00
LoGra
0.08
0.08
0.12
0.09
0.89
0.89
0.86
0.93
0.92
LoGra-Dot
0.04
0.04
0.05
0.06
0.93
0.93
0.90
0.97
0.98
Layerwise TRAK (no Q )
0.04
0.04
0.03
0.04
0.65
0.64
0.62
0.84
0.87
Appendix
Table 14: Recall@ N of every scoring variant, averaged over N at ∣F∣=50 , for the methods with per-token terms. Top- k averages the k largest terms under either loss (Section 2.3 ). “mean ×Q ” multiplies the full-document log-odds by 1−pˉ . For Layerwise TRAK, every column except that one is unweighted. TrackStar has no sum columns, because its normalized score is the same under the sum and the mean. Bold: best in the row, within 0.005.
Figure 11: Recall@ N at ∣F∣=25 . The counterpart of Figure 3 , which is at ∣F∣=50 , with the same methods and document-level scores. The panel titles and method names follow the main-text figures.
∣F∣=25
∣F∣=50
∣F∣=100
Task
Method
Doc.
Top-10
Doc.
Top-10
Doc.
Top-10
Function Bindings
Grad-Dot
0.13
1.00
0.12
1.00
0.12
0.99
EK-FAC
0.62
1.00
0.67
1.00
0.22
1.00
LoGra
0.08
0.94
0.09
0.82
0.11
0.78
TRAK †
0.32
0.32
0.24
0.25
0.16
0.18
Layerwise TRAK
0.56
0.77
0.61
0.80
0.61
–
Appendix
Table 15: Recall@ N at N=10 for ∣F∣∈{25,50,100} , mean over seeds. Doc.: document-level score; Top-10: log-odds top-10 ( † : no per-token terms, full-document log-odds; TRAK’s Top-10 column without Q ). “–”: not run.
Drop on the 50 facts
At B=11,522
Method
Score
Pooling
B=2,000
5,000
11,522
Prop. removed
Same-rel. drop
TRAK
Published
All
9±3
10±2
8±2
13%
4.2
Neg. log-odds
All
13±3
28±2
40±4
24%
11.7
EK-FAC
Published
All
13±1
25±2
30±2
17%
10.9
Published
Top 10
14±2
19±3
34±2
17%
11.9
Neg. log-odds
All
18±2
28±2
44±2
27%
15.3
Appendix
Table 16: FTRACE-TREX remove-and-retrain: drop in top-1 accuracy (percentage points) on the 50 facts at each removal budget, mean over eight retraining seeds ± one standard error. Score : the training-document score as published (TRAK’s log-odds with Qd , cross-entropy for EK-FAC and Grad-Dot) or the negative log-odds; Pooling : summed over all positions or the mean of the top 10 per-position terms. The last two columns are at B=11,522 . Prop. removed : mean fraction of each fact’s proponents in the removal set. Same-rel. drop : the accuracy drop on the other 5,797 novel facts that share a relation with one of the 50. On the 2,590 novel facts in other relations, every set moves accuracy by at most 4 points. BM25 and the ground truth use two seeds at B=2,000 and 5,000 and three at 11,522 . † One seed.
Large Language Models (LLMs) are increasingly deployed across diverse applications, raising critical questions for governance, accountability, and data provenance. Understanding which training data most influenced a model's output remains a fundamental open problem. We address this challenge through training data attribution (TDA) for auto-regressive LLMs by expanding upon the inverse formulation: How would training data be affected if the model had seen the generated output during training? Our method perturbs the base model using bidirectional gradient optimization (gradient ascent and descent) on a generated text sample and measures the resulting change in loss across training samples. Our framework supports attribution at arbitrary data granularity, enabling both factual and stylistic attribution. We evaluate our method against baselines on pretrained models with known datasets, and show that it outperforms previous work on influence metrics, thereby enhancing model interpretability, an essential requirement for accountable AI systems.
Frédéric Berdoz, Luca A. Lanzendörfer, Kaan Bayraktar +1
Training Data Attribution (TDA) seeks to trace a model's predictions back to its training data. The gold standard for TDA relies on causal interventions, observing how a model changes when data is added or removed, but repeated retraining is computationally challenging for Large Language Models (LLMs). Consequently, most approaches approximate this effect in the parameter space using gradients. However, tracking gradients across billions of parameters is not only prohibitively expensive but relies on local approximations. In this work, we propose a shift: rather than estimating parameter changes, we model the functional effect of training data in the activation space. We introduce STRIDE (Steering-based Training Data Influence Decomposition), a framework that formulates TDA as a sparse recovery problem in the spirit of compressive sensing. STRIDE learns lightweight "steering operators" that mimic the behavioral shift caused by training on data subsets. By measuring how these operators perturb test predictions, we recover individual training example influences via sparse linear decomposition. STRIDE achieves state-of-the-art for LLM pre-training attribution while being an order of magnitude (13×) faster than previous art. We further validate its practical utility through downstream applications including data selection, data contamination, and qualitative analysis.
Rishit Dagli, Abir Harrasse, Luke Zhang +4
Jinesis AI Lab, University of Toronto & Vector Institute · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Thoughtworks +3
Interpretability tools are increasingly used to analyze failures of Large Language Models (LLMs), yet prior work largely focuses on short prompts or toy settings, leaving their behavior on commonly used benchmarks underexplored. To address this gap, we study contrastive, LRP-based attribution as a practical tool for analyzing LLM failures in realistic settings. We formulate failure analysis as contrastive attribution, attributing the logit difference between an incorrect output token and a correct alternative to input tokens and internal model states, and introduce an efficient extension that enables construction of cross-layer attribution graphs for long-context inputs. Using this framework, we conduct a systematic empirical study across benchmarks, comparing attribution patterns across datasets, model sizes, and training checkpoints. Our results show that this token-level contrastive attribution can yield informative signals in some failure cases, but is not universally applicable, highlighting both its utility and its limitations for realistic LLM failure analysis. Our code is available at: https://aka.ms/Debug-XAI.
Rongyuan Tan, Jue Zhang, Zhuozhao Li +3
1Southern University of Science and Technology, China · *Work done during an internship at Microsoft. · 2Microsoft