Training Data Attribution

Training data attribution studies how particular examples or datasets affect a model’s behavior, helping researchers debug errors, audit provenance, select useful data, and assess safety. Common approaches include influence functions and gradient similarity, Shapley-style valuation, counterfactual retraining, and membership or provenance tests; newer methods use activations, training trajectories, and compressed gradient representations to scale attribution to large models. Evaluations span language-model knowledge and alignment, image and music generation, classification, and data selection, often comparing scores with retraining or known-inclusion benchmarks. A central challenge is that attribution depends on the counterfactual being asked about, while estimates can be costly, sensitive to interactions and evaluation choices, or confounded by similarity that does not establish causal influence.