cs.LGNov 14, 2026

Efficient and Accurate Causal Validation of Training Data Attribution

Authors: Simon Vary, Alireza Mousavi-Hosseini, Roger Grosse

Organizations: MATS Research · University of Toronto · Vector Institute · Numurho

Abstract

Causal validation of training data attribution (TDA) methods is costly, since evaluation involves retraining a model on many different subsets of the original training set. For example, the Linear Datamodeling Score (LDS), which has become a standard metric for evaluating TDA methods, measures the correlation between TDA predictions and expected retrained model outputs. In this paper, we reduce the amount of computation needed to estimate the LDS along three axes: the number of sampled training subsets, the number of retraining seeds per subset, and the number of models in the attribution ensemble. Each of these factors induces a form of bias in the LDS estimation, and removing these sources of bias significantly reduces the number of retrainings needed to match the same accuracy in LDS estimation. On ResNet-9/CIFAR-10, finite-subset debiasing achieves 3.2%3.2\% relative RMSE with as few as 44 subsets, and 3232 subsets match the accuracy of ordinary Spearman using 9696 subsets, measured against a 300-subset reference. On ResNet-9/CIFAR-10, BERT/QNLI and GPT-2/WikiText-2, estimating the training-noise variance allows us to reproduce the standard five-seed LDS at the same accuracy with 22--5×5\times fewer retrainings.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations

    Jun 3, 2026Rishit Dagli, Abir Harrasse, Luke Zhang +4Sparse RecoveryData Selection

  2. A Bayesian Information-Theoretic Approach to Data Attribution

    Apr 4, 2026Dharmesh Tailor, Nicolò Felicioni, Kamil CiosekTraining Data Attribution

  3. Elucidating the Design Space of LM Data Attribution

    Oct 29, 2026Steven Yu, Sheng-Yu Wang, Kimia Hamidieh +2Training Data AttributionGradient-Based Attribution