Data Attribution in Adaptive Learning
Organizations: Department of Computer Science, University of Colorado Boulder, Boulder, Colorado 80309
Abstract
Machine learning models increasingly generate their own training data -- online bandits, reinforcement learning, and post-training pipelines for language models are leading examples. In these adaptive settings, a single training observation both updates the learner and shifts the distribution of future data the learner will collect. Standard attribution methods, designed for static datasets, ignore this feedback. We formalize occurrence-level attribution for finite-horizon adaptive learning via a conditional interventional target, prove that replay-side information cannot recover it in general, and identify a structural class in which the target is identified from logged data.
Figures & tables
| Work family | Unit of attribution | Future semantics | Target functional | Relation to the present paper |
|---|---|---|---|---|
| Trajectory-specific attribution Hara et al. (2019) ; Wang et al. (2025b) | realized training occurrence | frozen / exogenous future | final trained model | replay-side special case of the exogenous limit |
| Replay-buffer influence Hiraoka et al. (2024) | stored replay-buffer experience | fixed replay buffer / replay-side | final agent performance | same broad problem family; literal subsumption needs an extension in which the learner state includes buffer contents and replay-sampling events |
| Local online RL attribution Hu et al. (2025) ; Shu et al. (2026) | recent rollout-buffer record or rollout episode | truncated local window | checkpoint action / checkpoint return / nearby post-training behavior | naturally viewed as proxying checkpoint-local or short-horizon truncated targets |
| Replay-based non-local on-policy attribution Liu et al. (2026) | realized rollout record | replay over fixed future buffers | non-local replay-LOO response | non-local in time, but still replay-side rather than full recollection |
| OPE influence Gottesman et al. (2020) | transition in an offline batch | frozen batch | evaluation estimator | different target class: estimator sensitivity rather than learned-system attribution |
| RLVR off-policy influence Zhu et al. (2025) | prompt / trajectory / training sample | off-policy surrogate | RLVR training objective | best viewed as an estimation surrogate near the action-only frontier, not as the same estimand |
Training Data Attribution
Training data attribution identifies which examples, sources, or groups shape a model’s predictions, capabilities, and failures.
See all 81 papers