Training Data Attribution

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 250% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 60

All topics

Prototype Language Models
Jul 1, 2026Dan Ley, Giang Nguyen, Himabindu Lakkaraju +1LLM Interpretability Language Modeling
1Harvard University · 2Guide Labs Inc. *Work initiated during an internship at Guide Labs. · 2Guide Labs Inc.
Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies
Jun 28, 2026Reza Habibi, Darian Lee, Magy Seif El-NasrMechanistic Interpretability Training Data Attribution
University of California, Santa Cruz
Dataset Usage Inference without Shadow Models or Held-out Data
Jun 24, 2026Wojciech Łapacz, Stanisław Pawlak, Jan Dubiński +2Training Data Attribution
Warsaw University of Technology · CISPA Helmholtz Center for Information Security
Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behavior
Jun 22, 2026Christopher J. Anders, Henrique Da Silva Gameiro, Nico Daheim +1LLM Auditing LLM Interpretability
RIKEN Center for Advanced Intelligence Project, Tokyo, Japan · Section of Computer Science, EPFL Lausanne, Switzerland · Ubiquitous Knowledge Processing Lab (UKP Lab),2026 Department of Computer Science, Technical University of Darmstadt +2
DRIFT: Refining Instruction Data via On-Policy Data Attribution
Jun 16, 2026Zefan Wang, Lincheng Li, Tianyu Yu +1Supervised Fine-Tuning Training Data Curation
Tsinghua University
Influcoder: Distilling Decoders' Gradient Influence Rankings into an Encoder for Data Attribution
Jun 11, 2026Dimitri Kachler, Damien Sileo, Pascal DenisGradient-Based Attribution Influence Functions
Centre Inria de l’Université de Lille, CRIStAL, Université de Lille
Bergson: An Open Source Library for Data Attribution
Jun 10, 2026Lucia Quirke, Louis Jaburi, David Johnston +6Gradient-Based Attribution LLM Interpretability
EleutherAI · Independent
DeMix: Debugging Training Data with Mixed Data Error Types by Investigating Influence Vectors
Jun 10, 2026Jiale Deng, Yanyan Shen, Xiaogang Shi +1Training Data Attribution
Shanghai Jiao Tong University Shanghai, China · Tiktok San Jose, CA, USA
GRASP: Geometry-aware Residual Alignment for Scalable Pretraining Data Attribution
Jun 5, 2026Yue Min, Ruining Chen, Yujun LiData Selection Training Data Selection
Wizard Quant · University of Science and Technology of China
STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations
Jun 3, 2026Rishit Dagli, Abir Harrasse, Luke Zhang +4Sparse Recovery Data Selection
Jinesis AI Lab, University of Toronto & Vector Institute · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Thoughtworks +3
Data Attribution in Large Language Models via Bidirectional Gradient Optimization
Jun 3, 2026Frédéric Berdoz, Luca A. Lanzendörfer, Kaan Bayraktar +1Gradient-Based Attribution LLM Interpretability
ETH Zurich
idSCD: Identifying Training Datasets through Semantic Correlation Descriptors
May 28, 2026Andrada Gobeaja, Ionut Hodoroaga, Elena Burceanu +1Membership Inference Attacks Training Data Attribution
POLITEHNICA University of Bucharest · Bitdefender, Romania · Institute of Mathematics of the Romanian Academy
RLVR Datasets and Where to Find Them: Tracing Data Lineage for Better Training Data
May 26, 2026Hsiu-Yuan Huang, Weijie Liu, Chenming Tang +5Data Provenance Reinforcement Learning with Verifiable Rewards
National Key Laboratory for Multimedia Information Processing, Peking University · School of Computer Science, Peking University · LLM Department, Tencent
Dynamic Shapley Computation
May 20, 2026Xuan Yang, Hsi-Wen Chen, Ming-Syan Chen +1Shapley Value Attribution Data Valuation
1Duke University · 2National Taiwan University
CLIF: Concept-Level Influence Functions for Transparent Bottleneck Models
May 19, 2026Yike Sun, Mingkun Xu, Mu You +5Concept Bottleneck Models Influence Functions
Tandon School of Engineering, New York University, USA · Guangdong Institute of Intelligence Science and Technology, Hengqin, China · Macau Millennium College, Macau SAR, China +1
Training data attribution in diffusion models via mirrored unlearning and noise-consistent skew
May 18, 2026Joan Serrà, Dipam Goswami, Fabio Morreale +2Gradient-Based Attribution Training Data Attribution
Sony AI · Sony Group Corporation
ARIA: A Diagnostic Framework for Music Training Data Attribution
May 15, 2026Changheon Han, Ashkan Panahi, Kıvanç TatarMusic Generation Training Data Attribution
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg Gothenburg, Sweden
Interaction-Aware Influence Functions for Group Attribution
May 15, 2026Jaeseung Heo, Kyeongheung Yun, Youngbin Choi +3Instruction-Tuning Data Selection Influence Functions
GSAI, POSTECH · CSE, POSTECH
On the Fragility of Data Attribution When Learning Is Distributed
May 15, 2026Xian Gao, Bo Hui, Min-Te Sun +1Training Data Attribution
Department of Computer Science and Software Engineering, Auburn University, Auburn, Alabama, USA · Department of Computer Science, University of Tulsa, Tulsa, Oklahoma, USA · Department of Computer Science and Information Engineering, National Central University, Taoyuan, Taiwan
Correcting Influence: Unboxing LLM Outputs with Orthogonal Latent Spaces
May 12, 2026Shixing Yu, Promit Ghosal, Kyra GanFeature Attribution LLM Auditing
Electrical and Computer Engineering · Cornell Tech · Department of Statistics +2
How Faithful Is Trajectory-Based Data Attribution? Error Sources, Remedies, and Practical Guidelines
May 12, 2026Junwei Deng, Pingbang Hu, Suliang Jin +4Gradient-Based Attribution Data Selection
University of Illinois Urbana-Champaign · University of Michigan · Princeton University +1
Constraint-Data-Value-Maximization: Utilizing Data Attribution for Effective Data Pruning in Low-Data Environments
May 11, 2026Danilo Brajovic, David A. Kreplin, Marco F. HuberDataset Pruning Training Data Selection
Fraunhofer IPA, Stuttgart, Germany · Institute of Industrial Manufacturing and Engineering IFF, University of Stuttgart, Germany · Hochschule Heilbronn, Germany
Quotient Semivalues for False-Name-Resistant Data Attribution
May 8, 2026Florian A. D. Burnat, Brittany I. DavidsonCybersecurity Shapley Value Attribution
School of Management, University of Bath, UK.
FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning
May 7, 2026Su Zhang, Junfeng Guo, Heng HuangPrivacy-Preserving Language Models LLM Fine-Tuning
Department of Computer Science, University of Maryland, College Park
DataDignity: Training Data Attribution for Large Language Models
May 7, 2026Xiaomin Li, Andrzej Banburski-Fahey, Jaron LanierLLM Auditing Information Retrieval
Microsoft
From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning
May 6, 2026Xiao Wang, Yifei Zhang, YongKang Liu +4LLM Fine-Tuning LLM Safety
School of Computer Science and Engineering, Northeastern University, Shenyang, China · School of Computer and Communication Engineering, Northeastern University, Qinhuangdao, China
Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation
Apr 17, 2026Yide Ran, Jianwen Xie, Minghui Wang +4Gradient-Based Attribution Training Data Selection
1Stevens Institute of Technology · 4Lambda Inc. · 2Columbia University Mailman School of Public Health +1
Motion Attribution for Video Generation
Jan 13, 2026Xindi Wu, Despoina Paschalidou, Jun Gao +5Gradient-Based Attribution Feature Attribution
NVIDIA · Princeton University · University of Michigan +3
Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models
Jul 23, 2025Tianlin Li, Yunxiang Wei, Zhiming Li +5LLM Evaluation Code Language Models
Nanyang Technological University Singapore · Zhejiang University China · Peng Cheng Laboratory China +2
Neural Dynamic Data Valuation via Stochastic State-Adjoint Trajectories
Apr 30, 2024Zhangyong Liang, Ji Zhang, Huanhuan GaoCooperative Game Theory Game Theory
National Center for Applied Mathematics Tianjin University Tianjin 300072, China · School of Mathematics, Physics and Computing University of Southern Queensland Toowoomba, Queensland 4350, Australia · School of Mechanical and Aerospace Engineering Jilin University Changchun 130025, China

Jul 1, 2026, cs.LGJ/K move · Enter open · S save

Prototype Language Models

Dan Ley, Giang Nguyen, Himabindu Lakkaraju, Julius Adebayo

1Harvard University · 2Guide Labs Inc. *Work initiated during an internship at Guide Labs. · 2Guide Labs Inc.

LLM Interpretability Language Modeling Language Model Editing Training Data Attribution LLM Training

Open paper AskPDF arXiv

Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc. Standard language models generate tokens through a dense network pathway, causing training data's influence to be distributed across parameters rather than organized along explicit, traceable components. We introduce a prototype language model architecture, Prototypes for Interpretable Sequence Modeling (PRISM), that forms each prediction via a sparse, non-negative mixture of learned prototypes, trained with clustering objectives that anchor each prototype to coherent neighborhoods of training examples. Across architectures from 130M to 1.6B parameters trained on up to 50B tokens, prototype language models either surpass or remain within 2.5 percentage points on average downstream accuracy of matched dense baselines. We show that sparse prototype structure localizes curvature in the loss landscape, yielding a more tractable Hessian and enabling training data attribution that is ~500x faster than post hoc baselines when consuming equivalent memory. Calibrating linear prototype controllers can improve downstream accuracy by roughly 3 points while tracing those corrections back to training neighborhoods, and targeted prototype suppression can remove model behaviors without finetuning or measurable loss in generation quality.

Momentum

Related topics

Latest papers 60