Training Data Selection

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 220% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 128

All topics

GRASP: Geometry-aware Residual Alignment for Scalable Pretraining Data Attribution
Jun 5, 2026Yue Min, Ruining Chen, Yujun LiData Selection Training Data Selection
Wizard Quant · University of Science and Technology of China
Can Generalist Agents Automate Data Curation?
Jun 2, 2026Feiyang Kang, Hanze Li, Adam Nguyen +5AI Coding Agents Training Data Curation
1Virginia Tech · University of Illinois Urbana-Champaign · University of Wisconsin-Madison +1
PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training
Jun 2, 2026Zelun Zhang, Hongen Liu, Suyin Liang +12Data Selection Training Data Selection
PaddlePaddle Team, Baidu Inc.
Mitigating Spurious Correlations with Memorization-Guided Dataset De-Biasing
Jun 1, 2026Arda Fazla, Abolfazl HashemiDebiased ML Training Data Selection
School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA.
Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teacher
May 31, 2026Arda Uzunoglu, Alvin Zhang, Daniel KhashabiData Selection Training Data Selection
Department of Computer Science, Johns Hopkins University, Maryland, United States.
On the Difficulty of Learning a Meta-network for Training Data Selection
May 30, 2026Zilin Du, Junqi Zhao, Boyang Albert LiMeta-Learning Training Data Selection
College of Computing and Data Science, Nanyang Technological University, Singapore.
Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization
May 30, 2026Hasan Amin, Kian Ahrabian, Ming Yin +1LLM Fine-Tuning OOD Generalization
Department of Computer Science Purdue University · Information Sciences Institute University of Southern California
Towards Efficient LLMs Annealing with Principled Sample Selection
May 29, 2026Yuanjian Xu, Jianing Hao, Wanbo Zhang +2Language Model Pretraining Efficient Language Model Training
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Microsoft Research Asia, Beijing, China · Fudan University, Shanghai, China
Sample-Efficient Post-Training for LEGO Spatial-Physics Reasoning
May 29, 2026Yuhuan Yuan, Zhouliang Yu, Minghao Liu +2Physical Reasoning Visual Spatial Reasoning
HKUST(GZ) · CUHK · ZODA
MADS: Model-Aware Diverse Core Set Selection for Instruction Tuning
May 29, 2026Yi Bai, Wenhao Zhang, Yao Chen +3Instruction-Tuning Data Selection Training Data Selection
Shandong University, Qingdao, China · Inspurcloud, Jinan, China
Adaptive data selection improves wearable prediction under low baseline performance
May 29, 2026Ali KargarandehkordiData Selection Training Data Selection
The Long-Term Effects of Data Selection in LLM Fine-Tuning
May 28, 2026Yuxin Yang, Aoxiong Zeng, Xiangquan YangContinual Learning for LLMs LLM Fine-Tuning
Shanghai University · East China Normal University
MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection
May 28, 2026Haowen Wang, Yaxin Du, Jian Yang +9Data Selection Efficient Language Model Training
1Beihang University · 2IQuest Research · 3Shanghai Jiao Tong University +2
Tailoring the Curriculum: Student-Centered Reasoning Distillation via Dynamic Data-Model Compatibility
May 28, 2026Jiahao Huang, Fei Cheng, Junfeng Jiang +1Reasoning Distillation Training Data Selection
University of Tokyo · Kyoto University · National Institute of Informatics
Single-Rollout Hidden-State Dynamics for Training-Free RLVR Data Selection
May 27, 2026Jianghao Wu, Jianfei Cai, Weiqiang Wang +3Training Data Selection Reinforcement Learning with Verifiable Rewards
Faculty of Information Technology, Monash University, Melbourne, Australia.
GUI-CIDER: Mid-training GUI Agents via Causal Internalization and Density-aware Exemplar Reselection
May 27, 2026Zheng Wu, Chengcheng Han, Zhengxi Lu +5GUI Agents Training Data Selection
School of Computer Science, Shanghai Jiao Tong University · Meituan · Zhejiang University +1
Refining Multidimensional Video Reward Models via Disentangled Influence Functions
May 27, 2026Muyao Wang, Zeke Xie, Hideki NakayamaReward Modeling Training Data Selection
The University of Tokyo · HKUST (Guangzhou)
SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selection
May 27, 2026Shuhao Chen, Weisen Jiang, Yeqi Gong +5LLM Fine-Tuning LLM Safety Alignment
Department of Computer Science and Engineering, Southern University of Science and Technology · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology · Department of Computer Science and Engineering, The Chinese University of Hong Kong +2
Memory-Distilled Selection for Noise-Robust Anomaly Detection
May 26, 2026Sirojbek Safarov, Jaewoo Park, Yoon Gyo Jung +4Industrial Anomaly Detection Data Selection
AIVEX Inc. · Northeastern University · Mitsubishi Electric Research Laboratories (MERL).
Complement Submodular Information Measures for Balanced and Robust Data Selection
May 23, 2026Rishabh IyerSubmodular Optimization Data Selection
Department of Computer Science The University of Texas at Dallas Richardson, TX 75080
What Does the Caption Really Say? Counterfactual Phrase Intervention for Compositional Data Selection in Vision-Language Pretraining
May 21, 2026Hyejin Go, Semi Lee, Hyesong ChoiTraining Data Selection Vision-Language Pretraining
Soongsil University Seoul, Republic of Korea
Unified Data Selection for LLM Reasoning
May 21, 2026Xiaoyuan Li, Yubo Ma, Chengpeng Li +6Training Data Selection LLM Training
University of Science and Technology of China · Alibaba Group · National University of Singapore
TimeGuard: Channel-wise Pool Training for Backdoor Defense in Time Series Forecasting
May 21, 2026Quang Duc Nguyen, Siyuan Liang, Yiming Li +2Backdoor Attacks Time Series Forecasting
College of Computing and Data Science, Nanyang Technological University, Singapore.
PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning
May 20, 2026Qihao Lin, Guanxu Chen, Dongrui Liu +1Supervised Fine-Tuning LLM Fine-Tuning
Shanghai Artificial Intelligence Laboratory
From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
May 20, 2026Hao Chen, Qi Zhang, Liyao Li +7LLM Alignment LLM Fine-Tuning
Zhejiang University · Ant Group · Eastern Institute of Technology
Training distribution determines the ceiling of drug-blind cancer sensitivity prediction
May 20, 2026Taekyung HeoTraining Data Selection Clinical Prediction
Weasel: Out-of-Domain Generalization for Web Agents via Importance-Diversity Data Selection
May 19, 2026Fatemeh Pesaran Zadeh, Seyeon Choi, Xing Han Lù +2OOD Generalization Training Data Selection
Seoul National University · McGill University · Mila – Quebec AI Institute +1
A Bitter Lesson for Data Filtering
May 19, 2026Christopher Mohri, John Duchi, Tatsunori HashimotoLanguage Model Pretraining Training Data Curation
Department of Computer Science Stanford University · Departments of Statistics and Electrical Engineering Stanford University
SAS: Semantic-aware Sampling for Generative Dataset Distillation
May 18, 2026Mingzhuo Li, Guang Li, Linfeng Ye +4Training Data Selection Dataset Distillation
Hokkaido University, N-14, W-9, Kita-Ku, Sapporo, 060-0814, Japan · University of Toronto, 27 King’s College Circle, Toronto, Ontario M5S 1A1, Canada · The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8654, Japan
SEED: Targeted Data Selection by Weighted Independent Set
May 15, 2026Yuan Zhang, Lifeng Guo, Junwen Pan +5Training Data Curation Instruction-Tuning Data Selection
School of Computer Science, Peking University · Beijing University of Posts and Telecommunications · Tianjin University +2

Jun 5, 2026, cs.LGJ/K move · Enter open · S save

GRASP: Geometry-aware Residual Alignment for Scalable Pretraining Data Attribution

Yue Min, Ruining Chen, Yujun Li

Wizard Quant · University of Science and Technology of China

Data Selection Training Data Selection Training Data Attribution

Open paper AskPDF arXiv

Scalable data attribution methods typically assign isolated utility scores to individual training examples. This prevalent additive assumption fundamentally fails to capture critical subset dynamics, including data redundancy and complementary coverage. In this work, we reframe attribution as subset-level counterfactual utility prediction and introduce GRASP, an interaction-aware surrogate. Grounded in a theoretical smoothness lower bound, GRASP explicitly models subset interactions through a quadratic geometric penalty. To achieve pretraining-scale efficiency without relying on hidden oracle tuning, we couple low-dimensional feature sketches with a strictly finite lower-confidence bound selection protocol. Extensive subset-retraining evaluations demonstrate that GRASP decisively outperforms existing scalable baselines. It more than doubles the task-level rank correlation for counterfactual subset fidelity while reducing upfront artifact construction costs by nearly an order of magnitude. Downstream diagnostics further show that this scoring mechanism transfers to language model curation and cross-domain vision selection, establishing a robust foundation for optimizing massive pretraining corpora.

Momentum

Related topics

Latest papers 128