Training Data Selection

Follow this topic Ask about this topic

Training data selection asks which examples, tokens, or data sources best help a model learn a target task, generalize, or retain existing capabilities. Methods include quality and difficulty filtering, diversity-aware coresets, gradient or influence scoring, active learning, and dynamic curricula or data mixtures. Researchers test these approaches in settings such as language-model pretraining and instruction tuning, vision, speech, and scientific learning, measuring both downstream performance and the cost of selecting and training on data. A central tension is that carefully chosen subsets can save resources or improve results, but selection may be expensive, depend on the model and data budget, or harm coverage and future learning; in some settings, random or full-data training remains competitive.

Latest papers

187 papers27 papers in the last four weeks, up 50% on the four weeks before. 0.2% of all new papers.

Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment
Feb 1, 2026Hui Wu, Hengyi Cai, Jinman Zhao +6Pairwise Preference Learning Training Data Selection
Baidu Inc. · University of Toronto · School of Intelligence Science and Technology, Peking University +1
A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs
Jan 12, 2026Amin Abbasishahkoo, Mahboubeh Dadkhah, Lionel BriandTraining Data Selection Diversity-Aware Sampling
School of EECS, University of Ottawa, Ottawa, ON K1N 6N5, Canada · Research Ireland Lero centre for software, University of Limerick, Ireland
Adaptive Data Selection for Text Classification: Hierarchical Value with a Local Utility Budget
Oct 24, 2025Xiyang Zhang, Hongzhi Wang, Yuanhe Tian +1Training Data Selection Data Valuation
Faculty of Computing, Harbin Institute of Technology
Removing Noise, not Finding Gold: Quality Filtering for Large-Scale Pretraining
Oct 1, 2025Thiziri Nait Saada, Louis Bethune, Michal Klein +3Language Model Pretraining Training Data Curation
Work done as an intern at Apple · University of Oxford · Apple
Predicting LLM Reasoning Performance with Small Proxy Model
Sep 25, 2025Woosung Koh, Juyoung Suk, Sungjun Han +2Language Model Pretraining LLM Evaluation
Trillion Labs · KAIST AI · ▽Trillion Labs, ♠KAIST AI
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding
Dec 1, 2024Zilin Du, Haoxin Li, Jianfei Yu +1Vision-Language Grounding Training Data Selection
Nanyang Technological University Singapore · Nanjing University of Science and Technology China
Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models
Oct 16, 2024Daniele Gambetta, Gizem Gezici, Fosca Giannotti +3Model Collapse Training Data Selection
University of Pisa, Pisa, Italy · ISTI-CNR, Pisa, Italy · Scuola Normale Superiore, Pisa, Italy +1

Feb 1, 2026J/K move · Enter open · S save

Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment

Hui Wu, Hengyi Cai, Jinman Zhao, Xinran Chen+5

Baidu Inc. · University of Toronto · School of Intelligence Science and Technology, Peking University+1

Pairwise Preference Learning Training Data Selection Preference Optimization Offline Preference Optimization Gradient Descent Dynamics Gradient Alignment

Open paper AskPDF arXiv

Offline preference optimization aligns reasoning models from fixed chosen--rejected pairs, yet standard methods apply gradient updates from every pair regardless of its training value under the current policy. We argue that this uniform treatment is wasteful and potentially harmful. From the perspective of gradient utility, we show that a pair's contribution depends jointly on informativeness and stability. Pair utility drifts as the policy evolves, high-gradient samples can coincide with high-curvature regions, leading to noisy and destabilizing updates, and the most effective supervision comes from stable confident errors where the model is reliably wrong yet curvature remains low. These findings motivate SAGE (Stability-Aware Gradient Efficiency), which maintains difficulty-stratified candidate pools refreshed during training and selects pairs within each pool by a forward-pass signal-to-curvature score. Only pairs with high current utility receive gradient computation; the rest are excluded from backpropagation. On mathematical reasoning benchmarks across multiple model scales, SAGE outperforms full-data and size-matched baselines while producing substantially smoother optimization trajectories.