Overview

Training data selection asks which examples, tokens, or data sources best help a model learn a target task, generalize, or retain existing capabilities. Methods include quality and difficulty filtering, diversity-aware coresets, gradient or influence scoring, active learning, and dynamic curricula or data mixtures. Researchers test these approaches in settings such as language-model pretraining and instruction tuning, vision, speech, and scientific learning, measuring both downstream performance and the cost of selecting and training on data. A central tension is that carefully chosen subsets can save resources or improve results, but selection may be expensive, depend on the model and data budget, or harm coverage and future learning; in some settings, random or full-data training remains competitive.

Latest papers

CardsList

196 papers11 papers in the last four weeks, down 67% on the four weeks before. 0.1% of all new papers.

  1. Which Past Matters? Training History and Multi-Forecast Horizon in Equity Transformers

    Nov 17, 2026Chariton ChalvatzisStock Price ForecastingTime Series Forecasting

  2. Ambiguous Examples Help, But Full Data Wins: Revisiting Dataset Cartography for Extractive Adversarial QA

    Nov 15, 2026Paul K. MandalTraining Data SelectionQuestion Answering

  3. Data Attribution Should Steer Pretraining

    Nov 15, 2026Glenn Matlin, Chandreyi Chakraborty, Mohammed Rehan Parwani +8Training Data SelectionTraining Data Curation

  4. Systematic Evaluation of Training Data Generation Strategies for Machine Learning-Based Molecular and Materials Property Prediction

    Nov 12, 2026Minh Nghia Nguyen, Beate Paulus, Akhil S. NairTraining Data SelectionActive Learning

  5. VWEN: A Curated Small Language Model for Vietnamese Paraphrasing

    Nov 11, 2026Sang Quang NguyenLanguage Model Post-TrainingText Generation

  6. Drowning in Noise: On Signal Saliency in Self-Supervised Learning for Ocean Acoustics

    Nov 4, 2026Shikhar Bharadwaj, Samuele Cornell, Shinji Watanabe +1Training Data SelectionBioacoustics

  7. From Weeks to Hours: Fast and Principled SFT Data Generation for Reasoning LLMs

    Oct 31, 2026Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari +2Training Data SelectionSynthetic Data Generation for Language Models

  8. OT-CAL: Optimal-Transport-Guided Adaptive Sample Selection for Efficient Visual Learning

    Oct 30, 2026Suryansh Talukdar, Jin Soung YooTraining Data SelectionData-Efficient Learning

  9. When Does Data Value Reduce to Class Balance? A Coverage View of Per-Point Data Valuation

    Oct 26, 2026Yingjie Bai, Weiming ZhiData SelectionTraining Data Selection

  10. Exact Unlearning After Data Curation

    Oct 24, 2026Priyank Jayraj, Devanshu sahoo, Uday MittalMachine UnlearningTraining Data Curation

  11. Towards Faithful Data Selection: Statistical Limits and Scalable Algorithms

    Oct 21, 2026Ahmed Mehdi Inane, Mahdi Haghifam, Gintare Karolina Dziugaite +1Training Data SelectionStatistical Learning Theory

  12. ICLHerd: Herding-Based Data Selection for LLMs with In-Context Learning as a Proxy

    Oct 19, 2026Yuqiao Wen, Lili MouTraining Data SelectionData Selection

  13. Hidden Cartographer: Hidden State Guided Data Selection for Dual Budgeted Instruction Tuning

    Oct 18, 2026Zixuan Qin, Qi Qin, Erbo Li +3Training Data SelectionInstruction-Tuning Data Selection

  14. Not All Unlabeled Data Are Equal: Unlabeled Data Selection for Semi-Supervised QSAR Modeling with Large Chemical Repositories

    Oct 15, 2026Oliver Majer, Saso Dzeroski, Jurica LevaticTraining Data SelectionSemi-Supervised Learning

  15. Efficient Pairwise Difference Learning: A Comparative Study of Encoding Strategies and Training Pair Selection

    Oct 15, 2026Bernard Georges, Mohamed Karim Belaid, Manuel RodriguesData-Efficient LearningTraining Data Selection

  16. Mind the Tail: Data Selection Through Sparse Concepts

    Oct 14, 2026Ziheng Zhang, Neha Hulkund, Colin Lee +7Training Data SelectionData Selection

  17. Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning

    Oct 9, 2026Sida Li, Yuanxin Liu, Shicheng Li +4AI Coding AgentsTraining Data Selection

  18. Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training

    Oct 9, 2026Suorong Yang, Jiaxin Wang, Jiaming Su +4Efficient Language Model ReasoningEfficient Language Model Training

  19. DataSense-Bench: The First Step Toward an AI Scientist

    Oct 8, 2026Yudi Zhang, Mingyu Cao, Lu Yin +2Training Data SelectionAI Agent Benchmarks

  20. MAP4CS: A Multi-dimensional Data Pruning Framework for Efficient Code Retriever Fine-tuning

    Oct 8, 2026Yuxuan Chen, Mingwei Liu, Guangsheng Ou +4Code SearchTraining Data Selection

  21. CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolution

    Oct 7, 2026Jixuan Chen, Jiaxin Zhang, Qinyuan Ye +11Terminal AgentsLLM Agent Harnesses

  22. Dataset Pruning from First Principles: A Label-Free Linear Programming Approach

    Oct 7, 2026Rodrigo Schuller, Francisco GanacimDataset PruningTraining Data Selection

  23. A Fine-Grained Analysis of the LoRA Fine-Tuning Landscape with Implications for Data Selection

    Oct 5, 2026Bowen Zhang, Changrui Fang, Xinsong Ma +2Low-Rank AdaptationTraining Data Selection

  24. LESSER: Post-Training Data Selection with Output-Layer Gradients

    Oct 2, 2026Lyuxin David Zhang, Eric Wong, Surbhi Goel +1Training Data SelectionData Selection for Language Model Fine-Tuning

  25. Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)

    Oct 1, 2026Zilin Du, Bowen Yang, Boyang Albert LiTraining Data Selection