Data Selection

Data selection studies which examples, sources, labels, or interactions a model should learn from, aiming to improve performance while reducing training or annotation costs. Common approaches include coreset and diversity methods, uncertainty or active learning, gradient and influence scoring, distribution matching, and adaptive sampling policies. These methods are used in settings such as image classification, LLM instruction and preference tuning, multimodal training, and clinical labeling. Selected subsets can match or outperform full-data training, but usefulness depends on the model, target task, and budget; a key challenge is balancing short-term gains with coverage of rare cases, robustness, and long-term learning.