Instruction-Tuning Data Selection

Overview

The field asks which examples, conversations, or task mixtures best teach a model to follow instructions, reason, and handle specialized tasks. Methods range from quality and difficulty filters to embedding-based coverage, gradient or influence scores, model-activation signals, and adaptive selection guided by validation results. Researchers apply these approaches to text, code, dialogue, and multimodal instruction tuning, often testing whether small selected subsets can match or exceed full-data training on task benchmarks. The central tension is that selection depends on the model, task, and training budget, while costly scoring and search—and possible shifts in safety or other behaviors—make reliable evaluation difficult.

Latest papers

CardsList

48 papers4 papers in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. Follow the Features: SAE Feature-Guided Data-Efficient Instruction-Tuning of Large Language Models

    Nov 2, 2026Ryan Swift, Konstantinos PsounisData Selection for Language Model Fine-TuningInstruction-Tuning Data Selection

  2. Query-based Submodular Data Selection for Efficient Instruction Tuning

    Nov 2, 2026Zhenghao Zhao, Yasutomo Kawanishi, Huihuo Zheng +2Data SelectionInstruction-Tuning Data Selection

  3. Robust LLM Data Selection via Content-Invariant Gradient Attribution

    Oct 27, 2026Jia Gu, Liang Pang, Huawei ShenInstruction-Tuning Data SelectionGradient-Based Attribution

  4. "High-Quality" Data for Early-Stage Instruction Tuning: An Empirical Study of Data Quality Metrics

    Oct 25, 2026Vladimir Necula, David Gvadzabia, Sofia SerranoInstruction-Tuning Data SelectionInstruction Tuning

  5. Hidden Cartographer: Hidden State Guided Data Selection for Dual Budgeted Instruction Tuning

    Oct 18, 2026Zixuan Qin, Qi Qin, Erbo Li +3Training Data SelectionInstruction-Tuning Data Selection

  6. AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning

    Oct 15, 2026Bingxiang He, Pengqi Sun, Shihao Peng +18Supervised Fine-TuningInstruction-Tuning Data Selection

  7. Data Strategies for Large Language Models: A Unified Perspective on the Training Lifecycle

    Oct 15, 2026Yuanjian Xu, Jianing Hao, Tianze Sun +7LLM TrainingData Mixture Optimization

  8. CLIVO: Differentiable Cluster-Based Instruction Mixing with Learned Validation Objectives

    Oct 12, 2026Yuhan Wang, Rui Hu, Batian Di +5Semantic ClusteringInstruction-Tuning Data Selection

  9. LESSER: Post-Training Data Selection with Output-Layer Gradients

    Oct 2, 2026Lyuxin David Zhang, Eric Wong, Surbhi Goel +1Training Data SelectionData Selection for Language Model Fine-Tuning

  10. Pick the Mixture, Not the Samples: Combinatorial Data Selection for LLMs

    Sep 30, 2026Luyang Zhang, Jialu Wang, Bebei LiData SelectionData Selection for Language Model Fine-Tuning

  11. A helps B while B hurts A: directed transfer in instruction-tuning mixture

    Sep 30, 2026Nima H. Siboni, Vahid RostamiLLM Fine-TuningInstruction-Tuning Data Selection

  12. How code helps different tasks? A decompositional lens on LLM post-training

    Sep 27, 2026Zheng Yu, Yiwei Li, Yishen Chen +4LLM Fine-TuningInstruction-Tuning Data Selection

  13. Model-Aware Data Selection from In-and-Out Information Interplay

    Sep 26, 2026Yifan Wang, Xiaomin Li, Yuexing Hao +7Training Data SelectionInstruction-Tuning Data Selection

  14. LIMIT: Less Is More for Instruction Tuning in Text-to-SQL

    Sep 21, 2026Haoyuan Ma, Hengwei Liu, Linjuan Wu +2Text-to-SQLInstruction-Tuning Data Selection

  15. Token Utility Is Selection-Conditioned: Coupled Selection of Prompt Context and Response Supervision for Instruction Tuning

    Sep 19, 2026Can Wu, Xinrui Chen, Ou Wu +1LLM Fine-TuningInstruction-Tuning Data Selection

  16. DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection

    Aug 31, 2026Yuyang Hong, Jinhui Guo, Jiaqi Gu +6Instruction-Tuning Data SelectionTraining Data Selection

  17. SWE-Prime: Fewer Trajectories, Better Performance

    Aug 27, 2026Dewu Zheng, Ruizhe Ye, Yanlin Wang +7Instruction-Tuning Data SelectionSoftware Engineering Agents

  18. TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

    Aug 26, 2026Sadhika Malladi, Samy Jelassi, Dylan Foster +2Supervised Fine-TuningInstruction-Tuning Data Selection

  19. ADE: Agentic Data Evolution Framework for Human-Centered Objectives

    Aug 24, 2026Yang Yu, Yilin Jiang, Zexuan Fei +6LLM AlignmentSynthetic Data Selection

  20. Capacity-Dependent Effects of Data Selection for Reasoning

    Aug 13, 2026Cuong Dang, Hoang Anh Just, Ruoxi JiaInstruction-Tuning Data SelectionSupervised Fine-Tuning

  21. Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent

    Aug 11, 2026Fanqi Zhou, Qiaosheng Chen, Zixian Huang +1Training Data CurationInstruction-Tuning Data Selection

  22. Online Data Selection Is Implicit Alignment

    Jul 8, 2026Aoxiong Zeng, Yuxin Yang, Xiangquan YangSupervised Fine-TuningLLM Alignment

  23. MADS: Model-Aware Diverse Core Set Selection for Instruction Tuning

    May 29, 2026Yi Bai, Wenhao Zhang, Yao Chen +3Instruction-Tuning Data SelectionTraining Data Selection

  24. OnceSelect: Reusable Data Selection for Efficient Multimodal Instruction Tuning

    May 26, 2026Mingkang Dong, Muxin Pu, Hongyi Cai +5Instruction-Tuning Data SelectionMultimodal Large Language Models