Data Selection for Language Model Fine-Tuning

Latest papers

CardsList

32 papers8 papers in the last four weeks, up 33% on the four weeks before. 0.1% of all new papers.

  1. Safety-Preserving Medical LLM Fine-Tuning via Bilevel Data Selection

    Nov 16, 2026Ruoxuan Shi, Sheng Yang, Zhengxing Su +2Safety-Preserving Fine-Tuning of Language ModelsData Selection for Language Model Fine-Tuning

  2. From Task Affinities to Optimal Allocations: Probabilistic Graphical Modelling for Supervised Finetuning

    Nov 7, 2026Prateek Chanda, Saral Sureka, Krishnateja Killamsetty +1Data Selection for Language Model Fine-TuningSupervised Fine-Tuning

  3. Rethinking Data Selection: The Importance of Coverage over Difficulty in LLM Fine-Tuning

    Nov 5, 2026Lalchand Pandia, Kanishka Misra, Allyson EttingerData SelectionData Selection for Language Model Fine-Tuning

  4. Follow the Features: SAE Feature-Guided Data-Efficient Instruction-Tuning of Large Language Models

    Nov 2, 2026Ryan Swift, Konstantinos PsounisData Selection for Language Model Fine-TuningInstruction-Tuning Data Selection

  5. Online Label-Free Gradient Matching for Continual Language Model Training

    Nov 2, 2026Sajjad Pakdamansavoji, Mohammad Reza Rezaei, Ilija Subasic +3Continual Learning for LLMsDataset Distillation

  6. Perturbation-Based Data Selection for Medical Question Answering

    Oct 24, 2026Yuexin Wu, Dayou Yu, Vasile RusData SelectionData Selection for Language Model Fine-Tuning

  7. ICLHerd: Herding-Based Data Selection for LLMs with In-Context Learning as a Proxy

    Oct 19, 2026Yuqiao Wen, Lili MouTraining Data SelectionData Selection

  8. NADS: Navigator-Guided Data Selection for Mitigating Catastrophic Forgetting in Fine-Tuning

    Oct 19, 2026Jianhao Zhang, Ou Wu, Yi DuLLM Fine-TuningData Selection for Language Model Fine-Tuning

  9. DeGMatch: Target-Aware Reasoning Data Selection via Deconfounded Gradient Matching

    Oct 18, 2026Changho Lee, Janghoon Han, Rajdeep Mukherjee +4Data Selection for Language Model Fine-TuningLLM Reasoning

  10. Which and When to Admit: Gradient Admission for Data-Centric Small Language Model Finetuning

    Oct 6, 2026Hongyu Cao, Yanchi Liu, Kunpeng Liu +5Gradient InterferenceLow-Rank Adaptation

  11. LESSER: Post-Training Data Selection with Output-Layer Gradients

    Oct 2, 2026Lyuxin David Zhang, Eric Wong, Surbhi Goel +1Training Data SelectionData Selection for Language Model Fine-Tuning

  12. Pick the Mixture, Not the Samples: Combinatorial Data Selection for LLMs

    Sep 30, 2026Luyang Zhang, Jialu Wang, Bebei LiData SelectionData Selection for Language Model Fine-Tuning

  13. Every Batch Is Its Own Validation Set: Leave-One-Out Gradient Matching for Online Data Selection in LLM Fine-Tuning

    Sep 30, 2026Hongyu Chen, Xinyi Luo, Ming Zhao +6LLM Fine-TuningData Selection for Language Model Fine-Tuning

  14. A helps B while B hurts A: directed transfer in instruction-tuning mixture

    Sep 30, 2026Nima H. Siboni, Vahid RostamiLLM Fine-TuningInstruction-Tuning Data Selection

  15. PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning

    Jul 20, 2026Hang Zhang, Warren J. GrossLLM Fine-TuningEfficient Language Model Training

  16. HARP: Efficient Data Selection for Finetuning Large Language Models

    Jun 5, 2026Ning Wang, Zhengxin Zhang, Maosen Tang +3LLM Fine-TuningActive Learning

  17. Two to Tango: Coupled Task-Reference Selection for Safe LLM Fine-tuning

    Jun 1, 2026Xinrui Chen, Jianhao Zhang, Ou Wu +1LLM Fine-TuningData Selection

  18. The Long-Term Effects of Data Selection in LLM Fine-Tuning

    May 28, 2026Yuxin Yang, Aoxiong Zeng, Xiangquan YangContinual Learning for LLMsLLM Fine-Tuning

  19. From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning

    May 13, 2026Haodong Wu, Jiahao Zhang, Lijie Hu +1Supervised Fine-TuningTraining Data Curation

  20. One Algorithm, Two Goals: Dual Scoring for Parameter and Data Selection in LLM Fine-Tuning

    May 7, 2026Xinrui Chen, Liu Yang, Ou WuSupervised Fine-TuningLLM Fine-Tuning

  21. From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models

    Apr 28, 2026Ling Shi, Xinwei Wu, Xiaohu Zhao +7LLM Fine-TuningData Selection for Language Model Fine-Tuning

  22. CRAFT: Clustered Regression for Adaptive Filtering of Training data

    Apr 24, 2026Parthasarathi Panda, Asheswari Swain, Subhrakanta PandaData SelectionTraining Data Selection

  23. Filter-then-Weight: Online Data Selection and Reweighting for LLM Fine-Tuning

    Mar 8, 2026Fangxin Wang, Peyman Baghershahi, Langzhou He +3Data Selection for Language Model Fine-TuningTraining Data Selection

  24. Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models

    Mar 1, 2026Adel Javanmard, Baharan Mirzasoleiman, Vahab MirrokniData Selection for Language Model Fine-TuningLLM Fine-Tuning

  25. Entropy-Based Data Selection for Language Models

    Feb 19, 2026Hongming Li, Yang Liu, Chao HuangData Selection for Language Model Fine-TuningTraining Data Selection

  26. A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)

    Feb 16, 2026Nihal V. Nayak, Paula Rodriguez-Diaz, Neha Hulkund +2Instruction-Tuning Data SelectionTraining Data Selection

  27. Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations

    Feb 9, 2026Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong +6Bayesian OptimizationLLM Fine-Tuning