Selective Fine-Tuning

Ask about this topic

Overview

Selective fine-tuning studies how to adapt pretrained models efficiently while preserving useful capabilities and reducing compute, memory, or privacy costs. Approaches rank layers and parameters using gradients, Fisher information, activations, routing, or sensitivity, and select training data or tokens using utility, influence, or alignment signals. These methods are used for language-model adaptation, safety and unlearning, continual learning, vision, and other domains, with comparisons often measuring task accuracy alongside retention, forgetting, and resource use. A central tension is that importance does not always mean a component can be trained effectively in isolation, and choices that help a current task may harm general capabilities or future learning.

Latest papers

CardsList

43 papers2 papers in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. Continual Safety Alignment via Gradient-Based Sample Selection

    Apr 19, 2026Thong Bach, Dung Nguyen, Thao Minh Le +1Continual Learning for LLMsLLM Fine-Tuning

    Applied Artificial Intelligence Initiative (A2I2), Deakin University · Pennsylvania State University

  2. Efficient Task Adaptation in Large Language Models via Selective Parameter Optimization

    Apr 18, 2026Weijie Wan, Jiangjiang ZhaoContinual Learning for LLMsLLM Fine-Tuning

    Shortest Path Technology · China Mobile Communications Group Co.,Ltd

  3. Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning

    Apr 15, 2026Zekai Lin, Chao Xue, Di Liang +8Supervised Fine-TuningSelective Fine-Tuning

    Tencent Hunyuan · University of New South Wales · Tencent Yuanbao +2

  4. A Layer-wise Analysis of Supervised Fine-Tuning

    Apr 12, 2026Qinghua Zhao, Xueling Gong, Xinyu Chen +2Supervised Fine-TuningFine-Tuning

    Hefei University · Lanzhou University

  5. MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning

    Mar 25, 2026Andrea ManzoniMixture-of-Experts Fine-TuningFine-Tuning

  6. Privacy-Preserving Federated Action Recognition via Differentially Private Selective Tuning and Efficient Communication

    Mar 22, 2026Idris Zakariyya, Pai Chet Ng, Kaushik Bhargav Sivangi +3Differentially Private Federated LearningVideo Action Recognition

    School of Computing Science, University of Glasgow, G12 8QQ, Glasgow, UK · Infocomm Technology Cluster, Singapore Institute of Technology, Singapore · The Edward S. Rogers Sr. Dept. of Electrical and Computer Engineering, University of Toronto, Canada

  7. Target Concept Tuning Improves Extreme Weather Forecasting

    Mar 17, 2026Shijie Ren, Xinyue Gu, Ziheng Peng +6Extreme Weather ForecastingWeather Forecasting

    Alibaba Group, Hangzhou, China

  8. Selective Fine-Tuning of GPT Architectures for Parameter-Efficient Clinical Text Classification

    Mar 15, 2026Fariba Afrin Irany, Sampson AkwafuoSelective Fine-TuningText Classification

    Clarksville, TN, USA · California State University Fullerton, California, USA

  9. GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection

    Mar 10, 2026Kai Yao, Zhenghan Song, Kaixin Wu +5Sparse Fine-TuningSelective Fine-Tuning

    1Ant Group · 2Cornell University · University of Liverpool +2

  10. NeST: Neuron Selective Tuning for LLM Safety

    Feb 18, 2026Sasha Behrouzi, Lichao Wu, Mohamadreza Rostami +1LLM Safety AlignmentSafety-Preserving Fine-Tuning of Language Models

    Technical University of Darmstadt · University of Bristol

  11. Selective Fine-Tuning for Targeted and Robust Concept Unlearning

    Feb 8, 2026Mansi, Avinash Kori, Francesca Toni +1Fine-TuningAdversarial Robustness

    Department of Computing Imperial College London, United Kingdom

  12. Beyond Transfer Accuracy: Mechanism-Guided Controlled Adaptation for Low-Resource Languages

    Jan 13, 2026Khumaisa Nur'aini, Ayu Purwarianti, Alham Fikri Aji +1Selective Fine-TuningCircuit Discovery

    Monash University Indonesia · MBZUAI · Institute Teknologi Bandung +1

  13. APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation

    May 26, 2025Javier MarínLLM Fine-TuningSelective Fine-Tuning