LLM Fine-Tuning

LLM: Large Language Model

LLM fine-tuning studies how additional training can adapt a pretrained model to new tasks, specialized knowledge, languages, or desired behaviors. Common approaches include supervised fine-tuning on curated or synthetic examples, preference optimization and reinforcement learning, and parameter-efficient methods such as LoRA and QLoRA that update only a small part of the model. Researchers also develop data-selection methods and selective layer or neuron updates to reduce training cost, and evaluate results on settings such as reasoning, code generation, multilingual tasks, and domain-specific question answering. A central challenge is gaining reliable task performance without losing general abilities or safety; results depend on the data, model, and evaluation, and retrieval can sometimes be a better alternative for adding knowledge.

Latest papers

758 papers76 papers in the last four weeks, up 15% on the four weeks before. 0.4% of all new papers.

Boosting Large Language Models with Mask Fine-Tuning
Mar 27, 2025Mingyuan Zhang, Yue Bai, Huan Wang +4LLM Fine-Tuning
College of Engineering, Northeastern University · Khoury College of Computer Science, Northeastern University
Distributionally Robust Reinforcement Learning with Human Feedback
Mar 1, 2025Debmalya Mandal, Paulius Sasnauskas, Goran RadanovicDirect Preference Optimization LLM Fine-Tuning
Dept. of Computer Science University of Warwick, UK · Department of Computing Science University of Alberta, Edmonton, Canada · Max-Planck Institute for Software Systems, Germany
LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuning
Feb 20, 2025Yansheng Mao, Yufei Xu, Jiaqi Li +5Long-Context QA LLM Fine-Tuning
Institute for Artificial Intelligence, Peking University · School of Electronics Engineering and Computer Science, Peking University · State Key Laboratory of General Artificial Intelligence, BIGAI
WAFFLE: Finetuning Multi-Modal Models for Automated Front-End Development
Oct 24, 2024Shanchao Liang, Nan Jiang, Shangshu Qian +1Image-to-Code Generation Web Application Generation
Purdue University
Tuning Language Models by Mixture-of-Depths Ensemble
Oct 16, 2024Haoyan Luo, Lucia SpeciaLLM Fine-Tuning Language Model Ensembles
University of Cambridge · Imperial College London
Mitigating Memorization In Language Models
Oct 3, 2024Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6Memorization in Language Models LLM Fine-Tuning
University of Chicago · Argonne National Laboratory · Lawrence Berkeley National Laboratory +3
Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian
May 22, 2024Aleksandr Nikolich, Konstantin Korolev, Sergei Bratchikov +2Language Model Pretraining Multilingual Language Models
ITMO University · HSE University · Vikhr +2
Amortizing intractable inference in large language models
Oct 6, 2023Edward J. Hu, Moksh Jain, Eric Elmoznino +4Amortized Inference LLM Fine-Tuning
Mila – Quebec AI Institute, Universit´e de Montr´eal · University of Oxford

Mar 27, 2025J/K move · Enter open · S save

Boosting Large Language Models with Mask Fine-Tuning

Mingyuan Zhang, Yue Bai, Huan Wang, Yizhou Wang+3

College of Engineering, Northeastern University · Khoury College of Computer Science, Northeastern University

LLM Fine-Tuning

Open paper AskPDF arXiv

The large language model (LLM) is typically integrated into the mainstream optimization protocol. However, it remains underexplored whether maintaining the model integrity is indispensable for promising performance. In this work, we introduce Mask Fine-Tuning (MFT), a novel LLM fine-tuning paradigm demonstrating that carefully breaking the model's structural integrity can surprisingly improve performance without updating model weights. MFT learns and applies binary masks to well-optimized models, using the standard LLM fine-tuning objective as supervision. Based on fully fine-tuned models, MFT uses the same fine-tuning datasets to achieve consistent performance gains across domains and backbones (e.g., an average gain of 2.70/4.15 on IFEval with LLaMA2-7B/3.1-8B). Detailed ablation studies and analyses examine the proposed MFT from different perspectives, including the sparse ratio and the loss surface. Additionally, when deployed on well-trained models, MFT is compatible with other LLM optimization procedures to improve overall model performance. Furthermore, this study extends the masking operation beyond its conventional use in network pruning for model compression to encompass a broader range of model capabilities.