LLM Fine-Tuning

LLM: Large Language Model

LLM fine-tuning studies how additional training can adapt a pretrained model to new tasks, specialized knowledge, languages, or desired behaviors. Common approaches include supervised fine-tuning on curated or synthetic examples, preference optimization and reinforcement learning, and parameter-efficient methods such as LoRA and QLoRA that update only a small part of the model. Researchers also develop data-selection methods and selective layer or neuron updates to reduce training cost, and evaluate results on settings such as reasoning, code generation, multilingual tasks, and domain-specific question answering. A central challenge is gaining reliable task performance without losing general abilities or safety; results depend on the data, model, and evaluation, and retrieval can sometimes be a better alternative for adding knowledge.

Latest papers

758 papers76 papers in the last four weeks, up 15% on the four weeks before. 0.4% of all new papers.

New Hybrid Fine-Tuning Paradigm for LLMs: Algorithm Design and Convergence Analysis Framework
Apr 10, 2026Shaocong Ma, Peiran Yu, Heng HuangFine-Tuning LLM Fine-Tuning
Department of Computer Science University of Maryland College Park, MD 20742, USA · Department of Computer Science and Engineering University of Texas Arlington Arlington, TX 76019, USA
Pioneer Agent: Continual Improvement of Small Language Models in Production
Apr 10, 2026Dhruv Atreja, Julia White, Nikhil Nayak +5LLM Agent Training LLM Fine-Tuning
Fastino Labs
Automated Instruction Revision (AIR): A Structured Comparison of Task Adaptation Strategies for LLM
Apr 10, 2026Solomiia Bilyk, Volodymyr Getmanskyi, Taras FirmanLLM Fine-Tuning Prompt Optimization
Eleks Ltd. · Ukrainian Catholic University, Lviv, Ukraine
OASIS: Online Activation Subspace Learning for Memory-Efficient Training
Apr 10, 2026Sakshi Choudhary, Utkarsh Saxena, Kaushik RoyMemory-Efficient Training Memory-Efficient Fine-Tuning
Purdue University · Advanced Micro Devices, Inc. (AMD)
The nextAI Solution to the NeurIPS 2023 LLM Efficiency Challenge
Apr 10, 2026Gyuwon Park, DongIl Shin, SolGil Oh +2LLM Fine-Tuning Efficient Language Model Training
Department of Computer Science and Engineering, UNIST · CJ Corporation
Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning
Apr 10, 2026Lorenzo Jaime Yu Flores, Cesare Spinoso Di-Piano, Jackie Chi Kit CheungConfidence Estimation in Language Models Language Model Calibration
Mila - Quebec AI Institute · McGill University · Canada CIFAR AI Chair, Mila
Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
Apr 10, 2026Taojie Zhu, Dongyang Xu, Ding Zou +4LLM Fine-Tuning RL for Language Model Reasoning
Shenzhen International Graduate School, Tsinghua University · Intelligent System Department, Zhongxing Telecom Equipment (ZTE) · Institute of Advanced Interdisciplinary Studies, Chongqing University of Posts and Telecommunications
Training Language Models for Bilateral Trade with Private Information
Apr 10, 2026Dirk Bergemann, Soheil Ghili, Xinyang Hu +2Multi-Agent Negotiation LLM Agent Training
Department of Economics, Yale University · Yale School of Management, Yale University · Department of Statistics and Data Science, Yale University +1
RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment
Apr 10, 2026Jaber Jaber, Osama JaberLanguage Model Vocabulary Expansion Arabic Language Models
RightNow AI
Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains
Apr 9, 2026Haokai Ma, Lee Yan Zhen, Gang Yang +4RL for Language Models Reinforcement Learning
National University of Singapore, Singapore · Singapore Management University, Singapore
Synthetic Data for any Differentiable Target
Apr 9, 2026Tristan Thrush, Sung Min Park, Herman Brunborg +5Synthetic Data Generation for Language Models RL for Generative Models
Stanford University
Soro: A Lightweight Foundation Model and Chatbot for Tajik
Apr 9, 2026Stanislav Liashkov, Haitz Sáez de Ocáriz Borde, Azizjon Azimi +3Low-Resource Language Processing LLM Fine-Tuning
Data Selection for Multi-turn Dialogue Instruction Tuning
Apr 9, 2026Bo Li, Shikun Zhang, Wei YeInstruction-Tuning Data Selection Data Selection
National Engineering Research Center for Software Engineering, Peking University · School of Computer Science, Peking University · PKU-CMCC(Hubei) Joint Research Lab for LLM Industrial Applications
GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
Apr 9, 2026Kaiyuan Tian, Yu Tang, Gongqingjian Jiang +5LLM Fine-Tuning Fine-Tuning
National University of Defense Technology · Information Support Force Engineering University
The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training
Apr 9, 2026Rui Zhang, Hongwei Li, Yun Shen +6LLM Alignment LLM Safety Alignment
University of Electronic Science and Technology of China · Flexera · CISPA Helmholtz Center for Information Security +1
MARS: Enabling Autoregressive Models Multi-Token Generation
Apr 8, 2026Ziqi Jin, Lei Wang, Ziwei Luo +1Multi-Token Prediction LLM Inference Acceleration
Nanyang Technological University · Singapore Management University · Uppsala University
An empirical study of LoRA-based fine-tuning of large language models for automated test case generation
Apr 8, 2026Milad Moradi, Ke Yan, David Colwell +1Automated Test Generation LLM Fine-Tuning
AI Research Lab, Tricentis, Vienna, Austria · AI Research Lab, Tricentis, Sydney, Australia
Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation
Apr 8, 2026Jeongho Yoon, Chanhee Park, Yongchan Chun +2Privacy-Preserving Language Models Privacy-Preserving ML
Department of Computer Science and Engineering, Korea University · Samsung Mobile eXperience Business
Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability
Apr 8, 2026Qihan Ren, Peng Wang, Ruikun Cai +8LLM Fine-Tuning Cross-Domain Generalization
Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · University of Science and Technology of China
State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation
Apr 7, 2026Navan Preet Singh, Anurag Garikipati, Ahmed Abulkhair +6Arabic Language Models Arabic NLP
Forta, Houston, TX · Incept Labs, Houston, TX · Titan Holdings, San Francisco, CA
Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised Fine-Tuning
Apr 7, 2026Navan Preet Singh, Xiaokun Wang, Anurag Garikipati +3LLM Fine-Tuning Language Modeling
Forta, Houston, TX · East China Normal University, Shanghai, China · Incept Labs, Houston, TX +1
A Severity-Based Curriculum Learning Strategy for Arabic Medical Text Generation
Apr 7, 2026Ahmed Alansary, Molham Mohamed, Ali HamdiText Generation Arabic NLP
October University for Modern Sciences & Arts, Egypt
Severity-Aware Weighted Loss for Arabic Medical Text Generation
Apr 7, 2026Ahmed Alansary, Molham Mohamed, Ali HamdiMedical LLMs Text Generation
October University for Modern Sciences & Arts, Egypt
Controlling Distributional Bias in Multi-Round LLM Generation via KL-Optimized Fine-Tuning
Apr 7, 2026Yanbei Jiang, Amr Keleg, Ryandito Diandaru +4LLM Alignment LLM Fine-Tuning
University of Melbourne · Mohamed bin Zayed University of Artificial Intelligence · Oracle
ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloads
Apr 7, 2026Jingwei Zuo, Xinze Feng, Zien Liu +5LLM Fine-Tuning Low-Rank Adaptation
Rice University · Independent Researcher
Data-Driven Function Calling Improvements in Large Language Model for Online Financial QA
Apr 7, 2026Xing Tang, Hao Chen, Shiwei Li +7LLM Tool Use Financial QA
Shenzhen Technology University Shenzhen, China · FiT, Tencent Shenzhen, China · Huazhong University of Science and Technology Wuhan, China +3
Instruction-Tuned LLMs for Parsing and Mining Unstructured Logs on Leadership HPC Systems
Apr 6, 2026Ahmad Maroof Karimi, Jong Youl Choi, Charles Qing Cao +1Document Parsing LLM Information Extraction
Oak Ridge National Laboratory, Oak Ridge, USA · University of Tennessee, Knoxville, USA
FLeX: Fourier-based Low-rank EXpansion for multilingual transfer
Apr 6, 2026Gaurav NarasimhanLLM Fine-Tuning Multilingual Code Generation
Department of Computer Science, Stanford University
Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning
Apr 6, 2026Miao Li, Irina Saparina, Alexander Gurung +1Long-Context Reasoning in Language Models LLM Fine-Tuning
School of Informatics, University of Edinburgh
OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning
Apr 6, 2026Jingkai He, Pengfei Chen, Chenghui Wu +7LLM Post-Training LLM Fine-Tuning
School of Systems Science and Engineering, Sun Yat-sen University, Guangzhou, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Alibaba Cloud Computing, Hangzhou, China

Apr 10, 2026J/K move · Enter open · S save

New Hybrid Fine-Tuning Paradigm for LLMs: Algorithm Design and Convergence Analysis Framework

Shaocong Ma, Peiran Yu, Heng Huang

Department of Computer Science University of Maryland College Park, MD 20742, USA · Department of Computer Science and Engineering University of Texas Arlington Arlington, TX 76019, USA

Fine-Tuning LLM Fine-Tuning Parameter-Efficient Fine-Tuning Stochastic Gradient Descent

Open paper AskPDF arXiv

Fine-tuning Large Language Models (LLMs) typically involves either full fine-tuning, which updates all model parameters, or Parameter-Efficient Fine-Tuning (PEFT), which adjusts a small subset of parameters. However, both approaches have inherent limitations: full fine-tuning is computationally expensive, while PEFT often struggles to learn new knowledge and exhibits suboptimal performance. To overcome these issues, we propose a novel hybrid fine-tuning approach that jointly updates both LLMs and PEFT modules using a combination of zeroth-order and first-order optimization methods. To analyze our new algorithm, we develop a theoretical framework centered on the concept of hybrid smoothness condition, which accounts for the heterogeneous nature of the optimization landscape in joint LLM and PEFT training. We derive a rigorous convergence analysis for the convergence of reshuffling-type SGD algorithm under multiple learning rates and demonstrate its effectiveness through extensive empirical studies across various downstream tasks and model architectures. On the practical side, our results demonstrate consistent performance improvement, making the approach a viable solution for large-scale language model fine-tuning.