LLM Fine-Tuning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 83% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 509

All topics

Efficient Task Adaptation in Large Language Models via Selective Parameter Optimization
Apr 18, 2026Weijie Wan, Jiangjiang ZhaoContinual Learning for LLMs LLM Fine-Tuning
Shortest Path Technology · China Mobile Communications Group Co.,Ltd
Learning to Trade Like an Expert: Cognitive Fine-Tuning for Stable Financial Reasoning in Language Models
Apr 18, 2026Yuchen Pan, Soung Chang LiewLLM Evaluation LLM Fine-Tuning
Department of Information Engineering, The Chinese University of Hong Kong
Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs
Apr 17, 2026Jaechul Roh, Virat Shejwalkar, Amir HoumansadrLanguage Model Safety Evaluation LLM Fine-Tuning
University of Massachusetts Amherst
LLM attribution analysis across different fine-tuning strategies and model scales for automated code compliance
Apr 16, 2026Jack Wei Lun Shi, Minghao Dang, Wawan Solihin +1Feature Attribution LLM Interpretability
Department of Civil and Environmental Engineering, National University of Singapore, 117576, Singapore · School of Transportation Science and Engineering, Harbin Institute of Technology, Harbin 150090, China · Research and Innovation, NovaCITYNETS Pte.Ltd., 348615, Singapore
Think Multilingual, Not Harder: A Data-Efficient Framework for Teaching Reasoning Models to Code-Switch
Apr 16, 2026Eleanor M. Lin, David JurgensMultilingual Language Models Code-Switching
University of Michigan
Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation
Apr 16, 2026Mu-Chi Chen, Po-Hsuan Huang, Yu-Hung Kao +6LLM Fine-Tuning Code Generation
Academia Sinica, Taipei, Taiwan · National Taiwan University, Taipei, Taiwan · National Cheng Kung University, Tainan, Taiwan
Fact4ac at the Financial Misinformation Detection Challenge Task: Reference-Free Financial Misinformation Detection via Fine-Tuning and Few-Shot Prompting of Large Language Models
Apr 16, 2026Cuong Hoang, Le-Minh NguyenLLM Fine-Tuning Automated Fact-Checking
Japan Advanced Institute of Science and Technology
Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains
Apr 9, 2026Haokai Ma, Lee Yan Zhen, Gang Yang +4RL for Language Models Reinforcement Learning
National University of Singapore, Singapore · Singapore Management University, Singapore
Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models
Apr 2, 2026Jae O Lee, Jaemin Kim, Sumyeong Ahn +1Retrieval-Augmented Generation Fine-Tuning
1Hanyang University · 2KENTECH
Adam's Law: Textual Frequency Law on Large Language Models
Apr 2, 2026Hongyuan Adam Lu, Z. L., Victor Wei +5LLM Prompting LLM Fine-Tuning
FaceMind Corporation · The Chinese University of Hong Kong
Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness
Apr 2, 2026Deeptanshu Malu, Deevyanshu Malu, Aditya Nemiwal +1In-Context Example Selection LLM Fine-Tuning
IIT Bombay
Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models
Mar 21, 2026Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg +1LLM Alignment Memorization in Language Models
Stony Brook University · Carnegie Mellon University · Columbia Law School
An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU
Mar 17, 2026Ruijia Yang, Zeyi WenFine-Tuning Memory-Efficient Fine-Tuning
Aligning the Query Space: Greedy Information Projection for Language Model Data Selection
Mar 14, 2026Victor Ye Dong, Kuan-Yun Lee, Jiamei Shuai +3LLM Fine-Tuning Instruction-Tuning Data Selection
Microsoft
TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning
Mar 3, 2026Christian Greisinger, Steffen EgerLLM Fine-Tuning
University of Technology Nuremberg
SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs
Feb 20, 2026Jihun Kim, Dongyeop Lee, Namhoon LeeCommunication-Efficient Distributed Training Zeroth-Order Optimization
POSTECH, pohang, Republic of Korea.
How to Train Your Long-Context Visual Document Model
Feb 16, 2026Austin VeselkaVisual Question Answering Vision-Language Models
LightOn
Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
Feb 16, 2026Yuchen Yang, Wenze Lin, Enhao Huang +6LLM Fine-Tuning Training Data Curation
1The State Key Laboratory of Blockchain and Data Security, Zhejiang University · 2Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · 3Qiuzhen College, Tsinghua University +2
The Appeal and Reality of Recycling LoRAs with Adaptive Merging
Feb 12, 2026Haokun Liu, Gyung Hyun Je, Marco Ciccone +3LLM Fine-Tuning Language Model Merging
University of Toronto, Canada · Vector Institute, Toronto, Canada · Mistral AI (work done before joining) +2
Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
Feb 11, 2026Jiacheng Wang, Zhijie Liu, Ping Jian +4Supervised Fine-Tuning Reinforcement Learning
Beijing Institute of Technology, Beijing, China · Tsinghua University, Beijing, China
The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Models
Feb 1, 2026Reem I. Masoud, Chen Feng, Shunta Asano +3LLM Fine-Tuning Cultural Alignment
University College London · King Abdulaziz University · Queen’s University Belfast +3
Distilling Token-Trained Models into Byte-Level Models
Feb 1, 2026Zishuo Bao, Jiaqi Leng, Junxiong Wang +2LLM Fine-Tuning Byte-Level Language Model
Fuzhou University · NYU Shanghai · Fudan University +2
Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
Jan 28, 2026Moule Lin, Shuhao Guan, Andrea Patane +2Bayesian Neural Networks Confidence Estimation in Language Models
School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland and Lero the Research Ireland Centre for Software, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland
Counterfactual Modeling with Fine-Tuned LLMs for Health Intervention Design and Sensor Data Augmentation
Jan 21, 2026Shovito Barua Soumma, Asiful Arefeen, Stephanie M. Carpenter +2Counterfactual Explanations LLM Fine-Tuning
College of Health Solutions, Arizona State University, Phoenix, AZ 85004, USA. · School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ 85281, USA. · School of Nutritional Sciences and Wellness, University of Arizona, Tucson, AZ 85721, USA.
Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding
Jan 8, 2026Sungmok Jung, Yeonkyoung So, Joonhak Lee +3LLM Evaluation LLM Fine-Tuning
Graduate School of Data Science, Seoul National University · Dept. of Computer Science and Engineering, Seoul National University
Shuttling Compiler for Trapped-Ion Quantum Computers Based on Fine-Tuned Large Language Models
Dec 19, 2025Fabian Kreppel, Reza Salkhordeh, Ferdinand Schmidt-Kaler +1Quantum Circuit Compilation LLM Fine-Tuning
Institute of Computer Science, Johannes Gutenberg University, Staudingerweg 9, 55128 Mainz, Germany · Institute of Physics, Johannes Gutenberg University, Staudingerweg 7, 55128 Mainz, Germany · Department of Computer Science, Saarland University, Saarland Informatics Campus, E1 1, 66123 Saarbrücken, Germany
Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning
Nov 26, 2025Zhenchao Tang, Fang Wang, Haohuai He +13LLM Alignment LLM Fine-Tuning
Tencent AI for Life Sciences Lab, Shenzhen, China. · Sun Yat-sen University, Shenzhen, China. · The Hong Kong Polytechnic University, Hong Kong SAR, China. +2
Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation
Nov 26, 2025Quan Xiao, Yutong Xuan, Gaowen Liu +2Supervised Fine-Tuning LLM Self-Refinement
Cornell University · Cisco Research
Fine-Tune, Then Rectify
Nov 23, 2025Zikun Ye, Jinglong Zhao, Lei WangLLM Fine-Tuning
Foster School of Business, University of Washington, Seattle, WA · Questrom School of Business, Boston University, Boston, MA
Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
Nov 8, 2025Yinjie Cheng, Paul Youssef, Christin Seifert +2Knowledge Editing LLM Fine-Tuning
School of Computer Science University of Sheffield Sheffield, UK · Mathematics and Computer Science Marburg University Marburg, Germany

Apr 18, 2026, cs.CLJ/K move · Enter open · S save

Efficient Task Adaptation in Large Language Models via Selective Parameter Optimization

Weijie Wan, Jiangjiang Zhao

Shortest Path Technology · China Mobile Communications Group Co.,Ltd

Continual Learning for LLMs LLM Fine-Tuning Domain Adaptation for LLMs Selective Fine-Tuning Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Large Language Models (LLMs) have demonstrated excellent performance in general language understanding, generation and other tasks. However, when fine-tuning for specific domain tasks, the general knowledge accumulated in the pre-training phase is often partially overwritten or forgotten due to parameter updates, which severely limits the generalization ability and transferability of LLMs. Traditional fine-tuning strategies mostly train on the entire parameter space, ignoring the heterogeneity of model parameters, that is, some parameters are extremely important for general tasks, while other parameters are more sensitive to specific tasks. To alleviate the above problems, this paper innovatively proposes a parameter element importance evaluation method, which divides parameters into "core parameters" and "non-core parameters" by distinguishing the importance of parameters for general language ability tasks and specific domain tasks, and fixes the core parameters during fine-tuning, and only fine-tunes the non-core parameters. Extensive experiments on scientific, medical and physical tasks using GPT-J and LLaMA-3 show that our method can mitigate catastrophic forgetting while enhancing the adaptability of the model.

Momentum

Related topics

Latest papers 509