LLM Fine-Tuning

LLM: Large Language Model

LLM fine-tuning studies how additional training can adapt a pretrained model to new tasks, specialized knowledge, languages, or desired behaviors. Common approaches include supervised fine-tuning on curated or synthetic examples, preference optimization and reinforcement learning, and parameter-efficient methods such as LoRA and QLoRA that update only a small part of the model. Researchers also develop data-selection methods and selective layer or neuron updates to reduce training cost, and evaluate results on settings such as reasoning, code generation, multilingual tasks, and domain-specific question answering. A central challenge is gaining reliable task performance without losing general abilities or safety; results depend on the data, model, and evaluation, and retrieval can sometimes be a better alternative for adding knowledge.

Latest papers

758 papers76 papers in the last four weeks, up 15% on the four weeks before. 0.4% of all new papers.

Evaluation of Embedding-Based and Generative Methods for LLM-Driven Document Classification: Opportunities and Challenges
Apr 5, 2026Rong Lu, Hao Liu, Song HouEarth Science Text Classification
Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures
Apr 4, 2026Abdulmalek SaketLLM Fine-Tuning Efficient Language Model Training
Royal Fenice Kft / ALETHEIA PROTOCOL research, Budapest, Hungary
Resource-Conscious Modeling for Next- Day Discharge Prediction Using Clinical Notes
Apr 3, 2026Ha Na Cho, Sairam Sutari, Alexander Lopez +2Clinical Prediction Clinical Outcome Prediction
University of California, Irvine, Donald Bren School of Information, Department of Informatics · University of California, Irvine, Department of Neurosurgery
The limits of bio-molecular modeling with large language models : a cross-scale evaluation
Apr 3, 2026Yaxin Xu, Yue Zhou, Tianyu Zhao +2LLM Evaluation Large Language Model-Guided Scientific Discovery
Southern University of Science and Technology, Shenzhen, 518055, China. · Pengcheng Laboratory, Shenzhen, 518055, China. · Institute of Mechanics, Chinese Academy of Sciences, Beijing, 100190, China.
Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspaces
Apr 3, 2026Irene Tenison, Stella Ahn, Miriam Kim +2Memory-Efficient Fine-Tuning LLM Fine-Tuning
MIT CSAIL · Harvard SEAS
Domain Adaptation of Large Language Models for Polymer-Composite Additive Manufacturing Using Retrieval-Augmented Generation and Fine-Tuning
Apr 2, 2026Saiful Islam Sagor, Tania Haghighi, Minhaj Nur Alam +1Domain Adaptation for LLMs LLM Fine-Tuning
Department of Mechanical Engineering and Engineering Science, University of North Carolina at Charlotte, NC 28223, USA. · Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, NC 28223, USA.
Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models
Apr 2, 2026Jae O Lee, Jaemin Kim, Sumyeong Ahn +1Retrieval-Augmented Generation Fine-Tuning
1Hanyang University · 2KENTECH
Adam's Law: Textual Frequency Law on Large Language Models
Apr 2, 2026Hongyuan Adam Lu, Z. L., Victor Wei +5LLM Prompting LLM Fine-Tuning
FaceMind Corporation · The Chinese University of Hong Kong
DEFT: Distribution-guided Efficient Fine-Tuning for Human Alignment
Apr 2, 2026Liang Zhu, Feiteng Fang, Yuelin Bai +4LLM Alignment LLM Fine-Tuning
Southern University of Science and Technology · Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Science and Technology of China +1
Taming CATS: Controllable Automatic Text Simplification through Instruction Fine-Tuning with Control Tokens
Apr 2, 2026Hanna Hubarava, Yingqiang GaoText Simplification Controllable Text Generation
Department of Computational Linguistics, University of Zurich, Switzerland · Department of Clinical Research, University of Bern, Switzerland
FourierMoE: Fourier Mixture-of-Experts Adaptation of Large Language Models
Apr 2, 2026Juyong Jiang, Fan Wang, Hong Qi +2LLM Fine-Tuning Parameter-Efficient Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
MiCA Learns More Knowledge Than LoRA and Full Fine-Tuning
Apr 2, 2026Sten Rüdiger, Sebastian RaschkaLLM Fine-Tuning Parameter-Efficient Fine-Tuning
Independent Researcher · RAIR Lab
Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness
Apr 2, 2026Deeptanshu Malu, Deevyanshu Malu, Aditya Nemiwal +1In-Context Example Selection LLM Fine-Tuning
IIT Bombay
Infeasibility Aware Large Language Models for Combinatorial Optimization
Apr 1, 2026Yakun Wang, Min Chen, Zeguan Wu +3Combinatorial Optimization Large Language Model-Guided Optimization
Lehigh University, Bethlehem, PA 18015, USA · Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260, USA · Johnson & Johnson, New Brunswick, NJ 08933, USA
Brainstacks: Cross-Domain Cognitive Capabilities via Frozen MoE-LoRA Stacks for Continual LLM Learning
Apr 1, 2026Mohammad R. Abu AyyashContinual Learning for LLMs LLM Fine-Tuning
Brains Build Research, Ramallah, Palestine
Positional Cognitive Specialization: Where Do LLMs Learn To Comprehend and Speak Your Language?
Apr 1, 2026Luis Frentzen Salim, Lun-Wei Ku, Hsing-Kuo Kenneth PaoMultilingual Language Models LLM Fine-Tuning
Institute of Information Science, Academia Sinica · Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology
More Human, More Efficient: Aligning Annotations with Quantized SLMs
Apr 1, 2026Jiayu Wang, Junyoung LeeLLM-Assisted Annotation LLM Fine-Tuning
Home Team Science and Technology Agency
Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
Mar 31, 2026Zhuowen Liang, Xiaotian Lin, Zhengxuan Zhang +3Question Answering Document QA
The Hong Kong University of Science and Technology (Guangzhou) · EvenUp, USA
KazByte: Adapting Qwen models to Kazakh via Byte-level Adapter
Mar 29, 2026Rauan AkylzhanovLow-Resource Language Processing Byte-Level Language Model
Independent Researcher, Almaty, Kazakhstan
QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning
Mar 29, 2026Mohammad AL-SmadiLegal Reasoning Arabic NLP
Qatar University Doha, Qatar
Development and Preliminary Evaluation of a Domain-Specific Large Language Model for Tuberculosis Care in South Africa
Mar 28, 2026Thokozile Khosa, Olawande DaramolaMedical LLMs Healthcare
Department of Computer Science, University of Pretoria, South Africa · Department of Informatics, University of Pretoria, South Africa
The Last Fingerprint: How Markdown Training Shapes LLM Prose
Mar 27, 2026E. M. FreeburgLLM Training Language Model Fingerprinting
Independent Researcher
Large Language Model as Token Compressor and Decompressor
Mar 26, 2026Wenbing Li, Yiran Wang, Zikai Song +4LLM Context Compression LLM Compression
AnyDoc: Enhancing Document Generation via Large-Scale HTML/CSS Data Synthesis and Height-Aware Reinforcement Optimization
Mar 26, 2026Jiawei Lin, Wanrong Zhu, Vlad I Morariu +1Text Generation Synthetic Data Generation for Language Models
Xi’an Jiaotong University · Adobe Research
OMIND: Framework for Knowledge Grounded Finetuning and Multi-Turn Dialogue Benchmark for Mental Health LLMs
Mar 26, 2026Suraj Racha, Prashant Harish Joshi, Utkarsh Maurya +6Medical LLMs LLM Fine-Tuning
Indian Institute of Technology Bombay
LiFT: How to Enable In-Context Learning for Longitudinal Modelling
Mar 25, 2026Iqra Ali, Talia Tseriotou, Mahmud Elahi Akhter +2In-Context Learning LLM Fine-Tuning
Queen Mary University of London (UK) · The Alan Turing Institute (UK)
Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset
Mar 25, 2026Mohammed Nowshad Ruhani Chowdhury, Mohammed Nowaz Rabbani Chowdhury, Sakari LukkarinenLLM Fine-Tuning Medical LLMs
School of ICT and Industrial Management, Metropolia University of Applied Sciences, Helsinki, Finland · Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
Fine-Tuning A Large Language Model for Systematic Review Screening
Mar 25, 2026Kweku Yamoah, Noah Schroeder, Emmanuel Dorley +2Systematic Review Automation LLM Fine-Tuning
University of Florida, Gainesville, FL, 32611
Lightweight Multimodal LLM-Enabled Cost-Effective Defect Grading of Power Transmission Equipment
Mar 25, 2026Tao Wang, Lipeng Zhu, Jiayong Li +2Multimodal Large Language Models LLM Fine-Tuning
Schema on the Inside: A Two-Phase Fine-Tuning Method for High-Efficiency Text-to-SQL at Scale
Mar 25, 2026Chinmay Soni, Shivam Chourasia, Gaurav Kumar +1Sports Analytics Text-to-SQL
Sporta Technologies Private Limited, Mumbai, Maharashtra, India

Apr 5, 2026J/K move · Enter open · S save

Evaluation of Embedding-Based and Generative Methods for LLM-Driven Document Classification: Opportunities and Challenges

Rong Lu, Hao Liu, Song Hou

Earth Science Text Classification VLM Evaluation Text Embedding Models Zero-Shot Text Classification LLM Fine-Tuning

Open paper AskPDF arXiv

This work presents a comparative analysis of embedding-based and generative models for classifying geoscience technical documents. Using a multi-disciplinary benchmark dataset, we evaluated the trade-offs between model accuracy, stability, and computational cost. We find that generative Vision-Language Models (VLMs) like Qwen2.5-VL, enhanced with Chain-of-Thought (CoT) prompting, achieve superior zero-shot accuracy (82%) compared to state-of-the-art multimodal embedding models like QQMM (63%). We also demonstrate that while supervised fine-tuning (SFT) can improve VLM performance, it is sensitive to training data imbalance.