LLM Fine-Tuning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

42 papers in the last four weeks, up 83% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 509

All topics

Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale
Apr 26, 2026Avi-ad Avraam BuskilaRetrieval-Augmented Generation Healthcare
Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel
The Override Gap: A Magnitude Account of Knowledge Conflict Failure in Hypernetwork-Based Instant LLM Adaptation
Apr 26, 2026Shuaizhi Cheng, Xiang Shi, Zhiwei Zhang +1Knowledge Conflicts in Language Models LLM Fine-Tuning
Harbin Institute of Technology · KigLand Machine Learning Lab · Imperial College London
Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer
Apr 26, 2026Ahmed Haj Ahmed, Ruochen Zhang, Alvin GrissomMultilingual Language Model Evaluation LLM Fine-Tuning
Haverford College · Brown University
$\mathcal{S}^2$ IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction
Apr 25, 2026Bingfeng Chen, Chenjie Qiu, Yifeng Xie +3LLM Fine-Tuning Aspect-Based Sentiment Analysis
School of Computer Science, Guangdong University of Technology · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Peng Cheng Laboratory +1
Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective
Apr 25, 2026Bishwamittra Ghosh, Soumi Das, Till Speicher +5LLM Evaluation LLM Fine-Tuning
Max Planck Institute for Software Systems, Germany · Boston University, USA
DeepImagine: Learning Biomedical Reasoning via Successive Counterfactual Imagining
Apr 24, 2026Youze Zheng, Jianyou Wang, Yuhan Chen +8LLM Fine-Tuning Counterfactual Reasoning in Language Models
Laboratory for Emerging Intelligence, University of California, San Diego · University of Chicago · Elsevier +1
Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge
Apr 24, 2026Mengyu Wang, Xiaoying Zhi, Zhiyi Li +4Document Understanding LLM Fine-Tuning
1The University of Edinburgh · 2JPMorgan Chase & Co.
mcdok at SemEval-2026 Task 13: Finetuning LLMs for Detection of Machine-Generated Code
Apr 23, 2026Adam Skurla, Dominik Macko, Jakub SimkoAI-Generated Text Detection LLM Fine-Tuning
Faculty of Information Technology, Brno University of Technology, Brno, Czechia · Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia
CARE: Counselor-Aligned Response Engine for Online Mental-Health Support
Apr 23, 2026Hagai Astrin, Ayal Swaid, Avi Segal +1Mental Health Counseling Emotional Support Conversation
Ben-Gurion University · School of Informatics, University of Edinburgh Edinburgh, UK
Align Generative Artificial Intelligence with Human Preferences: A Novel Large Language Model Fine-Tuning Method for Online Review Management
Apr 23, 2026Yanan Wang, Yong GeLLM Hallucination Mitigation LLM Fine-Tuning
Department of Information Systems and Operations Management, The University of Texas at Arlington · Department of Management Information Systems, University of Arizona
AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Models
Apr 22, 2026Tadesse Destaw Belay, Shahriar Kabir Nahin, Israel Abebe Azime +6Multilingual Language Models Intelligent Tutoring Systems
Instituto Politécnico Nacional, Mexico · University of South Florida, FL, USA · Saarland University, Germany +2
COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling
Apr 22, 2026Noah FlynnMultilingual Language Models Continual Learning for LLMs
UC Berkeley
Compatibility-Aware Dynamic Fine-Tuning for Large Language Models
Apr 22, 2026Yucheng Zhou, Junwei Sheng, Qianning Wang +1Supervised Fine-Tuning Fine-Tuning
SKL-IOTSC, CIS, University of Macau · Auckland University of Technology
IRIS: Interpolative Rényi Iterative Self-play for Large Language Model Fine-Tuning
Apr 22, 2026Wenjie Liao, Like Wu, Liangjie Zhao +2Fine-Tuning LLM Fine-Tuning
Graduate School of Information, Production and Systems, Waseda University, Fukuoka, Japan · Institute of Computing Technology, University of the Chinese Academy of Sciences
RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models
Apr 21, 2026Yusuf Çelebi, Yağız Asker, Özay Ezerceli +4LLM Fine-Tuning Low-Rank Adaptation
FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion
Apr 21, 2026Tao Fan, Guoqiang Ma, Yuanfeng Song +3Privacy-Preserving Language Models LLM Fine-Tuning
Hong Kong University of Science and Technology, Hong Kong, China · WeBank Co., Ltd, Shenzhen, China · Hong Kong Polytechnic University, Hong Kong, China
Fine-Tuning Small Reasoning Models for Quantum Field Theory
Apr 21, 2026Nathaniel S. Woodward, Zhiqi Gao, Yurii Kvasiuk +3Physical Reasoning LLM Fine-Tuning
Department of Physics, University of Wisconsin-Madison · Department of Computer Science, University of Wisconsin-Madison · Perimeter Institute for Theoretical Physics
Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
Apr 20, 2026Fei Wang, Li Shen, Liang Ding +3Zeroth-Order Optimization LLM Fine-Tuning
South China University of Technology · Shenzhen Campus of Sun Yat-sen University · University of Sydney +2
LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL
Apr 20, 2026Salmane Chafik, Saad Ezzini, Ismail BerradaLLM Fine-Tuning Code Generation
Mohammed VI Polytechnic University Morocco · King Fahd University of Petroleum and Minerals Saudi Arabia
TLoRA: Task-aware Low Rank Adaptation of Large Language Models
Apr 20, 2026Weicheng Lin, Yi Zhang, Jiawei Dang +1LLM Fine-Tuning Low-Rank Adaptation
College of Computer Science and Software Engineering, Shenzhen University, China
JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew
Apr 20, 2026Itay Razumenko, Arnon Sturm, Nir GrinbergLegal NLP LLM Personalization
Computer and Information Science Ben-Gurion University of the Negev Beer-Sheva, Israel
CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora
Apr 20, 2026Shangyu Li, Juyong Jiang, Meibo Ren +7Code Translation RL for Language Models
The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou) · Monash University +1
SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
Apr 20, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuContinual Learning for LLMs LLM Fine-Tuning
The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China
Evaluating Hallucinations in Domain-Adapted Large Language Models
Apr 19, 2026Sanchita Porwal, Sai Prasath S, Xingjian Bi +1LLM Evaluation Domain Adaptation
College of Computing Georgia Institute of Technology Atlanta, GA 30318, USA
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
Apr 19, 2026Wanru Zhao, Yihong Chen, Yuzhi Tang +6Language Model Pretraining LLM Fine-Tuning
University of Cambridge · OATML, University of Oxford · University of Toronto +2
REALM: Reliable Expertise-Aware Language Model Fine-Tuning from Noisy Annotations
Apr 19, 2026Sajjad Ghiasvand, Mark Beliaev, Mahnoosh Alizadeh +1Supervised Fine-Tuning LLM Fine-Tuning
Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA
Continual Safety Alignment via Gradient-Based Sample Selection
Apr 19, 2026Thong Bach, Dung Nguyen, Thao Minh Le +1Continual Learning for LLMs LLM Fine-Tuning
Applied Artificial Intelligence Initiative (A2I2), Deakin University · Pennsylvania State University
Beyond the Basics: Leveraging Large Language Model for Fine-Grained Medical Entity Recognition
Apr 19, 2026Nwe Ni Win, Jim Basilakis, Steven Thomas +6Named Entity Recognition Healthcare
Western Sydney University, Sydney, Australia · Western Sydney University, Australia · South Western Emergency Research Institute, Sydney, Australia +7
Guardrails in Logit Space: Safety Token Regularization for LLM Alignment
Apr 19, 2026Thong Bach, Truyen TranLLM Fine-Tuning LLM Safety Alignment
Applied Artificial Intelligence Initiative (A2I2), Deakin University
How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them
Apr 18, 2026Disen Liao, Freda ShiLLM Fine-Tuning Language Modeling
University of Waterloo · Vector Institute

Apr 26, 2026, cs.CLJ/K move · Enter open · S save

Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale

Avi-ad Avraam Buskila

Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel

Retrieval-Augmented Generation Healthcare LLM Fine-Tuning Question Answering Medical QA RAG Evaluation

Open paper AskPDF arXiv

Practitioners deploying small open-weight large language models (LLMs) for medical question answering face a recurring design choice: invest in a domain-fine-tuned model, or keep a general-purpose model and inject domain knowledge at inference time via retrieval-augmented generation (RAG). We isolate this trade-off by holding model size, prompt template, decoding temperature, retrieval pipeline, and evaluation protocol fixed, and varying only (i) whether the model has been domain-adapted (Gemma 3 4B vs. MedGemma 4B, both 4-bit quantized and served via Ollama) and (ii) whether retrieved passages from a medical knowledge corpus are inserted into the prompt. We evaluate all four cells of this 2x2 design on the full MedQA-USMLE 4-option test split (1,273 questions) with three repetitions per question (15,276 LLM calls). Domain fine-tuning yields a +6.8 percentage-point gain in majority-vote accuracy over the general 4B baseline (53.3% vs. 46.4%, McNemar p < 10^-4). RAG over MedMCQA explanations does not produce a statistically significant gain in either model, and in the domain-tuned model the point estimate is slightly negative (-1.9 pp, p = 0.16). At this scale and on this benchmark, domain knowledge encoded in weights dominates domain knowledge supplied in context. We release the full experiment code and JSONL traces to support replication.

Momentum

Related topics

Latest papers 509