Domain Adaptation for LLMs

LLM: Large Language Model

Ask about this topic

Overview

This field studies how to adapt general-purpose language models to settings such as medicine, law, finance, engineering, and less-resourced languages. Common approaches include continued pretraining on domain text, supervised fine-tuning with curated or synthetic examples, parameter-efficient methods such as LoRA, and retrieval-augmented generation; newer work also explores model merging, modular adapters, and test-time adaptation. Evaluations use domain-specific question answering, reasoning, generation, and structured-prediction benchmarks, sometimes alongside expert review. A central tension is gaining reliable domain expertise without losing general abilities or safety: adaptation can help, but it does not consistently outperform general models or prevent hallucinations.

Latest papers

CardsList

94 papers17 papers in the last four weeks, up 21% on the four weeks before. 0.1% of all new papers.

  1. Efficient Task Adaptation in Large Language Models via Selective Parameter Optimization

    Apr 18, 2026Weijie Wan, Jiangjiang ZhaoContinual Learning for LLMsLLM Fine-Tuning

    Shortest Path Technology · China Mobile Communications Group Co.,Ltd

  2. Training-Free Test-Time Contrastive Learning for Large Language Models

    Apr 15, 2026Kaiwen Zheng, Kai Zhou, Jinwu Hu +3Test-Time AdaptationLanguage Model Robustness

    South China University of Technology · Pazhou Laboratory

  3. GCA Framework: A GCC Countries-Grounded Dataset and Agentic Pipeline for Climate Decision Support

    Apr 14, 2026Muhammad Umer Sheikh, Khawar Shehzad, Salman Khan +2LLM GroundingLLM Tool Use

    Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), UAE · University of Missouri, USA · Australian National University, Australia +1

  4. Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?

    Apr 9, 2026Yatong Wang, Fali Wang, Naibin Gu +6Model MergingDomain Adaptation for LLMs

    Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · The Pennsylvania State University, University Park, USA +1

  5. Is Biomedical Specialization Still Worth It? Insights from Domain-Adaptive Language Modelling with a New French Health Corpus

    Apr 8, 2026Aidan Mannion, Cécile Macaire, Armand Violle +5Domain-Adaptive PretrainingDomain Adaptation for LLMs

    Université Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France · Sorbonne Université, LIMICS, 15 rue de l’École de Médecine, 75006 Paris, France

  6. To Adapt or not to Adapt, Rethinking the Value of Medical Knowledge-Aware Large Language Models

    Apr 8, 2026Ane G. Domingo-Aldama, Iker De La Iglesia, Maitane Urruela +2Medical LLMsDomain Adaptation for LLMs

  7. Do Domain-specific Experts exist in MoE-based LLMs?

    Apr 7, 2026Giang Do, Hung Le, Truyen TranMixture-of-Experts Language ModelsMixture-of-Experts Models

    Applied Artificial Intelligence Intiative (A2I2), Deakin University

  8. GAIN: Multiplicative Modulation for Domain Adaptation

    Apr 6, 2026Hengshuai Yao, Xing Chen, Ahmed Murtadha +1Domain Adaptation for LLMsContinual Learning for LLMs

    Sapient Intelligence · Department of Computing Science, University of Alberta

  9. Domain Adaptation of Large Language Models for Polymer-Composite Additive Manufacturing Using Retrieval-Augmented Generation and Fine-Tuning

    Apr 2, 2026Saiful Islam Sagor, Tania Haghighi, Minhaj Nur Alam +1Domain Adaptation for LLMsLLM Fine-Tuning

    Department of Mechanical Engineering and Engineering Science, University of North Carolina at Charlotte, NC 28223, USA. · Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, NC 28223, USA.

  10. Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging

    Apr 2, 2026Mengxian Lyu, Cheng Peng, Ziyi Chen +3Domain Adaptation for LLMsMedical LLMs

    Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA · Department of Engineering Education, Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL, USA · Preston A. Wells, Jr. Center for Brain Tumor Therapy, Lillian S. Wells Department of Neurosurgery, University of Florida, Gainesville, Florida, USA

  11. OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training

    Mar 30, 2026Haiyue Song, Masao UtiyamaContinued PretrainingData Mixture Optimization

    National Institute of Information and Communications Technology, Kyoto, Japan

  12. KazByte: Adapting Qwen models to Kazakh via Byte-level Adapter

    Mar 29, 2026Rauan AkylzhanovLow-Resource Language ProcessingByte-Level Language Model

    Independent Researcher, Almaty, Kazakhstan

  13. QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning

    Mar 29, 2026Mohammad AL-SmadiLegal ReasoningArabic NLP

    Qatar University Doha, Qatar

  14. Context-Mediated Domain Adaptation in Multi-Agent Sensemaking Systems

    Mar 25, 2026Anton Wolter, Leon Haag, Vaishali Dhanoa +1Human-AI InteractionDomain Adaptation for LLMs

    Aarhus University, Denmark · Maastricht University, Netherlands · TU Wien, Austria

  15. Benchmarking Linguistic Adaptation in Comparable-Sized LLMs: A Study of Llama-3.1-8B, Mistral-7B-v0.1, and Qwen3-8B on Romanized Nepali

    Mar 25, 2026Ananda Rimal, Adarsha RimalLow-Resource Language ProcessingDomain Adaptation for LLMs

    Nepal Engineering College · Dept. of Computer Science & Engineering, Nepal Engineering College · Tribhuvan University +1

  16. Domain Adapted Large Language Models for Additive Manufacturing

    Mar 23, 2026Peter Pak, Amir Barati FarimaniMultimodal Large Language ModelsDomain Adaptation for LLMs

    Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA · Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA

  17. DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation

    Mar 22, 2026Shuai Wang, Dhasarathy Parthasarathy, Robert Feldt +1Case-Based ReasoningCode Generation

    Chalmers University of Technology, Gothenburg, Sweden · Volvo Group, Gothenburg, Sweden

  18. Exploring different approaches to customize language models for domain-specific text-to-code generation

    Mar 17, 2026Luís Freire, Fernanda A. Andaló, Nicki Skafte DetlefsenCode GenerationLLM Code Generation

    Technical University of Denmark · The LEGO Group

  19. ASDA: Automated Skill Distillation and Adaptation for Financial Reasoning

    Mar 17, 2026Tik Yu Yim, Wenting Tan, Sum Yee Chan +2Domain Adaptation for LLMsAgent Skill Learning

    The University of Hong Kong, Hong Kong SAR, China

  20. An Industrial-Scale Insurance LLM Achieving Verifiable Domain Mastery and Hallucination Control without Competence Trade-offs

    Mar 15, 2026Qian Zhu, Xinnan Guo, Jingjing Huo +5Domain Adaptation for LLMsLLM Alignment

    Ant Group

  21. Adapting Methods for Domain-Specific Japanese Small LMs: Scale, Architecture, and Quantization

    Mar 12, 2026Takato YasunoLanguage ModelingSmall Language Models

  22. Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation

    Mar 10, 2026Luxi Lin, Zhihang Lin, Zhanpeng Zeng +5Speculative DecodingLLM Inference Acceleration

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Innovation Institute +1

  23. Cognitively Layered Data Synthesis for Domain Adaptation of LLMs to Space Situational Awareness

    Mar 10, 2026Ding Linghu, Cheng Wang, Da Fan +6Space Situational AwarenessDomain Adaptation for LLMs

    Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology, Beijing, China · China Academy of Space Technology, Beijing, China · State Key Laboratory of Space Information System and Integrated Application, Beijing, China

  24. AutoAdapt: An Automated Domain Adaptation Framework for LLMs

    Mar 9, 2026Sidharth Sinha, Anson Bastos, Xuchao Zhang +3Domain Adaptation for LLMsDomain Adaptation

    Microsoft, M365 Research, India · Microsoft, M365 Research, Redmond · Microsoft Research, India

  25. Replaying pre-training data improves fine-tuning

    Mar 5, 2026Suhas Kotha, Percy LiangLLM Fine-TuningData Repetition in Language Model Training

    Stanford University

  26. Building a Strong Instruction Language Model for a Less-Resourced Language

    Mar 2, 2026Domen Vreš, Tjaša Arčon, Timotej Petrič +3Low-Resource Language ProcessingLLM Fine-Tuning

    University of Ljubljana, Faculty of Computer and Information Science, Ljubljana, Slovenia

  27. Constructing Synthetic Instruction Datasets for Improving Reasoning in Domain-Specific LLMs: A Case Study in the Japanese Financial Domain

    Mar 2, 2026Yuma Okochi, Fabio Milentiansen Sim, Tomoyasu OkadaFinancial Language Model EvaluationFinancial Reasoning in Language Models

    Nomura Research Institute, Ltd. · NRI Indonesia

  28. When Metrics Disagree: Automatic Similarity vs. LLM-as-a-Judge for Clinical Dialogue Evaluation

    Feb 27, 2026Bian Sun, Zhenjian Wang, Orvill de la Torre +1Clinical Language Model EvaluationGenerative AI Evaluation

    Heinz College, Carnegie Mellon University · School of Computer Science, Carnegie Mellon University

  29. Full-Stack Domain Enhancement for Combustion LLMs: Construction and Optimization

    Feb 27, 2026Quanjia Xiao, Weimin Ouyang, Zonglin Yang +4Domain Adaptation for LLMsScientific Reasoning in Language Models

    Peking University · AI for Science Institute, Beijing · DP Technology

  30. MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation

    Feb 24, 2026Daniel Tamayo, Iñaki Lacunza, Paula Rivera-Hidalgo +4Multilingual Language ModelsLanguage Model Pretraining

    Barcelona Supercomputing Center