Domain Adaptation for LLMs

LLM: Large Language Model

Ask about this topic

Overview

This field studies how to adapt general-purpose language models to settings such as medicine, law, finance, engineering, and less-resourced languages. Common approaches include continued pretraining on domain text, supervised fine-tuning with curated or synthetic examples, parameter-efficient methods such as LoRA, and retrieval-augmented generation; newer work also explores model merging, modular adapters, and test-time adaptation. Evaluations use domain-specific question answering, reasoning, generation, and structured-prediction benchmarks, sometimes alongside expert review. A central tension is gaining reliable domain expertise without losing general abilities or safety: adaptation can help, but it does not consistently outperform general models or prevent hallucinations.

Latest papers

CardsList

94 papers17 papers in the last four weeks, up 21% on the four weeks before. 0.1% of all new papers.

  1. Generation and Verification: Human-Agent Collaboration for Safety-Critical Automotive Software Engineering

    Oct 26, 2026Elizaveta Zinovyeva, Aiham Taleb, Nikita Kozodoi +5AI-Assisted Software DevelopmentCoding Agents

    Amazon Web Services, Germany · AUMOVIO, Regensburg, Germany

  2. SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift

    Oct 25, 2026Muhammad Atif Saeed, Azka Atiq, Adarsh Kumar +3Online Language Model AdaptationSelective Prediction

    Department of AI & Data Science, National University of Computer & Emerging Sciences, Islamabad, Pakistan · Data Engineer, University of Arkansas at Little Rock, Arkansas, USA · Department of Computer Science, National University of Computer & Emerging Sciences, Islamabad, Pakistan +2

  3. TriageRAG: Failure-Aware Self-Training for Adapting Retrieval-Augmented Language Models to Specialized Domains

    Oct 24, 2026Samarth Varshney, Ritu YadavRetrieval-Augmented GenerationDomain Adaptation for LLMs

  4. Adaptive Transfer of Representational Signatures for LLM Reliability Detection

    Oct 24, 2026Yongtong Li, Yixuan Florence Wu, Jiawei Zhang +1Language Model Error DetectionLLM Reliability

    Northwestern University · University of Wisconsin–Madison

  5. Parameter-Efficient Adaptation of Large Language Models to a Low-Resource Agglutinative Language: A Corpus, Datasets, and a Reproducible Benchmark for the Bashkir Language

    Oct 22, 2026Mullosharaf K. Arabov, Svetlana S. KhaybullinaLow-Resource Language ProcessingParameter-Efficient Fine-Tuning

    Department of Data Analysis and Programming Technologies, Institute of Computational Mathematics and Information Technologies, Kazan (Volga Region) Federal University, Kazan, Russia

  6. AQL-Lite: Enabling Efficient Knowledge Acquisition for Arabic LLMs via Data-Centric Continual Pretraining

    Oct 21, 2026Houssam Eddine Boukhalfa, Mourad Bouache, Mourad DahmaneArabic Language ModelsLanguage Model Pretraining

    National School of Artificial Intelligence (ENSIA), Algiers, Algeria · Nojoom AI

  7. Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting

    Oct 20, 2026Zijie Liu, Zhen Tan, Charles Fleming +1Continual Learning for LLMsDomain Adaptation

    UNC at Chapel Hill · Stevens Institute of Technology · Cisco Research

  8. Thunder-DeID+: Boundary-Aware Korean De-identification from Court Judgments to Broader Sensitive Documents

    Oct 15, 2026Gyuseong Lee, Gyeongje Cho, Jongmin Kim +8Legal NLPSubword Tokenization

    Dept. of Data Science, Seoul National University · Dept. of Computer Science and Engineering, Seoul National University

  9. ixi-sLM: On-Device sLM for AI Call Agent Service

    Oct 14, 2026Han-Sang Lee, Kyuho Lee, Juneyoung Park +10On-Device Language Model InferenceSmall Language Models

    LG Uplus · OptAI Inc.

  10. In-Place Indic Tokenizer Expansion and Continual Pretraining of Nemotron-3-Nano-30B-A3B

    Oct 11, 2026Avinash Benki, Prasanjith Pasunuti, Sutanshu Saksena Raj +3Language Model PretrainingTokenizer Adaptation

  11. Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning

    Oct 8, 2026Lingxiang Wang, Hainan Zhang, Liang Pang +2Continual Learning for LLMsParameter-Efficient Fine-Tuning

    School of Artificial Intelligence, Beihang University · Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · Institute of Computing Technology, Chinese Academy of Sciences

  12. AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility

    Oct 7, 2026Josh McGiff, Salma Mekaoui, Robert Shanahan +1Domain Adaptation for LLMs

    University of Limerick, Ireland

  13. FinVector-Market-4B: A Controlled Study of LoRA Adaptation for Structured Financial Tasks

    Oct 6, 2026Alina KhaybullinaDomain Adaptation for LLMsLLM Fine-Tuning

    Independent Researcher

  14. Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation

    Oct 5, 2026Guangyuan Dong, Ziwei Hong, Xuehao Zhou +6Low-Rank AdaptationDomain Adaptation for LLMs

    National University of Singapore, Singapore · University of Pennsylvania, USA · Department of Urology, Shanghai General Hospital, Shanghai, China +3

  15. HuatuoGPT-3: RL-Only Domain Adaptation from Base Models

    Oct 5, 2026Junying Chen, Xinyuan Xie, Ziniu Li +7RL for Language ModelsReinforcement Learning

    The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data · Shenzhen Loop Area Institute +1

  16. Representation-Aligned Auxiliary Supervision for Language Model Adaptation

    Oct 2, 2026Kyuyoung Kim, Peiyao Sheng, Ashwin Hebbar +9Domain Adaptation for LLMsMulti-Task Learning

    KAIST AI · Sentient Labs · Princeton University +3

  17. Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases

    Sep 28, 2026Jiangtao Lin, Bangyang Wei, Yihang Ding +2Continual Learning for LLMsDomain Adaptation for LLMs

    Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · SZ DJI Technology Co., Ltd. +1

  18. CORTEX: Learning to Share and Specialize in Dense Language Models

    Sep 28, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongDomain Adaptation for LLMsLanguage Modeling

    Simon Fraser University · Southern University of Science and Technology · The University of British Columbia

  19. EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events

    Sep 27, 2026Andre R Goncalves, Vincent Liu, Priyadip RayDomain Adaptation for LLMsLong-Tail Learning

    Lawrence Livermore National Laboratory · Kaiser Permanente

  20. IndustryLLM: Failure-Driven LLM Training for Industrial Procurement

    Sep 25, 2026Liang Ding, Zhiang Xu, Yuyang Sheng +13LLM Fine-TuningDomain Adaptation for LLMs

    Project Lead · Multimodal and Industrial AI Team†∗ Alibaba

  21. Taramandal-GPT: Enhancing Astrodynamics Problem-Solving with Knowledge Retrieval and Structured Thinking

    Sep 21, 2026Akhil Sharma, Jatin Gupta, Ali Imam AbidiDomain Adaptation for LLMs

    Department of Computer Science and Engineering Sharda University

  22. Tutoring Large Language Models to be Domain-adaptive, Precise and Safe

    Sep 19, 2026Somnath BanerjeeLLM SafetyLLM Safety Alignment

  23. Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs

    Sep 16, 2026Mohan Shi, Zilai Wang, Natarajan Balaji Shankar +3Domain Adaptation for ASRAdapter Tuning

    Department of Electrical and Computer Engineering University of California Los Angeles Los Angeles, USA

  24. Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models

    Sep 15, 2026Vanessa Lama, Sanjay Das, Emily Herron +5LLM EvaluationScientific QA

    Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA · The Ohio State University, Columbus, Ohio, USA · Max-Planck-Institut für Astronomie, Königstuhl 17, D-69117 Heidelberg, Germany +1

  25. Merging the Knowledge of LLMs for Automatic Speech Recognition

    Sep 14, 2026Hayato Futami, Tatsuya KawaharaDomain Adaptation for ASRAutomatic Speech Recognition

    Graduate School of Informatics, Kyoto University, Japan

  26. SALUTE: Benchmarking and Adapting LLMs for the Defense Domain

    Sep 14, 2026Hyeongcheol Park, Sumin In, Suyeon Myeong +5LLM EvaluationDomain-Adaptive Pretraining

    Korea University, Republic of Korea · Sungkyunkwan University, Republic of Korea · Hanwha Systems, Republic of Korea

  27. Domain-specific Pretraining Profile and Transformer Performance: Evidence from Modeling Digital Pragmatics in Arabic-English Code-switching

    Sep 13, 2026Fahad Al Hussen, King Saud University, Riyadh +5Arabic NLPPragmatic Reasoning in Language Models

    King Saud University, Riyadh, Saudi Arabia · Ibb University, Ibb, Yemen

  28. ORDER: A Fictitious-World Benchmark for Domain-Adaptive Embodied AI

    Sep 13, 2026Sai Krishna Reddy Sathi, Anuj TiwariEmbodied AIEmbodied Reasoning

    Indian Institute of Technology Madras, India

  29. Domain-Specific Jargon in Large Language Models: A Comparative Analysis between General-Purpose and Specialist Models

    Sep 11, 2026Darin Keng, Zhewei SunDomain Adaptation for LLMsMedical LLMs

    Department of Statistics, University of Chicago, Chicago, Illinois · Toyota Technological Institute at Chicago, Chicago, Illinois

  30. When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models

    Sep 4, 2026Xiaodong Li, Peiwei LiuNumerical Reasoning in Language ModelsHallucination in Language Models

    School of Computer Science, Guangzhou College of Applied Science and Technology, Guangzhou, 511370, China · IT Department, COFCO Corporation, Beijing, 100020, China