Domain Adaptation for LLMs

LLM: Large Language Model

Overview

This field studies how to adapt general-purpose language models to settings such as medicine, law, finance, engineering, and less-resourced languages. Common approaches include continued pretraining on domain text, supervised fine-tuning with curated or synthetic examples, parameter-efficient methods such as LoRA, and retrieval-augmented generation; newer work also explores model merging, modular adapters, and test-time adaptation. Evaluations use domain-specific question answering, reasoning, generation, and structured-prediction benchmarks, sometimes alongside expert review. A central tension is gaining reliable domain expertise without losing general abilities or safety: adaptation can help, but it does not consistently outperform general models or prevent hallucinations.

Latest papers

CardsList

102 papers14 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching

    Nov 17, 2026Giannis Daras, Marios Papachristou, Adam Klivans +2Fine-Tuning Language ModelsSupervised Fine-Tuning

  2. MarineSparkLM-4B: A Lightweight Language Model for Marine and Offshore Engineering Built by Parameter-Efficient Adaptation

    Nov 14, 2026Cheng Siong ChinLLM Fine-TuningDomain Adaptation for LLMs

  3. Beyond Passive Imitation: On-Policy Mid-Training Distillation for Language Models

    Nov 10, 2026Ruize Zheng, Xuyang Liu, Siyu Ren +5Language Model DistillationLanguage Model Pretraining

  4. Adapting Qwen3 to Tajik from 0.6B to 8B: Lower Text Loss Did Not Mean Better Understanding

    Nov 4, 2026Tohir Saidzoda, Madina Abdusalomzoda, Nasimjon Rajabov +7LLM Fine-TuningLanguage Model Pretraining

  5. Upcycling Language Models for Multiple Specialized Domains

    Oct 29, 2026Mingzi Cao, Xi Wang, Nikolaos AletrasDomain Adaptation for LLMsDomain Adaptation

  6. Assessing Language Models for Scientific Applications: Inertial Confinement Fusion as a Use Case

    Oct 27, 2026P. B. Radha, D. D. Meyerhofer, M. Bhattarai +11Inertial Confinement FusionScientific Reasoning in Language Models

  7. Generation and Verification: Human-Agent Collaboration for Safety-Critical Automotive Software Engineering

    Oct 26, 2026Elizaveta Zinovyeva, Aiham Taleb, Nikita Kozodoi +5AI-Assisted Software DevelopmentCoding Agents

  8. SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift

    Oct 25, 2026Muhammad Atif Saeed, Azka Atiq, Adarsh Kumar +3Online Language Model AdaptationSelective Prediction

  9. Adaptive Transfer of Representational Signatures for LLM Reliability Detection

    Oct 24, 2026Yongtong Li, Yixuan Florence Wu, Jiawei Zhang +1Language Model Error DetectionLLM Reliability

  10. Parameter-Efficient Adaptation of Large Language Models to a Low-Resource Agglutinative Language: A Corpus, Datasets, and a Reproducible Benchmark for the Bashkir Language

    Oct 22, 2026Mullosharaf K. Arabov, Svetlana S. KhaybullinaLow-Resource Language ProcessingParameter-Efficient Fine-Tuning

  11. AQL-Lite: Enabling Efficient Knowledge Acquisition for Arabic LLMs via Data-Centric Continual Pretraining

    Oct 21, 2026Houssam Eddine Boukhalfa, Mourad Bouache, Mourad DahmaneArabic Language ModelsLanguage Model Pretraining

  12. Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting

    Oct 20, 2026Zijie Liu, Zhen Tan, Charles Fleming +1Continual Learning for LLMsDomain Adaptation

  13. Thunder-DeID+: Boundary-Aware Korean De-identification from Court Judgments to Broader Sensitive Documents

    Oct 15, 2026Gyuseong Lee, Gyeongje Cho, Jongmin Kim +8Legal NLPSubword Tokenization

  14. ixi-sLM: On-Device sLM for AI Call Agent Service

    Oct 14, 2026Han-Sang Lee, Kyuho Lee, Juneyoung Park +10On-Device Language Model InferenceSmall Language Models

  15. Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning

    Oct 8, 2026Lingxiang Wang, Hainan Zhang, Liang Pang +2Continual Learning for LLMsParameter-Efficient Fine-Tuning

  16. AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility

    Oct 7, 2026Josh McGiff, Salma Mekaoui, Robert Shanahan +1Domain Adaptation for LLMs

  17. Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation

    Oct 5, 2026Guangyuan Dong, Ziwei Hong, Xuehao Zhou +6Low-Rank AdaptationDomain Adaptation for LLMs

  18. HuatuoGPT-3: RL-Only Domain Adaptation from Base Models

    Oct 5, 2026Junying Chen, Xinyuan Xie, Ziniu Li +7RL for Language ModelsReinforcement Learning

  19. Representation-Aligned Auxiliary Supervision for Language Model Adaptation

    Oct 2, 2026Kyuyoung Kim, Peiyao Sheng, Ashwin Hebbar +9Domain Adaptation for LLMsMulti-Task Learning

  20. Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases

    Sep 28, 2026Jiangtao Lin, Bangyang Wei, Yihang Ding +2Continual Learning for LLMsDomain Adaptation for LLMs

  21. CORTEX: Learning to Share and Specialize in Dense Language Models

    Sep 28, 2026Chuiyang Meng, Ming Tang, Vincent W. S. Wongmodular language model architecturesDomain Adaptation for LLMs

  22. EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events

    Sep 27, 2026Andre R Goncalves, Vincent Liu, Priyadip RayDomain Adaptation for LLMsLong-Tail Learning

  23. IndustryLLM: Failure-Driven LLM Training for Industrial Procurement

    Sep 25, 2026Liang Ding, Zhiang Xu, Yuyang Sheng +13LLM Fine-TuningDomain Adaptation for LLMs

  24. Taramandal-GPT: Enhancing Astrodynamics Problem-Solving with Knowledge Retrieval and Structured Thinking

    Sep 21, 2026Akhil Sharma, Jatin Gupta, Ali Imam AbidiDomain Adaptation for LLMs