Domain Adaptation for LLMs

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 38

All topics

Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation
Oct 5, 2026Guangyuan Dong, Ziwei Hong, Xuehao Zhou +6Low-Rank Adaptation Domain Adaptation for LLMs
National University of Singapore, Singapore · University of Pennsylvania, USA · Department of Urology, Shanghai General Hospital, Shanghai, China +3
HuatuoGPT-3: RL-Only Domain Adaptation from Base Models
Oct 5, 2026Junying Chen, Xinyuan Xie, Ziniu Li +7RL for Language Models Reinforcement Learning
The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data · Shenzhen Loop Area Institute +1
Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases
Sep 28, 2026Jiangtao Lin, Bangyang Wei, Yihang Ding +2Continual Learning for LLMs Domain Adaptation for LLMs
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · SZ DJI Technology Co., Ltd. +1
CORTEX: Learning to Share and Specialize in Dense Language Models
Sep 28, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongDomain Adaptation for LLMs Language Modeling
Simon Fraser University · Southern University of Science and Technology · The University of British Columbia
EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events
Sep 27, 2026Andre R Goncalves, Vincent Liu, Priyadip RayLong-Tail Learning Adapter Tuning
Lawrence Livermore National Laboratory · Kaiser Permanente
Taramandal-GPT: Enhancing Astrodynamics Problem-Solving with Knowledge Retrieval and Structured Thinking
Sep 21, 2026Akhil Sharma, Jatin Gupta, Ali Imam AbidiDomain Adaptation for LLMs
Department of Computer Science and Engineering Sharda University
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs
Sep 16, 2026Mohan Shi, Zilai Wang, Natarajan Balaji Shankar +3Domain Adaptation for ASR Adapter Tuning
Department of Electrical and Computer Engineering University of California Los Angeles Los Angeles, USA
Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models
Sep 15, 2026Vanessa Lama, Sanjay Das, Emily Herron +5LLM Evaluation Scientific QA
Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA · The Ohio State University, Columbus, Ohio, USA · Max-Planck-Institut für Astronomie, Königstuhl 17, D-69117 Heidelberg, Germany +1
Merging the Knowledge of LLMs for Automatic Speech Recognition
Sep 14, 2026Hayato Futami, Tatsuya KawaharaDomain Adaptation for ASR Automatic Speech Recognition
Graduate School of Informatics, Kyoto University, Japan
SALUTE: Benchmarking and Adapting LLMs for the Defense Domain
Sep 14, 2026Hyeongcheol Park, Sumin In, Suyeon Myeong +5LLM Evaluation Domain-Adaptive Pretraining
Korea University, Republic of Korea · Sungkyunkwan University, Republic of Korea · Hanwha Systems, Republic of Korea
When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models
Sep 7, 2026Xiaodong Li, Peiwei LiuNumerical Reasoning in Language Models Hallucination in Language Models
School of Computer Science, Guangzhou College of Applied Science and Technology, Guangzhou, 511370, China · IT Department, COFCO Corporation, Beijing, 100020, China
Reading the News: Adapting Large Language Models to Swedish Journalism Through Continued Pre-Training
Aug 31, 2026Lukas Borggren, Jenny Kunz, Marco KuhlmannContinual Learning for LLMs LLM Evaluation
Linköping University · Bonnier News
CPR for LLMs: Critical-Point Routing against Catastrophic Forgetting in Domain Adaptation
Aug 31, 2026Kwangmin Ki, Yunhun Nam, Jongheon Jeong +1LLM Fine-Tuning Domain Adaptation for LLMs
Korea University · Yonsei University
DataFoundry: Evolving Data Preparators via Recursive Self-Improvement
Aug 30, 2026Cehao Yang, Xiaojun Wu, Xueyuan Lin +4Training Data Curation Domain Adaptation for LLMs
1IDEA Research, International Digital Economy Academy · 2Hong Kong University of Science and Technology (Guangzhou) · 3DataArc Tech Ltd.
TELLME: Test-Enhanced Learning for Language Model Enrichment
Aug 12, 2026Minjun Kim, Inho Won, Hyeonseok Lim +6Domain-Adaptive Pretraining Efficient Language Model Training
Korea Advanced Institute of Science and Technology · Seoul National University of Science and Technology · National Security Research Institute
An Agentic Generative Large Language Model for Treatment Planning of Colorectal Cancer
Aug 10, 2026Mengxian Lyu, Cheng Peng, Tim Jang +18Clinical Decision-Making Domain Adaptation for LLMs
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA. · Division of Hematology & Oncology, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA · University of Florida Health Cancer Institute, Gainesville, FL, USA +8
Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization
Aug 1, 2026Xuefei Wang, Jun Han, Zixuan Wang +4Few-Shot Domain Adaptation LLM Personalization
Beihang University · Nankai University
MemSFT: Mitigating Alignment Tax with an External Parametric Memory
Jul 28, 2026Jiarui Wang, Xiang Shi, Jiaqi Cao +8LLM Alignment Memory-Augmented Language Models
LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University · Shanghai AI Laboratory · School of Computer Science, Shanghai Jiao Tong University +1
Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias
Jul 24, 2026Ofek I. Cohen, Lior Shani, Aviv Rosenberg +3Black-Box Optimization Domain Adaptation for LLMs
Tel Aviv University · Google Research · Columbia University
A Better Start for Language Models: Domain-Conditional Position Offsets
Jul 17, 2026Ye QiaoDomain Adaptation Language Model Calibration
University of California, Irvine
Integrating knowledge graphs and multilingual scholarly corpora for domain-adaptive LLMs in SSH
Jul 7, 2026Adam Faci, Alessio Miaschi, Anne Combe +4Multilingual Language Models LLM Evaluation
Huma-Num, CNRS, Paris, France · CNR - Institute for Computational Linguistics “A. Zampolli” (CNR-ILC), Pisa, Italy · Inria, France +1
Enhancing Fitness Intelligence through Domain-Specific LLM Post-Training
Jul 2, 2026Xingtao Zhao, Tian Yang, Han JiangDomain Adaptation LLM Fine-Tuning
School of Cyber Science and Technology Beihang University Beijing, China · School of Information Renmin University of China Beijing, China
Conversational Domain Adaptation of IndicTrans2 across 21 Indic Languages via Experience Replay and Model Soups
Jun 27, 2026Aditya Pratap SinghDomain Adaptation for LLMs Machine Translation
Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA
Jun 17, 2026Ikram Belmadani, Oumaima El Khettari, Carlos Ramisch +3LLM Fine-Tuning Domain-Adaptive Pretraining
Aix-Marseille Univ., CNRS, LIS UMR 7020, 13000 Marseille, France · Nantes Univ., École Centrale Nantes, CNRS, LS2N UMR 6004, 44000 Nantes, France · Grenoble Alpes Univ., CNRS, INRIA, Grenoble INP, LIG UMR 5217, 38000 Grenoble, France
Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data
Jun 4, 2026Srinivasan Manoharan, Dilipkumar Nallusamy, Sachin Kumar +1Multi-Label Classification Small Language Models
PayPal Inc San Jose, USA · PayPal Inc Chennai, India · PayPal Inc Bangalore, India
RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
May 29, 2026Yuduo Li, Xiaofeng Shi, Qian Kou +2Supervised Fine-Tuning Fine-Tuning
Beijing Academy of Artificial Intelligence (BAAI) · Beijing Jiaotong University (BJTU)
Domain Adaptation and Reasoning Frameworks in Language Models: A Controlled Experiment with Historical Cosmology
May 28, 2026Francesco De BernardisDomain Adaptation LLM Fine-Tuning
Independent Researcher
EmbGen: Teaching with Reassembled Corpora
May 19, 2026Arun K Lenin, Kai Rouse, Andrea Nicastro +1Domain Adaptation for LLMs Synthetic Data Generation for Language Models
Commonwealth Bank of Australia Sydney, Australia
Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights
May 11, 2026Saba Pourhanifeh, AbdulAziz AbdulGhaffar, Ashraf MatrawyDomain Adaptation Cybersecurity
Department of Systems and Computer Engineering Carleton University · School of Information Technology Carleton University
Phoenix-VL 1.5 Medium Technical Report
May 11, 2026Team Phoenix, :, Arka Ray +30Multilingual Language Models Domain Adaptation
HTX × Mistral AI ∗
HPC-LLM: Practical Domain Adaptation and Retrieval-Augmented Generation for HPC Support
May 8, 2026Nourin Shahin, Izzat AlsmadiRetrieval-Augmented Generation Domain Adaptation
Texas A&M University San Antonio
CoDA: Towards Effective Cross-domain Knowledge Transfer via CoT-guided Domain Adaptation
Apr 21, 2026Jianzhi Yan, Le Liu, Buzhou Tang +3Domain Adaptation Cross-Domain Transfer Learning
Harbin Institute of Technology, Shenzhen, 518100, China · Pengcheng Laboratory, Shenzhen, 518100, China
Can Continual Pre-training Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain?
Apr 21, 2026Niclas Doll, Jasper Schulze Buschhoff, Shalaka Satheesh +3Domain-Adaptive Pretraining Domain Adaptation for LLMs
Fraunhofer IAIS · Lamarr Institute
SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
Apr 20, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuContinual Learning for LLMs LLM Fine-Tuning
The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China
Evaluating Hallucinations in Domain-Adapted Large Language Models
Apr 19, 2026Sanchita Porwal, Sai Prasath S, Xingjian Bi +1LLM Evaluation Domain Adaptation
College of Computing Georgia Institute of Technology Atlanta, GA 30318, USA
Efficient Task Adaptation in Large Language Models via Selective Parameter Optimization
Apr 18, 2026Weijie Wan, Jiangjiang ZhaoContinual Learning for LLMs LLM Fine-Tuning
Shortest Path Technology · China Mobile Communications Group Co.,Ltd
ASDA: Automated Skill Distillation and Adaptation for Financial Reasoning
Mar 17, 2026Tik Yu Yim, Wenting Tan, Sum Yee Chan +2Domain Adaptation for LLMs Agent Skill Learning
The University of Hong Kong, Hong Kong SAR, China
Diffract: Spectral View of LLM Domain Adaptation
Date pendingNikita Borodin, Maria Krylova, Artem Zabolotnyi +6Domain Adaptation Domain-Adaptive Pretraining
Risk AI Research Lab · Applied AI Institute

Oct 5, 2026, cs.AIJ/K move · Enter open · S save

Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation

Guangyuan Dong, Ziwei Hong, Xuehao Zhou, Zidong Yu+5

National University of Singapore, Singapore · University of Pennsylvania, USA · Department of Urology, Shanghai General Hospital, Shanghai, China+3

Low-Rank Adaptation Domain Adaptation for LLMs Medical QA Medical LLMs

Open paper AskPDF arXiv

Medical question answering spans specialties and clinical operations that may benefit from different adaptation directions. We propose ARBOR, a parameter-efficient method that selects rank-one components from a shared low-rank basis for each question. An additive gate combines question representations, specialty tags, operation tags, and their interaction; a learned coefficient scales the adapter residual. An illustrative separation under orthogonal, equiprobable subtasks shows how conditional selection can avoid an approximation floor faced by a fixed update with the same active rank. This result motivates the design without asserting a corresponding bound for medical corpora. On Qwen3-8B across CMB, CMExam, MedQA, and MedMCQA, five-seed experiments yield 69.69% mean accuracy across benchmarks, exceeding LoRA r16 and MoELoRA by 1.26 and 1.30 percentage points, respectively. The reported advantage over LoRA r16 increases from 0.08 to 1.94 points as training expands from one to seven specialties. Tag perturbations and atom masking support the usefulness of clinical routing, while atom clusters align with the supplied specialty labels (adjusted Rand index 0.62). Calibration, transfer, and measured costs further characterize the method. These findings support structured conditional adaptation for medical QA, while leaving clinical safety and broader deployment untested.

Momentum

Related topics

Latest papers 38