Domain Adaptation for LLMs

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 42

All topics

Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning
Oct 8, 2026Lingxiang Wang, Hainan Zhang, Liang Pang +2Continual Learning for LLMs Parameter-Efficient Fine-Tuning
School of Artificial Intelligence, Beihang University · Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · Institute of Computing Technology, Chinese Academy of Sciences
AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility
Oct 7, 2026Josh McGiff, Salma Mekaoui, Robert Shanahan +1Domain Adaptation for LLMs
University of Limerick, Ireland
FinVector-Market-4B: A Controlled Study of LoRA Adaptation for Structured Financial Tasks
Oct 6, 2026Alina KhaybullinaDomain Adaptation for LLMs LLM Fine-Tuning
Independent Researcher
Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation
Oct 5, 2026Guangyuan Dong, Ziwei Hong, Xuehao Zhou +6Low-Rank Adaptation Domain Adaptation for LLMs
National University of Singapore, Singapore · University of Pennsylvania, USA · Department of Urology, Shanghai General Hospital, Shanghai, China +3
HuatuoGPT-3: RL-Only Domain Adaptation from Base Models
Oct 5, 2026Junying Chen, Xinyuan Xie, Ziniu Li +7RL for Language Models Reinforcement Learning
The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data · Shenzhen Loop Area Institute +1
Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases
Sep 28, 2026Jiangtao Lin, Bangyang Wei, Yihang Ding +2Continual Learning for LLMs Domain Adaptation for LLMs
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · SZ DJI Technology Co., Ltd. +1
CORTEX: Learning to Share and Specialize in Dense Language Models
Sep 28, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongDomain Adaptation for LLMs Language Modeling
Simon Fraser University · Southern University of Science and Technology · The University of British Columbia
EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events
Sep 27, 2026Andre R Goncalves, Vincent Liu, Priyadip RayLong-Tail Learning Adapter Tuning
Lawrence Livermore National Laboratory · Kaiser Permanente
Taramandal-GPT: Enhancing Astrodynamics Problem-Solving with Knowledge Retrieval and Structured Thinking
Sep 21, 2026Akhil Sharma, Jatin Gupta, Ali Imam AbidiDomain Adaptation for LLMs
Department of Computer Science and Engineering Sharda University
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs
Sep 16, 2026Mohan Shi, Zilai Wang, Natarajan Balaji Shankar +3Domain Adaptation for ASR Adapter Tuning
Department of Electrical and Computer Engineering University of California Los Angeles Los Angeles, USA
Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models
Sep 15, 2026Vanessa Lama, Sanjay Das, Emily Herron +5LLM Evaluation Scientific QA
Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA · The Ohio State University, Columbus, Ohio, USA · Max-Planck-Institut für Astronomie, Königstuhl 17, D-69117 Heidelberg, Germany +1
Merging the Knowledge of LLMs for Automatic Speech Recognition
Sep 14, 2026Hayato Futami, Tatsuya KawaharaDomain Adaptation for ASR Automatic Speech Recognition
Graduate School of Informatics, Kyoto University, Japan
SALUTE: Benchmarking and Adapting LLMs for the Defense Domain
Sep 14, 2026Hyeongcheol Park, Sumin In, Suyeon Myeong +5LLM Evaluation Domain-Adaptive Pretraining
Korea University, Republic of Korea · Sungkyunkwan University, Republic of Korea · Hanwha Systems, Republic of Korea
When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models
Sep 7, 2026Xiaodong Li, Peiwei LiuNumerical Reasoning in Language Models Hallucination in Language Models
School of Computer Science, Guangzhou College of Applied Science and Technology, Guangzhou, 511370, China · IT Department, COFCO Corporation, Beijing, 100020, China
Reading the News: Adapting Large Language Models to Swedish Journalism Through Continued Pre-Training
Aug 31, 2026Lukas Borggren, Jenny Kunz, Marco KuhlmannContinual Learning for LLMs LLM Evaluation
Linköping University · Bonnier News
CPR for LLMs: Critical-Point Routing against Catastrophic Forgetting in Domain Adaptation
Aug 31, 2026Kwangmin Ki, Yunhun Nam, Jongheon Jeong +1LLM Fine-Tuning Domain Adaptation for LLMs
Korea University · Yonsei University
DataFoundry: Evolving Data Preparators via Recursive Self-Improvement
Aug 30, 2026Cehao Yang, Xiaojun Wu, Xueyuan Lin +4Training Data Curation Domain Adaptation for LLMs
1IDEA Research, International Digital Economy Academy · 2Hong Kong University of Science and Technology (Guangzhou) · 3DataArc Tech Ltd.
TELLME: Test-Enhanced Learning for Language Model Enrichment
Aug 12, 2026Minjun Kim, Inho Won, Hyeonseok Lim +6Domain-Adaptive Pretraining Efficient Language Model Training
Korea Advanced Institute of Science and Technology · Seoul National University of Science and Technology · National Security Research Institute
An Agentic Generative Large Language Model for Treatment Planning of Colorectal Cancer
Aug 10, 2026Mengxian Lyu, Cheng Peng, Tim Jang +18Clinical Decision-Making Domain Adaptation for LLMs
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA. · Division of Hematology & Oncology, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA · University of Florida Health Cancer Institute, Gainesville, FL, USA +8
Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization
Aug 1, 2026Xuefei Wang, Jun Han, Zixuan Wang +4Few-Shot Domain Adaptation LLM Personalization
Beihang University · Nankai University
MemSFT: Mitigating Alignment Tax with an External Parametric Memory
Jul 28, 2026Jiarui Wang, Xiang Shi, Jiaqi Cao +8LLM Alignment Memory-Augmented Language Models
LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University · Shanghai AI Laboratory · School of Computer Science, Shanghai Jiao Tong University +1
Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias
Jul 24, 2026Ofek I. Cohen, Lior Shani, Aviv Rosenberg +3Black-Box Optimization Domain Adaptation for LLMs
Tel Aviv University · Google Research · Columbia University
A Better Start for Language Models: Domain-Conditional Position Offsets
Jul 17, 2026Ye QiaoDomain Adaptation Language Model Calibration
University of California, Irvine
Integrating knowledge graphs and multilingual scholarly corpora for domain-adaptive LLMs in SSH
Jul 7, 2026Adam Faci, Alessio Miaschi, Anne Combe +4Multilingual Language Models LLM Evaluation
Huma-Num, CNRS, Paris, France · CNR - Institute for Computational Linguistics “A. Zampolli” (CNR-ILC), Pisa, Italy · Inria, France +1
Enhancing Fitness Intelligence through Domain-Specific LLM Post-Training
Jul 2, 2026Xingtao Zhao, Tian Yang, Han JiangDomain Adaptation LLM Fine-Tuning
School of Cyber Science and Technology Beihang University Beijing, China · School of Information Renmin University of China Beijing, China
Conversational Domain Adaptation of IndicTrans2 across 21 Indic Languages via Experience Replay and Model Soups
Jun 27, 2026Aditya Pratap SinghDomain Adaptation for LLMs Machine Translation
Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA
Jun 17, 2026Ikram Belmadani, Oumaima El Khettari, Carlos Ramisch +3LLM Fine-Tuning Domain-Adaptive Pretraining
Aix-Marseille Univ., CNRS, LIS UMR 7020, 13000 Marseille, France · Nantes Univ., École Centrale Nantes, CNRS, LS2N UMR 6004, 44000 Nantes, France · Grenoble Alpes Univ., CNRS, INRIA, Grenoble INP, LIG UMR 5217, 38000 Grenoble, France
Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data
Jun 4, 2026Srinivasan Manoharan, Dilipkumar Nallusamy, Sachin Kumar +1Multi-Label Classification Small Language Models
PayPal Inc San Jose, USA · PayPal Inc Chennai, India · PayPal Inc Bangalore, India
RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
May 29, 2026Yuduo Li, Xiaofeng Shi, Qian Kou +2Supervised Fine-Tuning Fine-Tuning
Beijing Academy of Artificial Intelligence (BAAI) · Beijing Jiaotong University (BJTU)
Domain Adaptation and Reasoning Frameworks in Language Models: A Controlled Experiment with Historical Cosmology
May 28, 2026Francesco De BernardisDomain Adaptation LLM Fine-Tuning
Independent Researcher

Oct 8, 2026, cs.LGJ/K move · Enter open · S save

Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning

Lingxiang Wang, Hainan Zhang, Liang Pang, Hongwei Zheng+1

School of Artificial Intelligence, Beihang University · Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · Institute of Computing Technology, Chinese Academy of Sciences

Continual Learning for LLMs Parameter-Efficient Fine-Tuning Catastrophic Forgetting Fine-Tuning Domain Adaptation for LLMs

Open paper AskPDF arXiv

Domain-specific continual adaptation of LLMs risks catastrophic forgetting, creating a fundamental tension between acquiring new capabilities and preserving those learned during pretraining. PEFT mitigates this problem by restricting the number of trainable parameters, but existing methods lack a principled unit for deciding where plasticity should be allocated and stability should be preserved. We identify the singular-vector channel as a natural unit for managing this trade-off. Each channel represents an input-output transformation, which can be updated to acquire new knowledge or fixed to preserve pretrained capabilities. Based on this perspective, we introduce SVC, a parameter-efficient continual-learning method that selectively updates Singular-Vector Channels. Before fine-tuning, SVC uses domain-specific data to estimate each channel's adaptation benefit and a fixed public general-domain corpus only as a history activation proxy for estimating forgetting cost. It then adaptively selects trainable channels based on these scores via knee-based cost screening, Pareto-front filtering, and Otsu thresholding. Experimental results across four LLM families and eight downstream tasks show that SVC better preserves pretrained capabilities while achieving strong downstream performance relative to existing PEFT baselines. Further analysis of channel scoring and selection demonstrates that selective plasticity at the singular-vector-channel level enables effective continual LLM adaptation.

Momentum

Related topics

Latest papers 42