Domain Adaptation for LLMs

This field studies how to adapt general-purpose language models to settings such as medicine, law, finance, engineering, and less-resourced languages. Common approaches include continued pretraining on domain text, supervised fine-tuning with curated or synthetic examples, parameter-efficient methods such as LoRA, and retrieval-augmented generation; newer work also explores model merging, modular adapters, and test-time adaptation. Evaluations use domain-specific question answering, reasoning, generation, and structured-prediction benchmarks, sometimes alongside expert review. A central tension is gaining reliable domain expertise without losing general abilities or safety: adaptation can help, but it does not consistently outperform general models or prevent hallucinations.