Guardrail Erosion in Domain Fine-Tuned Large Language Models: A Paired Audit of Knowledge-Centric Abstention in Biomedical LLMs
Organizations: BCG X AI Science Institute, Boston Consulting Group · Centre for Human-Inspired Artificial Intelligence, University of Cambridge
Abstract
The number of domain fine-tuned models, particularly in the biomedical domain, is rapidly expanding. Fine-tuning promises high-performing models at a fraction of the cost of training a base model, but it can also produce unintended behavioral changes. Multiple studies have shown that safety guardrails can erode with fine-tuning, yet large-scale studies across model architectures, data mixtures, and fine-tuning methodologies remain limited. Where prior work in this space has studied value-centric refusal, this work focuses on knowledge-centric abstention. We audit 25 biomedical domain fine-tuned large language models against 11 paired base checkpoints on MedAbstain and ClinDet-Bench to evaluate the impact of fine-tuning on knowledge-centric abstention. We find that defined appropriate abstention measured by these benchmarks, already low in base models (median 8.3% on MedAbstain), falls further after fine-tuning (median 4.2%, decreasing in 21 of 25 pairs on MedAbstain), and that the magnitude and direction of change are heterogeneous across pairs. These results motivate fine-tuning methods that improve domain-specific capability while preserving appropriate caution.
Figures & tables
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Adapted checkpoint | Base checkpoint | Completion | Native chat |
|---|---|---|---|
| Aloe-Beta-70B | Llama-3.1-70B-Instruct | ||
| Aloe-Beta-8B | Llama-3.1-8B-Instruct | ||
| HuatuoGPT-o1-70B | Llama-3.1-70B-Instruct | ||
| HuatuoGPT-o1-8B | Llama-3.1-8B-Instruct | ||
| Med42-v2-70B | Llama-3-70B-Instruct | ||
| Med42-v2-8B | Llama-3-8B-Instruct |
| Benchmark | Domain | Concept | Description | Ref. |
| Knowledge-centric guardrails | ||||
| MedQA (USMLE) | Clinical | Capability | 4/5-option board-exam MCQA | Jin et al. (2021) |
| MedMCQA | Clinical | Capability | Large multi-subject Indian medical entrance MCQA | Pal et al. (2022) |
| PubMedQA | Clinical | Capability | Yes/no/maybe QA on PubMed abstracts | Jin et al. (2019) |
| EHRNoteQA | Clinical | Capability (notes) | QA requiring synthesis across MIMIC-IV discharge notes | Kweon et al. (2024a) |
| Med-HALT | Clinical | Hallucination / reasoning | Reasoning + memory hallucination tests | Pal et al. (2023) |
| Base checkpoint | Family | Params | Type | HF repository | Ref. |
|---|---|---|---|---|---|
| Llama-2-7B | Llama-2 | 7B | PT | meta-llama/Llama-2-7b-hf | Touvron et al. (2023) |
| Llama-2-13B | Llama-2 | 13B | PT | meta-llama/Llama-2-13b-hf | Touvron et al. (2023) |
| Llama-2-70B | Llama-2 | 70B | PT | meta-llama/Llama-2-70b-hf | Touvron et al. (2023) |
| Llama-3-8B | Llama-3 | 8B | PT | meta-llama/Meta-Llama-3-8B | Grattafiori et al. (2024) |
| Llama-3-8B-Instruct | Llama-3 | 8B | IT | meta-llama/Meta-Llama-3-8B-Instruct | Grattafiori et al. (2024) |
| Llama-3-70B-Instruct | Llama-3 | 70B | IT | meta-llama/Meta-Llama-3-70B-Instruct | Grattafiori et al. (2024) |
| Domain-adapted model | Params | Base checkpoint | Recipe | Stg. | Style | MA cont. | Ref. |
| Llama-2 family | |||||||
| Asclepius-Llama2-7B | 7B | Llama-2-7B | CPT + SFT | 2 | compl. | no | Kweon et al. (2024b) |
| Asclepius-Llama2-13B | 13B | Llama-2-13B | CPT + SFT | 2 | compl. | no | Kweon et al. (2024b) |
| Camel-Platypus2-13B | 13B | Llama-2-13B | MERGE | 1 | compl. | plaus. | Lee et al. (2023) |
| qCammel-13 | 13B | Llama-2-13B | QLoRA | 1 | compl. | plaus. | Toma et al. (2023) |
| Camel-Platypus2-70B | 70B | Llama-2-70B | MERGE | 1 | compl. | plaus. | Lee et al. (2023) |
LLM Fine-Tuning
LLM fine-tuning adapts pretrained large language models to tasks, domains, languages, and behaviors through additional training.
See all 794 papers