Prompt Sensitivity and Answer Consistency of Small Open-Source Language Models for Clinical Question Answering in Low-Resource Healthcare

Shravani Hariprasad

cs.CLMar 1, 2026

Prompt Sensitivity and Answer Consistency of Small Open-Source Language Models for Clinical Question Answering in Low-Resource Healthcare

Authors: Shravani Hariprasad

Organizations: Independent Researcher

Abstract

Small open-source language models are gaining attention for healthcare applications in low-resource settings where cloud infrastructure and GPU hardware may be unavailable. However, the reliability of these models under different phrasings of the same clinical question remains poorly understood. We evaluate five open-source models (Gemma 2 2B, Phi-3 Mini 3.8B, Llama 3.2 3B, Mistral 7B, and Meditron-7B, a domain-pretrained model without instruction tuning) across three clinical question answering datasets (MedQA, MedMCQA, and PubMedQA) using five prompt styles: original, formal, simplified, roleplay, and direct. Model behavior is evaluated using consistency scores, accuracy, and instruction-following failure rates. All experiments were conducted locally on consumer CPU hardware without fine-tuning. Consistency and accuracy were largely independent across models. Gemma 2 achieved the highest consistency (0.845-0.888) but the lowest accuracy (33.0-43.5%), while Llama 3.2 showed moderate consistency (0.774-0.807) alongside the highest accuracy (49.0-65.0%). Roleplay prompts consistently reduced accuracy across all models, with Phi-3 Mini dropping 21.5 percentage points on MedQA. Meditron-7B exhibited near-complete instruction-following failure on PubMedQA (99.0% UNKNOWN rate), indicating that domain pretraining alone is insufficient for structured clinical question answering. These findings show that high consistency does not imply correctness: models can be reliably wrong, a dangerous failure mode in clinical AI. Llama 3.2 demonstrated the strongest balance of accuracy and reliability for low-resource deployment. Safe clinical AI requires joint evaluation of consistency, accuracy, and instruction adherence.

Figures & tables

Model	Params	Base Model	Inst. Tuned	Fine-Tuning Method	Medical Domain	Chat Template
Phi-3 Mini	3.8B	Phi-3 (Microsoft)	Yes	SFT + DPO + RLHF	No	<\|user\|>..<\|end\|>
Llama 3.2	3B	Llama 3.1 (Meta)	Yes	SFT + RS + DPO (multi-round)	No	<\|start_header_id\|>..<\|eot_id\|>
Gemma 2	2B	Gemma 2 (Google)	Yes	KD + SFT + RLHF + Model Merging	No	<start_of_turn>..<end_of_turn>
Mistral 7B	7B	Mistral (Mistral AI)	Yes	SFT (public datasets)	No	[INST]..[/INST]
Meditron-7B $\dagger$	7B	Llama-2 (Meta)	No	Continued pretraining only (PubMed + guidelines)	Yes	None

Table 1: Architectural and training characteristics of evaluated models. SFT = Supervised Fine-Tuning; DPO = Direct Preference Optimisation; RLHF = Reinforcement Learning from Human Feedback; RS = Rejection Sampling; KD = Knowledge Distillation.

\dagger

Meditron-7B is a continued pretraining model without instruction tuning; its chat template is effectively undefined.

Dataset	Model	Mean	Fully	Overall	UNKNOWN	Acc.	Acc.	Acc.	Acc.	Acc.
		Cons.	Cons.	Acc.	Rate	Orig.	Form.	Simp.	Role.	Dir.
			(%)	(%)	(%)	(%)	(%)	(%)	(%)	(%)
MedQA	Phi-3 Mini	0.698	19.5	48.0	10.5	44.5	44.5	42.5	26.5	48.0
	Llama 3.2	0.776	35.0	49.0	0.8	47.5	44.5	46.0	47.0	43.5
	Gemma 2	0.888	63.5	40.0	2.1	39.0	39.0	37.0	35.0	35.0
	Mistral 7B	0.800	41.0	45.0	4.7	45.5	41.5	43.0	34.0	50.5

Table 2: Summary of consistency scores, overall accuracy, and instruction-following failure rates across all models and datasets. The best value among instruction-tuned models is shown in bold .

\dagger

Meditron-7B is a pretraining model without instruction tuning; high UNKNOWN rates reflect instruction-following failure rather than lack of domain knowledge.

Dataset	M1	M2	Metric	p	Sig.
MedQA	Llama 3.2	Phi-3 Mini	Cons.	0.0005
	Gemma 2	Phi-3 Mini	Cons.	$<$ .0001
	Mistral	Phi-3 Mini	Cons.	$<$ .0001
	Llama 3.2	Gemma 2	Cons.	$<$ .0001
	Meditron	Phi-3 Mini	Cons.	$<$ .0001
	Meditron	Llama 3.2	Cons.	0.3342	ns

Table 3: Pairwise statistical significance tests. Wilcoxon signed-rank test for consistency; McNemar test for accuracy. *** p

<

0.001, ** p

<

0.01, * p

<

0.05, ns = not significant.

\dagger

Meditron PubMedQA comparisons reflect instruction-following failure (99% UNKNOWN rate).

Figure 1: Mean consistency scores across models and datasets. Higher scores indicate greater agreement across prompt styles.

Figure 2: Overall accuracy across models and datasets, calculated using majority answer against ground truth labels.

Figure 3: Accuracy by prompt style across models and datasets. Roleplay consistently underperforms relative to other styles.

Figure 4: Roleplay prompt accuracy versus best performing non-roleplay style across all models and datasets, with annotated performance gaps.

Figure 5: Instruction-following failure rates (UNKNOWN responses) across models and datasets. Phi-3 Mini shows the highest failure rate on MedQA at 10.5%.

Figure 6: Scatter plot of mean consistency score versus overall accuracy across all models and datasets. No clear positive correlation is observed, indicating that consistency and accuracy are independent metrics.

Figure 7: Distribution of consistency scores across models and datasets. Phi-3 Mini shows the widest spread on MedQA; Gemma 2 shows a tight but high distribution reflecting systematic consistency without accuracy.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Style	Prompt Text (abbreviated)	Raw Response	Extracted	Correct?
Original	“Answer the following medical question by choosing the correct option. …Answer with only the option letter (A, B, C, or D).”	Option A: Inferior vena c…	A	No
Formal	“You are a medical expert. Based on your clinical knowledge, select the most appropriate answer …Respond with only the letter of the correct choice (A, B, C, or D).”	B: Coronary sinus	B	Yes
Simplified	“Read this medical question carefully and pick the best answer. …Which letter is correct? Reply with just A, B, C, or D.”	B: Coronary sinus The…	B	Yes
Roleplay	“You are a physician taking a medical licensing exam. Answer this question as you would on the exam. …Your answer (A, B, C, or D):”	The correct answer is A: Inferior v…	A	No
Direct	“Medical question: …What is the correct option? State only the letter.”	B	B	Yes

Appendix

Table 4: Prompt texts, raw model output, and extracted answers for Phi-3 Mini on a representative MedQA question. The correct answer is B (Coronary sinus). The Roleplay style produces an incorrect answer (A) despite identical question content across all five variations.

Style	Prompt Text (abbreviated)	Raw Response	Extracted	Correct?
Original	“Answer the following medical question by choosing the correct option. …Answer with only the option letter (A, B, C, or D).”	D	D	No
Formal	“You are a medical expert. Based on your clinical knowledge, select the most appropriate answer …Respond with only the letter of the correct choice (A, B, C, or D).”	D	D	No
Simplified	“Read this medical question carefully and pick the best answer. …Which letter is correct? Reply with just A, B, C, or D.”	D	D	No
Roleplay	“You are a physician taking a medical licensing exam. Answer this question as you would on the exam. …Your answer (A, B, C, or D):”	The correct answer is **(D) Propran…	D	No
Direct	“Medical question: …What is the correct option? State only the letter.”	D	D	No

Appendix

Table 5: Gemma 2 responses across all five prompt styles for a representative MedQA question demonstrating reliable incorrectness. The correct answer is B (Isosorbide dinitrate). Gemma 2 selects D (Propranolol) under every prompt style, yielding a consistency score of 1.0 and 0% accuracy on this question.

Explore similar work

Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework
Apr 12, 2026Avi-ad Avraam BuskilaLLM Evaluation LLM Reliability
Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel
When Chain-of-Thought Backfires: Evaluating Prompt Sensitivity in Medical Language Models
Mar 26, 2026Binesh Sadanandan, Vahid BehzadanPrompt Sensitivity LLM Prompting
SAIL Lab, University of New Haven, West Haven, CT 06516, USA
BIT.UA-AAUBS at ArchEHR-QA 2026: Evaluating Open-Source and Proprietary LLMs via Prompting in Low-Resource QA
May 5, 2026Richard A. A. Jonker, Alexander Christiansen, Alexandros Maniatis +4LLM Grounding Healthcare
Aalborg University Business School Fibigerstræde 2, Aalborg East 9220, Denmark · IEETA, DETI, LASI, University of Aveiro Campus Universitário de Santiago, Aveiro 3810-193, Portugal

Apr 12, 2026J/K move · Enter open · S save

Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework

Avi-ad Avraam Buskila

Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel

LLM Evaluation LLM Reliability Clinical Language Model Evaluation Medical LLMs Biomedical QA

Open paper AskPDF arXiv

Incorporating large language models (LLMs) in medical question answering demands more than high average accuracy: a model that returns substantively different answers each time it is queried is not a reliable medical tool. Online health communities such as Reddit have become a primary source of medical information for millions of users, yet they remain highly susceptible to misinformation; deploying LLMs as assistants in these settings amplifies the need for output consistency alongside correctness. We present a practical, open-source evaluation framework for assessing small, locally-deployable open-weight LLMs on medical question answering, treating reproducibility as a first-class metric alongside lexical and semantic accuracy. Our pipeline computes eight quality metrics, including BERTScore, ROUGE-L, and an LLM-as-judge rubric, together with two within-model reproducibility metrics derived from repeated inference (N=10 runs per question). Evaluating three models (Llama 3.1 8B, Gemma 3 12B, MedGemma 1.5 4B) on 50 MedQuAD questions (N=1,500 total responses) reveals that despite low-temperature generation (T=0.2), self-agreement across runs reaches at most 0.20, while 87-97% of all outputs per model are unique -- a safety gap that single-pass benchmarks entirely miss. The clinically fine-tuned MedGemma 1.5 4B underperforms the larger general-purpose models on both quality and reproducibility; however, because MedGemma is also the smallest model, this comparison confounds domain fine-tuning with model scale. We describe the methodology in sufficient detail for practitioners to replicate or extend the evaluation for their own model-selection workflows. All code and data pipelines are available at https://github.com/aviad-buskila/llm_medical_reproducibility.

Abstract

Explore similar work

Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework

When Chain-of-Thought Backfires: Evaluating Prompt Sensitivity in Medical Language Models

BIT.UA-AAUBS at ArchEHR-QA 2026: Evaluating Open-Source and Proprietary LLMs via Prompting in Low-Resource QA