From Small to Large: Assessing Medical Reasoning, Benchmark Contamination, and Model Size in Language Models

Navapat Nananukul; Mayank Kejriwal

cs.AIOct 30, 2026

From Small to Large: Assessing Medical Reasoning, Benchmark Contamination, and Model Size in Language Models

Authors: Navapat Nananukul, Mayank Kejriwal

Organizations: Information Sciences Institute, University of Southern California, Marina del Rey, CA, USA · Lead contact

Abstract

Large language models (LLMs) now answer medical exam questions at or above expert level, but it is unclear whether such scores reflect clinical competence, or are instead affected by the format of the questions being asked (e.g., multiple-choice versus open-ended), prior exposure to the benchmarks, and the difficulty of the questions. These benchmarks are also public and widely used online, which may be resulting in LLM memorization. Models may perform better on questions that they have seen before. It is also not evident whether smaller models, which are more locally deployable in HIPAA-compliant hospital settings, have achieved human performance. In this study, we evaluate fourteen current models from six leading providers, one smaller and one larger model from each provider with two additional open-weight models, on 3,076 questions from four medical benchmarks (MedQA-USMLE, MedMCQA, PubMedQA, and MedBullets) under standard multiple choice, the same questions with the answer choices withheld, and reconstruction of a hidden question from its choices alone. In the standard multiple-choice setting, the strongest models exceed the human-expert baseline on MedQA-USMLE and PubMedQA, but the smaller models trail their larger models by 8.7 percentage points and only two of the six pass the human-expert bar on the licensing examination. Withholding the answer choices, however, makes every model struggle: mean accuracy falls from 82.3% to 50.9%, and the two largest drops belong to the medically specialised models. On memorization, the strongest models reconstruct one question in four from the answer choices alone, yet accuracy holds on the questions they cannot reconstruct. We further weigh accuracy against inference cost and test whether ensembling several models improves medical question answering.

Figures & tables

Figure 1: The three evaluation settings, illustrated on one question from MedMCQA. Hatched panels mark the information withheld from the model in each setting.

	MedQA-USMLE 19	MedMCQA 37	PubMedQA 20	MedBullets 6
What the question asks	A clinical scenario from one patient, followed by a question about diagnosis or health management	A single fact drawn from one of 21 preclinical and clinical subjects	Whether the abstract of a published study supports the research question that study asked	A clinical scenario written for board preparation, followed by a question about diagnosis or management
Answer choices	A/B/C/D	A/B/C/D	yes / no / maybe	A/B/C/D
Question source	US licensing exam	AIIMS/NEET PG	PubMed abstracts	MedBullets question bank
No. of questions	1,273	1,000	500	303

Table 1: The four benchmarks and the evaluation sets used in this paper.

Figure 2: One question example from each benchmark. Green marks the recorded answer key.

Provider	Model	Size (total / active)	Context	Released	Last Evaluated on (2026)
OpenAI	GPT-OSS 20B	21B / 3.6B	131K	Aug-2025	23 Sep
	GPT-5.6 Luna	undisclosed	1.05M	Jul-2026	23 Sep
DeepSeek	DeepSeek V4 Flash	284B / 13B	1M	Apr-2026	23 Sep
	DeepSeek V4 Pro	1.6T / 49B	1M	Apr-2026	23 Sep
Qwen	Qwen 3.5 9B	9B	262K	Mar-2026	23 Sep
	Qwen 3.7 Plus	undisclosed	1M	Jun-2026	23 Sep

Table 2: Language models evaluated in this study, grouped by provider. Each provider contributes a smaller and a larger model. We report model size, context window size, and “Evaluated on” which is the dates on which the runs reported here were executed.

This work: current models (2026)
Model	MedQA-USMLE	MedMCQA	PubMedQA	MedBullets
Qwen 3.7 Plus	94.66 [93.3, 95.8]	83.80 [81.4, 86.0]	81.60 [78.0, 84.8]	87.79 [83.6, 91.0]
GPT-5.6 Luna	94.66 [93.3, 95.8]	83.80 [81.4, 86.0]	78.00 [74.2, 81.4]	89.11 [85.1, 92.1]
DeepSeek V4 Pro	94.42 [93.0, 95.6]	82.40 [79.9, 84.6]	76.00 [72.1, 79.5]	87.46 [83.3, 90.7]
DeepSeek V4 Flash	93.95 [92.5, 95.1]	81.80 [79.3, 84.1]	77.80 [74.0, 81.2]	88.45 [84.4, 91.6]
MedGemma 27B	91.99 [90.4, 93.4]	79.70 [77.1, 82.1]	77.40 [73.5, 80.8]	82.51 [77.8, 86.4]

Table 3: Multiple-choice accuracy (%) on four benchmarks, reported as accuracy followed by its 95% confidence interval in brackets. Current models (2026) are ordered by their mean accuracy across all 3,076 questions. The lower block reports prior-generation models and human baselines from prior work 26 , and no intervals are computed for them. The best current-model value in each column is in bold. † Open-weight models added to compare smaller and larger models whose sizes are known.

Figure 3: Multiple-choice accuracy on all 3,076 questions of the four benchmarks, with each provider’s smaller and larger model side by side. The dashed line is the human-expert reference of 86.5%, weighted by benchmark size over MedQA-USMLE, MedMCQA and PubMedQA, the benchmarks that have one; Table 3 gives each benchmark separately.

Figure 4: Accuracy on MedQA-USMLE, MedMCQA and MedBullets pooled (2,576 questions), comparing multiple choice (setting 1) and generative setting (setting 2), one row per model. Teal markers are multiple-choice accuracy, red markers the generative setting. The connecting segment is the difference between them, given in points.

Figure 5: Question reconstruction (Setting 3). (a) How often each model reconstructs the hidden question from the answer choices alone. (b) Every MedQA-USMLE, MedMCQA and MedBullets question resolved four ways, by whether the model reconstructed it and whether it answered it correctly. (c) Reconstruction rate against multiple-choice accuracy on all four benchmarks, one point per model; the dashed line is the least-squares fit. (d) How one reconstruction is judged, on a real MedMCQA question: the model sees only the answer choices, and the judge compares its question with the hidden original.

Figure 6: Accuracy on the questions each model can and cannot reconstruct, in the multiple-choice and generative settings. Filled markers are questions the model reconstructed, hollow markers those it could not. The length of each segment is the difference between the two pools. Questions from MedQA-USMLE, MedMCQA and MedBullets are pooled.

Figure 7: Accuracy against inference cost per thousand questions, one point per model, on a logarithmic cost axis. (A) Multiple choice, on all four benchmarks. (B) Answer choices withheld, on MedQA-USMLE, MedMCQA and MedBullets, with Setting 2 cost on an equal-prompt basis. The dashed line joins the models on the cost-accuracy frontier. The horizontal line in (A) is the human-expert baseline weighted by benchmark size over MedQA-USMLE, MedMCQA and PubMedQA, the benchmarks that have one.

Model	Setting 1: multiple choice			Setting 2: choices withheld
Model	Tokens	Cost ($)	Cost per 1,000	Tokens	Cost ($)	Cost per 1,000
			correct answers ($)			correct answers ($)
Qwen 3.7 Plus	8.06M	5.92	2.18	5.03M	5.26	3.36
GPT-5.6 Luna	4.97M	8.89	3.29	1.26M	3.80	2.24
DeepSeek V4 Pro	5.80M	7.60	2.85	2.55M	4.84	3.13
DeepSeek V4 Flash	6.00M	0.84	0.32	3.26M	0.65	0.43

Table 4: Recorded inference cost in both answering settings, ordered by multiple-choice accuracy. Setting 1 covers all 3,076 questions of the four benchmarks, and Setting 2 covers the 2,576 MedQA-USMLE, MedMCQA and MedBullets questions. Tokens are prompt plus completion. The final column of each block divides cost by accuracy, giving the price of obtaining a thousand correct answers. † The two MedGemma models were served through a different provider (Dr7.ai).

Figure 8: Majority-vote accuracy against committee size. At each size

k

the best-scoring of the possible committees is shown. The dashed line in each panel is that pool’s own best single model; shading marks where voting gains on it and where it loses. All 3,076 questions of the four benchmarks are pooled.

Benchmark	Questions	Solved by all 12	Missed by all 12
MedQA-USMLE	1,273	668 (52%)	12 (0.9%)
MedMCQA	1,000	395 (40%)	48 (4.8%)
PubMedQA	500	286 (57%)	42 (8.4%)
MedBullets	303	99 (33%)	5 (1.7%)
Overall	3,076	1,448 (47%)	107 (3.5%)

Table 5: Question difficulty by benchmark: questions solved by all twelve models versus missed by all twelve.

Figure 9: Distribution of question difficulty across the 3,076 questions of the four benchmarks. Each bar shows a number of questions that the twelve models answered correctly. The right-hand axis represents the same bars as a percentage of the full question set.

Benchmark	Question (abbreviated)	Key	Consensus
MedMCQA	“In mandibular primary second molar the true statement is…”, with choices including “All buccal cusp are fo same size $>$ all lingual cusp are of same size”	B	A (10/12)
PubMedQA	“Longer-term quality of life and outcome in stroke patients: is the Barthel index alone an adequate measure of outcome?”	yes	no (12/12)
PubMedQA	“Antral follicle assessment as a tool for predicting outcome in IVF—is it a better predictor than age and FSH?”	maybe	yes (12/12)
MedBullets	“A 43-year-old man presents to the emergency department with bright red blood in his stool this morning … admits having ‘too much to drink’ and vomited profusely overnight … Which of the following is the most likely cause of this patient’s symptoms?”	C	B (12/12)
MedBullets	“A 9-hour-old newborn girl is found in the newborn nursery with diffuse swelling of the scalp not present at birth … This patient’s condition most likely affects which of the following spaces or potential spaces?”	C	B (8/12)

Table 6: Example questions missed by all twelve models.

Word overlap (%)
	Generative answers	Reconstructions
Reference	Answer key	Original question
Avg. word length	3.4	82.0
Judged as correct	71.1	18.5
Judged as incorrect	20.8	10.9
AUC [95% CI]	0.835 [0.826, 0.844]	0.631 [0.616, 0.646]

Table 7: Agreement between the LLM judge and two text-similarity measures (word overlap percentange and embedding similarity), over the 30,912 outputs in settings 2 and 3.

Developer	Model pair (smaller $\rightarrow$ larger)	MC	Gen	Recon
OpenAI	GPT-OSS 20B $\rightarrow$ GPT-5.6 Luna	+10.9	+16.0	+10.4
DeepSeek	V4 Flash $\rightarrow$ V4 Pro	+0.0	+1.1	$-0.1$
Qwen	3.5 9B $\rightarrow$ 3.7 Plus	+5.5	+14.3	+12.9
Meta	Llama 3.1 8B $\rightarrow$ Llama 4 Maverick	+18.7	+23.3	+8.0
Mistral	Ministral 8B $\rightarrow$ Mistral Large 3	+10.4	+15.9	+9.8
Google	MedGemma 4B $\rightarrow$ MedGemma 27B	+6.6	+9.7	+2.4

Table S1: Within-developer size comparisons across the three evaluation settings (difference = larger

-

smaller model, percentage points). Paired

t

-tests across the six families: multiple choice

t(5)=3.37

p=.020

; generative

t(5)=4.41

p=.007

; reconstruction

t(5)=3.51

p=.017

. Multiple choice covers all 3,076 questions, and the generative and reconstruction settings the 2,576 questions of MedQA-USMLE, MedMCQA and MedBullets. The last rows give the two added open-weight models of reported size (Supplementary Material S16), which are not included in the tests; Qwen 3.5 35B ran with its reasoning mode off, against the thinking-mode Qwen 3.5 9B in multiple choice.

Model	MedQA-USMLE		MedMCQA		MedBullets		Overall		$\Delta$
Model	MC	Gen	MC	Gen	MC	Gen	MC	Gen	$\Delta$
GPT-5.6 Luna	94.7	76.5	83.8	52.1	89.1	67.3	89.8	65.9	$-23.9$
Qwen 3.7 Plus	94.7	70.2	83.8	48.5	87.8	61.4	89.6	60.7	$-28.9$
DeepSeek V4 Pro	94.4	69.1	82.4	48.1	87.5	61.1	88.9	60.0	$-28.9$
DeepSeek V4 Flash	94.0	68.9	81.8	46.3	88.4	58.7	88.6	58.9	$-29.7$
Llama 4 Maverick	89.9	60.8	79.4	44.7	79.2	51.2	84.5	53.4	$-31.2$

Table S2: Multiple-choice (MC) and generative (Gen; answer choices withheld) accuracy (%) on MedQA-USMLE, MedMCQA and MedBullets, which underpin the open-ended claim. “Overall” and

\Delta

are example-weighted over the 2,576 MedQA-USMLE, MedMCQA and MedBullets questions (MedBullets in its four-choice version), with

\Delta=\text{Gen}-\text{MC}

computed from unrounded values. PubMedQA is excluded because its yes , no , or maybe answers make the generative setting non-open-ended. The last rows give the two added open-weight models of reported size (Supplementary Material S16), which are not included in the means or tests.

Model	MedQA-USMLE	MedMCQA	MedBullets	Overall	95% CI
Qwen 3.7 Plus	30.27	21.50	29.37	26.76	[25.1, 28.5]
GPT-5.6 Luna	29.12	21.10	28.05	25.88	[24.2, 27.6]
DeepSeek V4 Flash	24.03	17.90	20.13	21.18	[19.6, 22.8]
Mistral Large 3	24.37	15.20	25.08	20.89	[19.4, 22.5]
DeepSeek V4 Pro	22.24	17.70	27.39	21.08	[19.5, 22.7]
Llama 4 Maverick	18.10	14.50	22.85	17.26	[15.8, 18.8]

Table S3: Question-reconstruction fidelity (%). Reconstruction is defined for MedQA-USMLE, MedMCQA and MedBullets (four-choice version). PubMedQA is excluded because its yes , no , or maybe choices are identical across questions and carry nothing to reconstruct. Brackets give 95% confidence intervals on the overall rate, which pools MedQA-USMLE, MedMCQA and MedBullets. The last rows give the two added open-weight models of reported size (Supplementary Material S16), which are not included in the means or tests.

Model	Reconstruction when correct	Reconstruction when wrong	Difference
Qwen 3.7 Plus	28.1	15.4	+12.7
GPT-5.6 Luna	26.9	17.1	+9.8
DeepSeek V4 Flash	22.4	12.0	+10.4
Mistral Large 3	22.1	14.1	+8.0
DeepSeek V4 Pro	21.9	14.7	+7.1
GPT-OSS 20B	17.5	8.8	+8.7

Table S4: Question-reconstruction fidelity (%) split by whether the model answered the multiple-choice question correctly. Reconstruction is defined for MedQA-USMLE, MedMCQA and MedBullets, which are pooled. The last rows give the two added open-weight models of reported size (Supplementary Material S16), which are not included in the means or tests.

Model	Un-rec.	% of	Multiple choice			Generative
Model	(n)	set	un-rec.	rec.	diff.	un-rec.	rec.
GPT-5.6 Luna	1,905	74.1	88.6	93.2	+4.7	63.1	74.0
Qwen 3.7 Plus	1,886	73.2	88.0	94.0	+6.0	58.1	68.1
DeepSeek V4 Pro	2,029	78.9	88.0	92.3	+4.2	57.1	70.7
DeepSeek V4 Flash	2,024	78.8	87.3	93.6	+6.3	56.7	67.3
Mistral Large 3	2,037	79.1	83.6	89.8	+6.2	53.3	71.6

Table S5: Performance on each model’s un-reconstructable pool: the MedQA-USMLE, MedMCQA and MedBullets questions it could not reproduce from the answer choices. Multiple-choice (MC) and generative (Gen) accuracy (%) are given separately for the un-reconstructable (un-rec.) and reconstructable (rec.) pools. The last rows give the two added open-weight models of reported size (Supplementary Material S16), which are not included in the means or tests.

Full twelve-model pool and six-larger-model pool
Approach	MedQA-USMLE	MedMCQA	PubMedQA	MedBullets	Overall
Best single model in both pools (Qwen 3.7 Plus)	94.7	83.8	81.6	87.8	88.3
Majority vote (12 models)	93.9	83.4	82.2	88.1	88.0
Larger-6 majority vote	95.1	85.3	80.8	89.1	89.0
Six-smaller-model pool (smaller model per developer)
Best single smaller model (DeepSeek V4 Flash)	94.0	81.8	77.8	88.4	86.8

Table S6: Majority-vote accuracy (%) against the best single model, for the full twelve-model pool and for the pools built from the larger and the smaller model of each developer family. Votes are cast on answer letters; a tie is broken by the highest-ranked member voting for one of the tied letters. Overall pools all 3,076 questions of the four benchmarks. Single-model accuracy for all twelve models is in Table 3 . The last rows give the vote of all fourteen models and of the six larger models with GPT-OSS 120B and Qwen 3.5 35B in place of GPT-5.6 Luna and Qwen 3.7 Plus (Supplementary Material S16); the best single model of the fourteen is still Qwen 3.7 Plus.

Figure S1: The prompt issued in each of the three settings. Braces mark fields substituted per question; role labels indicate the message sequence sent to the provider.

Figure S2: The two grading prompts used by the judge. Braces mark fields substituted per judgment.

Figure S3: A complete model output under Setting 1, with the question that produced it. The numbered steps are the model’s own formatting.

Figure S4: The smaller and larger model from one provider on the same question, with the keyed answer marked. Both outputs are reproduced in full.

Figure S5: Three questions answered incorrectly by all twelve models. The recorded key and the answer the models converged on are both marked; the note gives the pattern each question illustrates.

Output	Measure	Chance	AUC by model	Same question [95% CI]
Answers	Word overlap	3.9	.79–.88	41.6 [39.8, 43.3]
	Embedding	0.100	.84–.88	0.345 [0.333, 0.357]
Reconstructions	Word overlap	5.6	.59–.66	5.8 [5.2, 6.5]
	Embedding	0.209	.70–.76	0.085 [0.079, 0.091]

Table S7: Additional results for Table 7 , over all outputs on the 2,576 MedQA-USMLE, MedMCQA and MedBullets questions. Chance: mean similarity when the same outputs are scored against a random other question of the same benchmark (word overlap in percent, embedding similarity as cosine). AUC by model: lowest and highest AUC across the twelve models. Same question: accepted minus rejected among different models answering the same question, with a 95% interval from resampling questions.

Questions	n	First pass	Kept	Wrong	Outdated	Alternative	Ambiguous
Leading models disagree	445	68	60	4	2	15	39
Missed by all twelve	107	68	65	18	1	22	24
Solved by all twelve	60	0	0	0	0	0	0

Table S8: Audit of answer keys. First pass: questions the auditor flagged. Kept: flags that stood after the second review, split by type.

Claim	$p$	Holm	BH	Clustered 95% CI
Larger vs smaller, multiple choice (F2)	.020	.068	.024	[8.0, 9.3]
Larger vs smaller, MedBullets (F2)	.010	.049	.014	[11.0, 15.9]
Drop without choices (F3)	$<$ .0001	$<$ .0001	$<$ .0001	[-32.6, -30.1]
Reconstruction vs accuracy, $r$ (F4)	.0002	.002	.0005	[0.83, 0.90]
Reconstruction higher if answered (F4)	$<$ .0001	$<$ .0001	$<$ .0001	[6.0, 9.6]
Accuracy higher if reconstructed, MC (F5)	$<$ .0001	$<$ .0001	$<$ .0001	[5.8, 9.0]

Table S9: The 13 confirmatory tests with multiplicity-adjusted

p

-values and question-clustered 95% intervals for the statistic reported in the text (points, or

r

). F: finding.

Figure S6: Figure 3 with GPT-OSS 120B and Qwen 3.5 35B as the larger models of OpenAI and Qwen, in place of GPT-5.6 Luna and Qwen 3.7 Plus. Multiple-choice accuracy on all 3,076 questions of the four benchmarks, with each provider’s smaller and larger model side by side. The dashed line is the human-expert reference of 86.5%.

Figure S7: Figure 4 with GPT-OSS 120B and Qwen 3.5 35B in place of GPT-5.6 Luna and Qwen 3.7 Plus. Accuracy on MedQA-USMLE, MedMCQA and MedBullets pooled (2,576 questions) in multiple choice (Setting 1) and the generative setting (Setting 2), one row per model; the segment gives the difference in points.

Figure S8: Three items from the expert review of the LLM judge, as shown to the expert, with the expert’s ratings. Examples 1 and 2 are an acceptance and a rejection the expert agreed with; Example 3 is the one acceptance the expert rated borderline. Model answers are shown in full.

Medical LLMs

Medical large language models are AI systems adapted to clinical and biomedical data for health reasoning, information extraction, and communication.

See all 416 papers

Jun 10, 2026

Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance?

Large language models (LLMs) in medicine are mainly evaluated using multiple-choice question answering (MCQA), which can overestimate real clinical ability due to guessing strategies and answer biases. To address these limitations, we introduce an expanded and more challenging benchmark based on Polish medical exams, adding over 15,000 questions, two new domains, and four structural modifications that reduce MCQA-specific artifacts and better test reasoning. We evaluate 21 LLMs and show that evaluation design strongly affects results. Under our harder setup, the best model (Qwen3.5-122B) drops by 28.4 and 31 pp on English and Polish exams, respectively. Despite low evidence of data contamination, standard MCQA scores do not reliably reflect true medical competence. To facilitate further research, we make our benchmark publicly available.

Antoni Lasik, Jakub Pokrywka, Łukasz Grzybowski +7

NASK National Research Institute · Adam Mickiewicz University · ARAAI Poland +4

LLM Evaluation LLM Reliability Medical QA Language Model Bias Evaluation Clinical Language Model Evaluation

Ask PDF

Apr 12, 2026

Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework

Incorporating large language models (LLMs) in medical question answering demands more than high average accuracy: a model that returns substantively different answers each time it is queried is not a reliable medical tool. Online health communities such as Reddit have become a primary source of medical information for millions of users, yet they remain highly susceptible to misinformation; deploying LLMs as assistants in these settings amplifies the need for output consistency alongside correctness. We present a practical, open-source evaluation framework for assessing small, locally-deployable open-weight LLMs on medical question answering, treating reproducibility as a first-class metric alongside lexical and semantic accuracy. Our pipeline computes eight quality metrics, including BERTScore, ROUGE-L, and an LLM-as-judge rubric, together with two within-model reproducibility metrics derived from repeated inference (N=10 runs per question). Evaluating three models (Llama 3.1 8B, Gemma 3 12B, MedGemma 1.5 4B) on 50 MedQuAD questions (N=1,500 total responses) reveals that despite low-temperature generation (T=0.2), self-agreement across runs reaches at most 0.20, while 87-97% of all outputs per model are unique -- a safety gap that single-pass benchmarks entirely miss. The clinically fine-tuned MedGemma 1.5 4B underperforms the larger general-purpose models on both quality and reproducibility; however, because MedGemma is also the smallest model, this comparison confounds domain fine-tuning with model scale. We describe the methodology in sufficient detail for practitioners to replicate or extend the evaluation for their own model-selection workflows. All code and data pipelines are available at https://github.com/aviad-buskila/llm_medical_reproducibility.

Avi-ad Avraam Buskila

Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel

LLM Evaluation LLM Reliability Clinical Language Model Evaluation Medical LLMs Biomedical QA

Ask PDF

May 2, 2026

Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks

Evaluating large language models (LLMs) for medical applications remains challenging due to benchmark saturation, limited data accessibility, and insufficient coverage of relevant tasks. Existing suites have either saturated, heavily depend on restricted datasets, or lack comprehensive model coverage. We introduce Medmarks, a fully open-source evaluation suite with 30 benchmarks spanning question answering, information extraction, medical calculations, and open-ended clinical reasoning. We perform a systematic evaluation of 61 models across 71 configurations using verifiable metrics and LLM-as-a-Judge. Our results show that frontier reasoning models (Gemini 3 Pro Preview, GPT-5.1, & GPT-5.2) achieve the highest performance across both benchmarks, most frontier proprietary models are significantly more token efficient than open-weight alternatives, medically fine-tuned models outperform their generalist counterparts, and that models are susceptible to answer-order bias (particularly smaller models and Grok 4). A subset of our evals (Medmarks-T) can be directly used as reinforcement learning environments to post-train LLMs for medical reasoning. Code is available at https://github.com/MedARC-AI/Medmarks

Benjamin Warner, Ratna Sagari Grandhi, Max Kieffer +32

Sophont · MedARC · Prime Intellect

LLM Evaluation Healthcare Clinical Language Model Evaluation Medical LLMs

Ask PDF

Abstract

Medical LLMs

Explore similar work

Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance?

Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework

Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks