MedPsy: State-of-the-Art Small Medical Language Models for Efficient Edge Deployment

Davide Vitabile; Alexandro Buffa; Akshay Nambiar; Amril Nazir

cs.CLNov 1, 2026

MedPsy: State-of-the-Art Small Medical Language Models for Efficient Edge Deployment

Authors: Davide Vitabile, Alexandro Buffa, Akshay Nambiar, Amril Nazir

Organizations: Tether Data, S.A. de C.V. d.b.a. Tether AI Research

Abstract

Medical language models promise expert clinical support, but the strongest open-source LLMs remain too large for private edge-device deployment, while compact medical models still trail substantially on knowledge-intensive and real-world healthcare tasks. Reasoning models that fit edge memory budgets can produce long reasoning traces, increasing sequential decoding, end-to-end latency, energy consumption, and KV-cache memory pressure. We present MedPsy, a family of text-only 1.7B and 4B medical language models designed for edge deployment, targeting this accuracy-generation-efficiency trade-off. Our recipe combines a synthetic medical data pipeline over biology, medicine, and health seeds with chain-of-thought targets from a 235B medical-focused LLM teacher; a four-stage post-training curriculum comprising two SFT stages followed by two RL stages with hard-sample mining; and an edge deployment evaluation spanning multiple GGUF quantization variants and real-device inference on laptop and smartphone hardware. On seven closed-ended medical benchmarks, MedPsy-4B surpasses MedGemma-1.5-4B by 19.34 points and matches MedGemma-27B while being 6.75x smaller; MedPsy-1.7B outperforms MedGemma-1.5-4B by 11.42 points despite being less than half its size. On HealthBench-Hard, MedPsy-4B surpasses MedGemma-27B by 15.33 points, while MedPsy-1.7B surpasses it by 11.66 points despite being ~16x smaller. Beyond accuracy, MedPsy reduces average response length by 1.7x (1.7B) and 3.2x (4B) versus its Qwen3 backbones, while Q4_K_M quantization preserves the aggregate score within 1 point of BF16 with ~69% lower disk footprint. In real-device inference, MedPsy-4B's shorter generations translate into 75% lower median end-to-end latency and estimated energy per answer than its Qwen3 backbone at matched quantization, while peak RAM remains below 5 GB across our tested laptop and smartphone configurations.

Figures & tables

Panel of Figure 1 — Figure 1: Efficiency along both deployment axes. MedPsy models deliver frontier-level medical accuracy per parameter (a) and per gigabyte of quantized footprint (b) .

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Group	Source / Role	Rows	License / Note
Corpus 1: Broad SFT & Seeds
Seeds	Genesis III (Biology/Medicine)	8,089,891	CC-BY-NC-4.0
Seeds	FineFineWeb (Health)	2,133,642	Apache-2.0
SFT-1	Synthetic Broad Med Corpus	10,223,533	Inherited
Corpus 2: Focused SFT
SFT-2	AfriMedQA MCQ Train	124	CC-BY-4.0

Appendix

Table 5: Final Data Mix and Source Licenses. SFT row counts represent final selected input for assistant target generation; RL counts describe the AlphaMed19K prompt pool used for DAPO.

Teacher	CompassJudger	Llama-3.3-70B	GPT-OSS-120B
Baichuan-M3-235B	77	71	58
GPT-OSS-120B	69.67	63	52.33
Fleming-R1-32B	67	61	45.67

Appendix

Table 7: Teacher HealthBench overall scores across three graders.

SFT target source	Closed-ended avg	HealthBench
Fleming-R1-32B only	63.43	59.5
Baichuan-M3-235B only	63.02	71.0
Both teachers, mixed	63.40	60.0

Appendix

Table 8: Student-level teacher ablation (4B students, identical 170K prompts, varying only the SFT target source). Closed-ended averages differ by at most 0.41 points, but HealthBench separates the arms by 11 points; the mixed model stays near Fleming-only rather than at the 65.25 midpoint of the single-teacher arms.

Model	CompassJudger	Llama-3.3-70B	GPT-OSS-120B
Qwen3-4B-Thinking-2507	63	56	36.67
Qwen3-4B (Thinking)	62	55	37.67
gemma-3-4b-it	59	53	33.33
SmolLM3-3B	50	43.67	24
Llama-3.2-3B-Instruct	37.33	32	14.67

Appendix

Table 11: 4B-class backbone HealthBench results across three graders.

Model	CompassJudger	Llama-3.3-70B	GPT-OSS-120B
Qwen3-1.7B (Thinking)	53	47.33	27.67
LFM2.5-1.2B-Thinking	49	41.67	22.33
Llama-3.2-1B-Instruct	25	22	5.33
SmolLM2-1.7B-Instruct	23	18.67	6

Appendix

Table 13: Sub-2B-class backbone HealthBench results across three graders.

Parameter	Qwen3-1.7B	Qwen3-4B
Hidden size	2,048	2,560
FFN hidden size	6,144	9,728
Layers	28	36
Attention heads	16	32
KV groups (GQA)	8	8
Vocab size	151,936	151,936

Appendix

Table 15: Architecture summary of the chosen Qwen3 backbones.

Hyperparameter	S1 (1.7B)	S1 (4B)	S2 (1.7B)	S2 (4B)
Asymmetric clip $(\epsilon_{\text{low}},\epsilon_{\text{high}})$	$(0.20,\;0.28)$
Entropy coefficient	$0$
Optimizer / precision	AdamW, bf16 mixed
Learning rate	$1\!\times\!10^{-6}$ (constant)
Train batch (prompts)	$512$
PPO mini-batch	$128$

Appendix

Table 16: RL training hyperparameters (DAPO).

Benchmark	MedPsy -4B outcomes			Pairwise score
	Win	Tie	Loss	MedPsy -4B	MedGemma-27B-text-it
AfriMedQA	640	10,258	832	49.1816%	50.8184%
MMLU-Health	147	2,944	176	49.5562%	50.4438%
MedQA-USMLE	325	3,209	285	50.5237%	49.4763%
PubMedQA	123	1,298	79	51.4667%	48.5333%
MedMCQA	1,155	10,164	1,230	49.7012%	50.2988%

Appendix

Table 22: Matched-instance pairwise scores by benchmark, averaged over three runs. W/T/L counts are summed across runs and shown from the MedPsy -4B perspective.

Format	$\Delta$ Score 4B	$\Delta$ Score 1.7B	1.7B / 4B ratio
Q8_0	$-$ 0.15	0.00	—
Q4_K_M	$-$ 0.81	$-$ 0.73	$\approx$ 1 $\times$
IQ4_NL	$-$ 1.02	$-$ 1.70	1.7 $\times$
IQ4_XS	$-$ 1.08	$-$ 1.79	1.7 $\times$
IQ3_M	$-$ 1.50	$-$ 3.58	2.4 $\times$
IQ3_XXS	$-$ 5.56	$-$ 12.46	2.2 $\times$

Appendix

Table 25: Per-format AVG Score changes by model size (absolute change vs. BF16, on the 0–100 scale).

Model	Peak RAM	tok/s	tokens/answer	TTFT	time/answer	energy/answer
MedPsy-1.7B	2.22 GB	15.27	789	8.90 s	55.6 s	420 J
MedPsy-4B	3.98 GB	6.63	422	15.04 s	105.4 s	508 J

Appendix

Table 31: OnePlus 13 local-inference measurements (three MedQA prompts per model). Both models use Q4_K_M GGUF builds with Vulkan partial offload ( -ngl 20 ) and six CPU threads. Energy is from the Android battery charge counter (unplugged device at 49% charge, Wi-Fi debugging).

Figure 2: Accuracy and cap-hit rate under explicit thinking-token caps. Left: seven-task closed-ended accuracy at thinking budgets of 300–1,500 tokens; dashed lines mark unbounded generation. Right: fraction of traces reaching the cap. MedPsy -4B degrades gracefully and rarely reaches the cap beyond a 600-token budget, whereas Qwen3-4B-Thinking-2507 hits the cap on 99% of traces at 300 tokens and is still cap-bound on over half of traces at 1,500.

Model/quant	Decode @512	Decode @8K	Peak RAM @8K	TTFT @512
MedPsy -1.7B Q4_K_M	164 tok/s	109 tok/s	3.16 GB	0.06 s
MedPsy -4B IQ3_M	91 tok/s	66 tok/s	4.32 GB	0.12 s
MedPsy -4B Q4_K_M	89 tok/s	63 tok/s	4.83 GB	0.12 s

Appendix

Table 33: Apple M5 Pro context-length sweep (llama.cpp, Metal): decode throughput at 512-token and 8K-token inputs, peak RAM at 8K, and TTFT at 512 tokens.

Model	Task 1	Task 2
MedPsy -4B	0.855	0.875
MedGemma-27B-text-it	0.828	0.802
MedPsy -1.7B	0.590	0.670
MedGemma-1.5-4B-it	0.565	0.364

Appendix

Table 35: LongHealth accuracy on long clinical documents (5,090–6,754 words). Best per size class in bold.

Figure 3: Training-time mean response length (top) and avg accuracy (bottom) at the best-accuracy checkpoint of each RL variant. MedPsy S2 (orange) wins on both axes at both model sizes.

Figure 4: Training-time mean response length over training, traces truncated at the best-accuracy checkpoint. MedPsy S2 (orange) traces the full pipeline: Easy/Moderate Stage 1 compresses the policy from $\sim 1.8$ – $2.0$ K tokens down to $\sim 620$ – $760$ tokens, then mined-hard Stage 2 (after the dotted vertical line) converges to $\sim 580$ – $870$ tokens. The Difficult-only S1 ablation (green) reaches its best-accuracy checkpoint very early at $\sim 2.4$ K tokens (diamond marker), and only matches MedPsy S2’s response length late in training, after losing $6$ – $9$ pp of accuracy in the process. No overlong (red) plateaus much higher; Vanilla GRPO (gray) barely moves. Diamond markers show the best-accuracy checkpoint values for MedPsy S2 and Difficult-only S1 .

Figure 5: Token–accuracy Pareto across all RL variants at their best-accuracy checkpoints (upper-left is better; MedPsy S2 is the orange diamond). Response length on the x-axis is the training-time mean at the same checkpoint used for the accuracy on the y-axis. MedPsy S2 is the unique Pareto-dominant point at both model sizes: shortest responses and highest accuracy.

Medical LLMs

Medical large language models are AI systems adapted to clinical and biomedical data for health reasoning, information extraction, and communication.

See all 416 papers

Jul 29, 2026

MedLLM: An Open Medical Language Model at the Sub-Billion Scale

Open medical language models have converged on a single scale: every widely used system runs at 7B parameters or more, leaving the sub-billion regime uncharacterized. We present MedLLM, an open 0.1B-parameter medical language model trained through a fully open three-phase pipeline: general pretraining with curriculum sequence-length scheduling, domain fine-tuning on MedFineWeb, a reference-guided medical corpus we release that is selected from general web data by embedding similarity to medical question-answering (QA) data, and preference-aligned fine-tuning combining SFT with direct preference optimization (DPO). Across medical benchmarks, MedLLM shows a pattern visible only at sub-billion scale: medical competence does not degrade uniformly under compression but splits by task type. On context-grounded QA it comes within

2.9

pp of a medically adapted 7B model and surpasses the instruction-tuned and general-purpose 7B baselines; on knowledge-recall QA it stays near the task floor on clinical-vignette MedQA yet significantly exceeds every 7B and sub-7B baseline on MedMCQA, indicating that where recall fails the constraint is model capacity rather than adaptation. This dissociation is masked at 7B, where both capabilities are present, and surfaces only when capacity is scarce.

Maxx Richard Rahman, Asim Ahmed, Mihan Mohagheghzadeh +1

German Research Center for Artificial Intelligence (DFKI), Germany · Saarland University, Germany

LLM Fine-Tuning Small Language Models Medical QA Medical LLMs

Ask PDF

Oct 30, 2026

From Small to Large: Assessing Medical Reasoning, Benchmark Contamination, and Model Size in Language Models

Large language models (LLMs) now answer medical exam questions at or above expert level, but it is unclear whether such scores reflect clinical competence, or are instead affected by the format of the questions being asked (e.g., multiple-choice versus open-ended), prior exposure to the benchmarks, and the difficulty of the questions. These benchmarks are also public and widely used online, which may be resulting in LLM memorization. Models may perform better on questions that they have seen before. It is also not evident whether smaller models, which are more locally deployable in HIPAA-compliant hospital settings, have achieved human performance. In this study, we evaluate fourteen current models from six leading providers, one smaller and one larger model from each provider with two additional open-weight models, on 3,076 questions from four medical benchmarks (MedQA-USMLE, MedMCQA, PubMedQA, and MedBullets) under standard multiple choice, the same questions with the answer choices withheld, and reconstruction of a hidden question from its choices alone. In the standard multiple-choice setting, the strongest models exceed the human-expert baseline on MedQA-USMLE and PubMedQA, but the smaller models trail their larger models by 8.7 percentage points and only two of the six pass the human-expert bar on the licensing examination. Withholding the answer choices, however, makes every model struggle: mean accuracy falls from 82.3% to 50.9%, and the two largest drops belong to the medically specialised models. On memorization, the strongest models reconstruct one question in four from the answer choices alone, yet accuracy holds on the questions they cannot reconstruct. We further weigh accuracy against inference cost and test whether ensembling several models improves medical question answering.

Navapat Nananukul, Mayank Kejriwal

Information Sciences Institute, University of Southern California, Marina del Rey, CA, USA · Lead contact

Medical LLMs Medical QA Clinical Language Model Evaluation Benchmark Contamination Language Model Scaling

Ask PDF

Apr 12, 2026

Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework

Incorporating large language models (LLMs) in medical question answering demands more than high average accuracy: a model that returns substantively different answers each time it is queried is not a reliable medical tool. Online health communities such as Reddit have become a primary source of medical information for millions of users, yet they remain highly susceptible to misinformation; deploying LLMs as assistants in these settings amplifies the need for output consistency alongside correctness. We present a practical, open-source evaluation framework for assessing small, locally-deployable open-weight LLMs on medical question answering, treating reproducibility as a first-class metric alongside lexical and semantic accuracy. Our pipeline computes eight quality metrics, including BERTScore, ROUGE-L, and an LLM-as-judge rubric, together with two within-model reproducibility metrics derived from repeated inference (N=10 runs per question). Evaluating three models (Llama 3.1 8B, Gemma 3 12B, MedGemma 1.5 4B) on 50 MedQuAD questions (N=1,500 total responses) reveals that despite low-temperature generation (T=0.2), self-agreement across runs reaches at most 0.20, while 87-97% of all outputs per model are unique -- a safety gap that single-pass benchmarks entirely miss. The clinically fine-tuned MedGemma 1.5 4B underperforms the larger general-purpose models on both quality and reproducibility; however, because MedGemma is also the smallest model, this comparison confounds domain fine-tuning with model scale. We describe the methodology in sufficient detail for practitioners to replicate or extend the evaluation for their own model-selection workflows. All code and data pipelines are available at https://github.com/aviad-buskila/llm_medical_reproducibility.

Avi-ad Avraam Buskila

Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel

LLM Evaluation LLM Reliability Clinical Language Model Evaluation Medical LLMs Biomedical QA

Ask PDF

Abstract

Medical LLMs

Explore similar work

MedLLM: An Open Medical Language Model at the Sub-Billion Scale

From Small to Large: Assessing Medical Reasoning, Benchmark Contamination, and Model Size in Language Models

Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework