Perturbation-Based Data Selection for Medical Question Answering

Yuexin Wu; Dayou Yu; Vasile Rus

cs.CLOct 24, 2026

Perturbation-Based Data Selection for Medical Question Answering

Authors: Yuexin Wu, Dayou Yu, Vasile Rus

Organizations: Department of Computer Science, University of Memphis, Memphis, TN, USA

Abstract

Selecting a compact training subset is an important step in adapting medical language models from existing question-answering banks. We investigate perturbation-based data selection using two signals derived from Gaussian perturbations of the final-block representation: prediction-change frequency and non-flipping margin contraction. After a shared entropy prefilter, Representation-Sensitivity Prioritization (RSP) combines the signals using automatic root-mean-square scaling, while RSP-Drop ranks candidates by contraction alone. Both selectors use only questions and answer options during selection. We evaluate them with MedGemma-4B on five medical QA datasets at retained-set budgets of 64, 128, and 256 examples. At the primary budget of 128, RSP and RSP-Drop obtain dataset-averaged Macro-F1 scores of 47.55 and 48.28, respectively, compared with 45.11 for entropy-pool-matched random selection. Their relative performance varies across datasets and budgets. A selected-set case study identifies contrasting clean-margin profiles among the examples exchanged by the two rules, while a full-pool ablation shows that entropy prefiltering is not uniformly beneficial. These retrospective results characterize the empirical utility and limitations of perturbation-based selection under a fixed adaptation protocol. Evaluation uses one backbone and benchmarks with documented training-family overlap; it does not establish clinical effectiveness.

Figures & tables

Panel of Figure 1 — Fig. 1: Shared protocol for RSP and RSP-Drop . Both selectors are label- and auxiliary-text-blind; auxiliary text enters only after selection, as LoRA training context.

Dataset	Answer format	Pool	Dev	Test
AfriMed-QA	5 options	1,500	559	892
MedExpQA	4/5 options	434	63	121
MedExQA	4 options	600	164	200
PubMedQA	Yes/no/maybe	600	200	200
MedMCQA	4 options	512	256	4,096

TABLE I: Datasets and fixed study partitions. The MedExpQA test count excludes four pool-overlap questions.

$B$	Method	Training seconds	Input tokens $(10^{3})$	Optimizer updates
64	RSP	43.55	32.47	16
64	Random@ $4B$	43.48	34.45	16
128	RSP	86.72	64.42	NA
128	Random@ $4B$	87.30	69.41	NA
256	RSP	172.86	130.96	64
256	Random@ $4B$	172.78	137.58	64

TABLE II: Resource inventory: mean per run across five datasets and three seeds (15 runs per row). Training time is measured; optimizer updates are measured where available; input tokens are reconstructed.

Question/options-only controls
Method	AfriMed-QA	MedExpQA	MedExQA	PubMedQA	MedMCQA	Dataset-macro avg.
Random@ $4B$	49.36 / 46.04	51.79 / 49.64	50.33 / 49.64	69.50 / 48.58	36.09 / 31.68	51.42 / 45.11
Entropy	50.34 / 47.19	51.24 / 49.00	48.67 / 48.60	70.00 / 50.74	38.00 / 33.52	51.65 / 45.81
Margin	51.49 / 49.34	52.07 / 48.63	48.67 / 48.54	69.00 / 49.17	38.48 / 34.72	51.94 / 46.08
BALD	53.33 / 50.86	50.96 / 46.99	48.67 / 47.83	69.17 / 48.79	37.64 / 35.64	51.95 / 46.02
BatchBALD	49.96 / 46.66	50.14 / 45.88	49.50 / 49.19	71.67 / 54.34	37.65 / 35.00	51.78 / 46.21

TABLE III: Overall performance at

B=128

: test Accuracy / Macro-F1 (%, three-seed means). Dataset-macro avg. is the unweighted mean across the five datasets, computed before rounding. Bold marks the largest mean separately for each metric and column, not statistical significance. Seed-level dataset averages are in Table IV .

Method	Seed 13	Seed 23	Seed 37
Random@ $4B$	51.60 / 45.56	50.23 / 43.89	52.42 / 45.90
Entropy	51.24 / 45.46	53.29 / 47.93	50.41 / 44.04
Margin	52.35 / 46.78	50.65 / 45.16	52.83 / 46.31
BALD	52.36 / 45.55	51.36 / 45.62	52.13 / 46.90
BatchBALD	53.90 / 50.06	50.86 / 43.79	50.59 / 44.79
Noise-Stability	49.48 / 44.16	51.32 / 45.17	51.63 / 45.41

TABLE IV: Primary-budget dataset averages by branch seed: Accuracy / Macro-F1 (%),

B=128

. These are seed-level summaries, not additional runs.

Exclusive group	$n$	$F_{i}$	$D_{i}$	Mean clean margin
RSP only	26	1	0	0.098
RSP-Drop only	26	0	$>0$	0.707

TABLE V: Selected-set comparison on MedMCQA,

B=128

, seed 13. Statistics concern each selector’s exclusive examples, not its complete selected set.

$B$	Comparison	Accuracy	Macro-F1
64	RSP – Random@ $4B$	$+1.12\!\pm\!1.22$	$+0.60\!\pm\!1.52$
64	RSP-Drop – Random@ $4B$	$+1.02\!\pm\!1.18$	$+1.22\!\pm\!0.89$
64	RSP-Drop – RSP	$-0.10\!\pm\!0.71$	$+0.62\!\pm\!0.69$
128	RSP – Random@ $4B$	$+1.58\!\pm\!0.99$	$+2.44\!\pm\!0.96$
128	RSP-Drop – Random@ $4B$	$+2.18\!\pm\!0.90$	$+3.17\!\pm\!0.95$
128	RSP-Drop – RSP	$+0.60\!\pm\!0.25$	$+0.73\!\pm\!0.98$

TABLE VI: Paired dataset-macro differences from unrounded retained metrics (percentage points; mean

\pm

sample SD across seeds 13/23/37). Each comparison is first method minus second.

$B$	Method	AfriMed-QA	MedExpQA	MedExQA	PubMedQA	MedMCQA	Dataset-macro avg.
64	RSP	48.43 / 43.52	45.73 / 41.35	44.50 / 44.37	68.17 / 46.75	35.42 / 31.06	48.45 / 41.41
64	RSP-Drop	47.83 / 44.13	48.21 / 45.29	44.17 / 44.16	65.17 / 44.39	36.39 / 32.19	48.35 / 42.03
64	Random@ $4B$	48.13 / 44.12	45.45 / 42.34	43.00 / 42.70	64.00 / 43.18	36.07 / 31.73	47.33 / 40.81
128	RSP	53.70 / 51.26	51.24 / 49.48	52.50 / 52.31	70.00 / 51.00	37.54 / 33.72	53.00 / 47.55
128	RSP-Drop	52.39 / 50.11	53.72 / 50.82	54.50 / 54.28	68.83 / 49.92	38.56 / 36.28	53.60 / 48.28
128	Random@ $4B$	49.36 / 46.04	51.79 / 49.64	50.33 / 49.64	69.50 / 48.58	36.09 / 31.68	51.42 / 45.11

TABLE VII: Budget sensitivity of RSP, RSP-Drop, and entropy-prefiltered Random: Accuracy / Macro-F1 (%, three-seed means). All three use the same top-

\min(4B,N)

entropy pool within each budget. Bold marks the largest mean among the three methods for each metric, dataset, and budget. Dataset-macro avg. uses equal dataset weight. Paired branch variability is summarized in Table VI .

$B$	Dataset	Method	Top- $4B$ / full pool	Entropy-prefiltered	Full-pool	$\Delta$
64	AfriMed-QA	RSP	256 / 1,500	48.43 / 43.52	46.82 / 42.09	+1.61 / +1.44
		RSP-Drop		47.83 / 44.13	46.79 / 42.53	+1.05 / +1.59
64	MedExpQA	RSP	256 / 434	45.73 / 41.35	46.01 / 42.33	-0.28 / -0.98
		RSP-Drop		48.21 / 45.29	46.28 / 42.32	+1.93 / +2.97
64	MedExQA	RSP	256 / 600	44.50 / 44.37	42.33 / 42.61	+2.17 / +1.76
		RSP-Drop		44.17 / 44.16	44.33 / 43.51	-0.17 / +0.65

TABLE VIII: Entropy-prefilter ablation for RSP and RSP-Drop on settings where top-

4B

is smaller than the full pool. Entries are Accuracy / Macro-F1 (%, three-seed means);

\Delta

is entropy-prefiltered minus full-pool performance in percentage points. Differences are computed before rounding and may therefore differ slightly from subtraction of the displayed means. Separate training runs share seeds and the frozen recipe, but the analysis is descriptive rather than a significance test.

Data Selection

Data selection chooses informative, representative, or reliable examples to improve model training and evaluation while reducing data, compute, or labeling costs.

See all 84 papers

An Empirical Study of Development-Tuned Representation Sensitivity for Medical QA Data Selection
Sep 9, 2026Yuexin Wu, Dayou Yu, Vasile RusData Selection Medical QA
Department of Computer Science, University of Memphis
Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale
Apr 26, 2026Avi-ad Avraam BuskilaRetrieval-Augmented Generation Healthcare
Department of Information Science and Applied Artificial Intelligence, Bar-Ilan University, Ramat-Gan, Israel
Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation
Oct 5, 2026Guangyuan Dong, Ziwei Hong, Xuehao Zhou +6Low-Rank Adaptation Domain Adaptation for LLMs
National University of Singapore, Singapore · University of Pennsylvania, USA · Department of Urology, Shanghai General Hospital, Shanghai, China +3

Sep 9, 2026J/K move · Enter open · S save

An Empirical Study of Development-Tuned Representation Sensitivity for Medical QA Data Selection

Yuexin Wu, Dayou Yu, Vasile Rus

Department of Computer Science, University of Memphis

Data Selection Medical QA Language Model Robustness

Open paper AskPDF arXiv

We study medical question-answering (QA) data selection when a developer already has questions, labels, and auxiliary explanatory texts but can retain only

B

examples for a final LoRA update. Representation-Sensitivity Prioritization (RSP) first restricts the candidate bank to its top-

4B

predictive-entropy examples, then combines the frequency of answer flips with non-flipping contraction of the top-two answer margin under hidden-state perturbations. We evaluate this operational score on five medical QA datasets with google/medgemma-4b-it,

B=128

, and three seed branches. After removing four MedExpQA test questions that also occur in its acquisition pool, dataset-specific development-tuned RSP attains the highest observed five-dataset mean accuracy (53.44%) and Macro-F1 (48.21%) among the evaluated selectors. Relative to random selection within the same entropy-restricted pool, the descriptive differences are +1.49 and +2.10 percentage points, respectively; paired 95% bootstrap intervals include zero. The audit mixes benchmarks with different relationships to documented backbone post-training sources, and the same test cohorts had been used in earlier analyses before the expanded tuning grid was designed. Results are therefore exploratory, modest, and dataset-dependent rather than evidence of statistical superiority. RSP point estimates also exceed an archived no-update reference, but unpinned model revisions and incomplete historical input identifiers limit attribution of that difference to fine-tuning. Because every selected training example includes auxiliary text and no matched answer-only control is evaluated, the experiments also do not identify a causal benefit from auxiliary-text conditioning itself.

Abstract

Data Selection

Explore similar work

An Empirical Study of Development-Tuned Representation Sensitivity for Medical QA Data Selection

Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale

Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation