SAGE: Semantic Anchor-Guided Evolution for Grounded Medical QA Data Synthesis

Chuan Li; Chengyu Wang; Cen Chen; Ye Lyu; Mingyuan Fan; Ming Gao

cs.CLOct 6, 2026

SAGE: Semantic Anchor-Guided Evolution for Grounded Medical QA Data Synthesis

Authors: Chuan Li, Chengyu Wang, Cen Chen, Ye Lyu, Mingyuan Fan, Ming Gao

Organizations: East China Normal University, Shanghai, China · Alibaba Group, Hangzhou, China

Abstract

Developing reliable models for clinical tasks, such as Medical Question Answering (QA), is severely constrained by the limited availability of high-quality, expert-annotated training data. This challenge is exacerbated by stringent privacy requirements and the impracticality of utilizing large open-source corpora or proprietary cloud APIs within resource-limited clinical settings. To address these obstacles, we introduce SAGE (\textit{Semantic Anchor-Guided Evolution}), a novel data synthesis framework that enables small, locally deployed models to generate high-quality medical training data. SAGE leverages lightweight, publicly available taxonomies such as MeSH as semantic anchors, imposing a structured prior to effectively guide and ground the data generation process. At its core, SAGE iteratively interleaves atomic (individual concept-based) and associative (relation-based) synthesis, bootstrapping training data from minimal seeds. This approach eliminates the need for large collections of medical documents or reliance on external APIs, providing a practical solution for on-premises data creation. Extensive experiments across multiple medical question-answering benchmarks demonstrate that models fine-tuned with SAGE-synthesized data consistently outperform those trained using self-derived or conventional document-based paradigms, highlighting tangible improvements in data efficiency and resource utilization for medical LLM development. Code is available at https://github.com/DIaacKr/SAGE.

Figures & tables

Figure 1: Comparison of data synthesis paradigms. Conventional methods either derive data using implicit parametric knowledge or synthesize data from document corpus. In contrast, SAGE synthesizes high-quality medical data via semantic anchor.

Figure 2: Overview of the SAGE framework. MeSH anchors guide dual-track generation, while consistency and diversity checks select QA instances whose projected concepts update the anchor population across epochs.

Open Source Datasets
Model	Method/Dataset	Scale	AVG	MedMCQA	MedQA	PubMedQA	MMLU-Pro	GPQA
							(med)	(med)
Qwen3-4B-Base	Few-shot Prompting	NA	26.15	29.14	29.69	35.20	14.66	22.05
Qwen3-4B NoThinking	NA	NA	62.54	56.78	63.63	71.00	64.63	56.67
Qwen3-4B Thinking	NA	NA	66.18	59.81	72.03	72.60	71.60	54.87
Qwen3-4B-Base	m1k ( 2025a )	1k	63.40	59.26	72.19	74.20	67.49	43.85

Table 1: Performance of models trained on datasets generated or collected by various paradigms, evaluated on mainstream medical QA benchmarks. The highest average score within each paradigm is underlined . The performance of our method is highlighted in bold . Units are in percentage (%).

Figure 3: Comparison between our method and baselines across diverse medical QA benchmarks, including out-of-distribution tasks.

Figure 4: Performance scaling analysis comparing our method with CoT-Self-Instruct across different training data sizes from 1k to 10k samples.

Figure 5: A detailed case on dual-track evolutionary synthesis.

Figure 6: Comparison of different methods across factuality, reasoning, and utility.

Figure 7: Comparison of semantic depth (MeSH Tree) of generated concepts during the generation process.

Figure 8: Comparison of dynamic diversity with increasing generated samples. Diversity is measured by the discovery rate

R=\frac{N_{new}}{B}

, where

N_{new}

denotes the number of unique unseen concept combinations, and

B

is the batch size.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Performance on the benchmark after training Qwen2.5-3B-Instruct and Llama3.2-3B-Instruct.

Strategy	MedNLI	MTS
Wrap	72.08	83.62
CoT-Self-Instruct	84.60	83.16
SAGE	87.27	85.02

Appendix

Table 2: Generalization performance of SAGE compared to baselines. All trainee models are based on Qwen3-4B-Base. We report Accuracy for MedNLI and BERTScore for MTS-Dialog. All results are scaled to percentages (%).

$\sim$ 8B Scale
Model	MedMCQA	MedQA	PubMedQA	MMLU-Pro	GPQA	AVG
				(med)	(med)
SAGE-4B	64.98	70.07	80.80	63.19	61.54	68.12
UltraMedical-8B Zhang et al. (2024)	59.17	71.64	70.30	61.43	50.26	62.56
m1-7B Huang et al. (2025a)	61.49	71.96	74.00	63.65	46.67	63.55
MedReason-8B Wu et al. (2025)	60.17	70.78	78.50	65.08	52.56	65.42

Appendix

Table 3: Cross-scale comparison and performance ceiling on clinical benchmarks. Results are reported in percentages (%). SAGE-4B matches the performance of SOTA 8B medical models and approaches the 72B general model.

Strategy	TruthfulQA $\downarrow$	MedHALT $\downarrow$
Wrap	32.66	90.84
CoT-SI	22.61	91.59
SAGE	20.60	89.28

Appendix

Table 4: Hallucination rates on TruthfulQA (Medical subset) and MedHALT (Fake Question subset). Lower values indicate better factuality. All trainee models are based on Qwen3-4B-Base. “CoT-SI” stands for CoT-Self-Instruct. Results are reported in percentages (%).

Strategy	Data Source	Tokens	MedMCQA	MedQA	PubMedQA	MMLU-Pro	GPQA	AVG
Genie	Scope Notes	1.69M	49.32	51.77	63.90	48.99	35.90	49.98
Genie	Notes + Distilled Q&A	3.51M	46.86	55.15	75.30	52.31	35.13	52.95
Genie	PubMed Abstracts	3.51M	52.38	48.63	61.00	51.21	40.00	50.64
SAGE	Notes + Distilled Q&A	3.51M	64.98	70.07	80.80	63.19	61.54	68.12

Appendix

Table 5: Ablation study on data sources and token budgets. All trainee models are Qwen3-4B-Base. MMLU-Pro and GPQA refer to their respective medical subsets. Results are reported in percentages (%).

Extraction Method	Precision	Recall
Naive String Matching	36.52	55.30
SAGE Mapping (Ours)	55.68	12.21

Appendix

Table 6: Quantitative evaluation of extraction methods against the MeSH on Demand pseudo-gold standard. Results are reported in percentages (%).

Extraction Method	Avg Time / Sample	Privacy	MedMCQA	MedQA	PubMedQA	MMLU-Pro	GPQA	AVG
Naive Matching	1.15s	Offline	63.81	69.60	70.40	61.95	54.36	64.02
MeSH on Demand	48.24s	Online API	66.08	67.16	83.20	62.74	61.54	68.14
SAGE Mapping	1.31s	Offline	64.98	70.07	80.80	63.19	61.54	68.12

Appendix

Table 7: Comparison of extraction methods regarding downstream QA performance, computational overhead, and privacy. MMLU-Pro and GPQA refer to their respective medical subsets. Downstream metrics are reported in percentages (%).

Filtering Strategy	MedMCQA	MedQA	PubMedQA	MMLU-Pro	GPQA	AVG
Contradiction Filtering	64.67	68.03	70.80	60.72	58.46	64.54
SAGE (Consistency + Atomic Gen.)	64.98	70.07	80.80	63.19	61.54	68.12

Appendix

Table 8: Performance comparison between post-hoc contradiction filtering and the SAGE approach. MMLU-Pro and GPQA refer to their respective medical subsets. Results are reported in percentages (%).

Figure 10: A detailed case on self-derived and document-based synthesis.

Model	Method/Dataset	Scale	AVG-5	AVG-10	Lancet	MedBul_op4	MedBul_op5	MedXpert	NEJM
Qwen3-4B-Base	Few-shot Prompting	NA	26.15	24.88	30.58	29.55	24.03	11.18	22.72
Qwen3-4B NoThinking	NA	NA	62.54	54.15	59.47	50.32	48.05	13.04	57.88
Qwen3-4B Thinking	NA	NA	66.18	57.98	62.86	59.09	50.97	13.73	62.19
Open Source Datasets
Qwen3-4B-Base	m1k	1k	63.40	56.98	61.65	59.74	54.87	17.05	59.54
Qwen3-4B-Base	m23k	23k	67.39	61.44	65.05	67.53	61.04	20.50	63.35

Appendix

Table 9: Performance comparison on additional datasets. Since the first five medical datasets are detailed in the main text, this table focuses on the remaining datasets and the overall AVG-10 scores (%).

Figure 11: Distribution chart of token lengths in the test set on Qwen3-4B with thinking mode.

Method	AVG-5	MedMCQA	MedQA	PubMedQA	MMLU-Pro (med)	GPQA (med)
SAGE (thinking)	$67.85\pm 0.26$	$65.11\pm 0.89$	$67.61\pm 0.39$	$82.27\pm 0.81$	$62.48\pm 0.75$	$61.80\pm 1.18$
SAGE (non-thinking)	$68.05\pm 0.22$	$65.26\pm 0.98$	$68.24\pm 1.85$	$82.03\pm 0.98$	$62.21\pm 1.02$	$62.50\pm 2.31$
CoT-Self-Instruct (thinking)	$63.57\pm 0.50$	$62.34\pm 0.13$	$68.34\pm 0.95$	$69.63\pm 0.23$	$60.98\pm 0.55$	$56.58\pm 0.97$
CoT-Self-Instruct (non-thinking)	$63.75\pm 0.10$	$61.47\pm 0.92$	$67.11\pm 0.94$	$74.30\pm 4.07$	$62.26\pm 0.44$	$53.59\pm 3.20$

Appendix

Table 10: Comparison between thinking and non-thinking modes for data synthesis. All results are reported as mean

\pm

standard deviation over three runs. AVG-5 denotes the average score over the five medical benchmarks (%).

Method	AVG-5	MedMCQA	MedQA	PubMedQA	MMLU-Pro (med)	GPQA (med)
SAGE	$68.05\pm 0.22$	$65.26\pm 0.98$	$68.24\pm 1.85$	$82.03\pm 0.98$	$62.21\pm 1.02$	$62.50\pm 2.31$
CSI	$63.75\pm 0.10$	$61.47\pm 0.92$	$67.11\pm 0.94$	$74.30\pm 4.07$	$62.26\pm 0.44$	$53.59\pm 3.20$
WRAP	$49.18\pm 0.95$	$47.81\pm 0.32$	$47.03\pm 0.79$	$66.37\pm 3.10$	$46.40\pm 0.69$	$38.29\pm 1.41$

Appendix

Table 11: Mean accuracy and standard deviation over three independent runs. SAGE shows consistently strong performance with low variance across five medical QA benchmarks (%).

Dataset	Train-Set	SAGE
MedMCQA	$57.42\pm 0.76$	$65.26\pm 0.98$
MedQA	$56.79\pm 1.85$	$68.24\pm 1.85$
PubMedQA	$50.60\pm 1.12$	$82.03\pm 0.98$

Appendix

Table 12: Comparison between fine-tuning on original dataset training splits and fine-tuning on SAGE-generated data. Results are reported as mean accuracy and standard deviation across multiple runs (%).

Figure 12: Distribution chart of token lengths in the test set after training with m23k dataset on Qwen3-4B-Base.

Figure 13: Distribution chart of token lengths in the test set after training the data synthesized by the Wrap method on Qwen3-4B-Base.

Figure 14: Distribution chart of token lengths in the test set after training the data synthesized by our method on Qwen3-4B-Base.

Panel of Figure 15 — Figure 15: The prompt template used for associative generation. Placeholders denote: { terms_str } (concepts associated with the few-shot examples), { q_text } (the content of the few-shot example questions), and { target_terms } (the set of MeSH terms to be incorporated into the new question).

Panel of Figure 16 — Figure 16: The prompt template used for atomic generation. Placeholders denote: { term } (the target medical concept), { scope_note } (the official definition/explanation of the term), and { qual_str } (relevant sub-aspects or qualifiers associated with the term).

Explore similar work

Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation
Apr 29, 2026Guillermo Iglesias, Gema Bello-Orgaz, María Navas-Loro +3Mental Health Large Language Model Evaluation
Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain · Department of Psychiatry, University Hospital Rey Juan Carlos, Mostoles, Spain · Department of Psychiatry, University Hospital Jimenez Diaz Foundation, Madrid, Spain +5
SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning
May 16, 2026Yongfeng Huang, Ruiying Chen, James ChengClinical Reasoning Training Hievi-Rag
CSE, The Chinese University of Hong Kong · Wuhan University of Technology
MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering
May 12, 2026Rezarta Islamaj, Robert Leaman, Joey Chan +13Biomedical Text Clinical Reasoning Training
National Library of Medicine, Division of Intramural Research, Bethesda, MD, US · University of Illinois at Urbana-Champaign, Department of Computer Science, Urbana, IL, US · University of Michigan Medical School, Ann Arbor, Michigan, US

Apr 29, 2026, cs.LGJ/K move · Enter open · S save

Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation

Guillermo Iglesias, Gema Bello-Orgaz, María Navas-Loro, Cristian Ramirez-Atencia+2

Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain · Department of Psychiatry, University Hospital Rey Juan Carlos, Mostoles, Spain · Department of Psychiatry, University Hospital Jimenez Diaz Foundation, Madrid, Spain+5

Mental Health Large Language Model Evaluation Electronic Health Records Text Generation Privacy Fidelity

Open paper AskPDF arXiv

The scarcity of high-quality annotated medical data, particularly in mental health, poses a significant bottleneck for training robust machine learning models. Privacy regulations restrict data sharing, making synthetic data generation a promising alternative. The use of Large Language Models (LLMs) in a data augmentation pipeline could be leveraged as an alternative in this field. In the proposed methodology, DeepSeek-R1, OpenBioLLM-Llama3 and Qwen 3.5 are used to generate synthetic mental health evaluation reports conditioned on specific International Classification of Diseases, Tenth Revision (ICD-10) codes. Because naive text generation can lead to mode collapse or privacy breaches (memorization), a comprehensive evaluation framework is introduced. The generated diagnostic texts are assessed across three dimensions: semantic fidelity, lexical diversity, and privacy/plagiarism. The results demonstrate that all models can generate clinically coherent, diverse, and privacy-safe synthetic reports, significantly expanding the available training data for clinical natural language processing tasks without compromising patient confidentiality.

Abstract

Explore similar work

Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation

SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning

MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering