Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering
Authors: Byeolhee Kim, Min-Kyung Kim, Young-Hak Kim, Tae-Joon Jeon
Organizations: Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea · Department of Computer Science, Yonsei University, Seoul, Republic of Korea · INMED DATA, Seoul, Republic of Korea · Division of Cardiology, Department of Information Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
Retrieval-augmented generation (RAG) grounds large language models in external medical knowledge, yet standard retrievers frequently surface hard negatives that are semantically close to the query but describe clinically distinct conditions. While existing query-expansion methods improve query representation to mitigate ambiguity, they typically focus on enriching target-relevant semantics without an explicit mechanism to selectively suppress specific, clinically plausible hard negatives. This leaves the system prone to retrieving plausible mimics that overshadow the actual diagnosis, particularly when such mimics are dominant within the corpus. We propose Contrastive Hypothesis Retrieval (CHR), a framework inspired by the process of clinical differential diagnosis. CHR generates a target hypothesis H+ for the likely correct answer and a mimic hypothesis H− for the most plausible incorrect alternative, then scores documents by promoting H+-aligned evidence while penalizing H−-aligned content. Across three medical QA benchmarks and three answer generators, CHR outperforms all five baselines in every configuration, with improvements of up to 10.4 percentage points over the next-best method. On the n=587 pooled cases where CHR answers correctly while embedded hypothetical-document query expansion does not, 85.2% have no shared documents between the top-5 retrieval lists of CHR and of that baseline, consistent with substantive retrieval redirection rather than light re-ranking of the same candidates. By explicitly modeling what to avoid alongside what to find, CHR bridges clinical reasoning with retrieval mechanism design and offers a practical path to reducing hard-negative contamination in medical RAG systems.
Figures & tables
Figure 1. Overview of Contrastive Hypothesis Retrieval (CHR). Given a clinical question, CHR generates a contrastive hypothesis pair consisting of a target hypothesis ( H+ ) and a mimic hypothesis ( H− ), retrieves documents using contrastive scoring that promotes target-aligned content while penalizing mimic-aligned content, and generates the final answer from the retrieved evidence.
Figure 2. Prompt template for contrastive hypothesis generation. The system prompt establishes the role of a medical specialist, and the user prompt instructs the model to generate a target hypothesis ( H+ ) for the likely correct diagnosis and a mimic hypothesis ( H− ) for the most plausible incorrect alternative.
Generator
Method
MMLU-Med
MedQA-US
BioASQ-Y/N
Avg.
Llama-3-8B-Instruct
Standard RAG
36.8
36.1
41.8
38.2
HyDE ( H+ only)
47.2
44.6
51.3
47.7
Query2doc
49.1
46.2
46.9
47.4
CSQE
49.7
46.5
56.5
50.9
ThinkQE
41.3
45.6
52.1
46.3
CHR (Ours)
50.4
51.5
63.1
55.0
Table 1. Accuracy (%) across three medical QA benchmarks. All query-expansion baselines use the same hypothesis generator (Qwen2.5-72B-Instruct), retriever (MedCPT), and corpus (MedCorp). CHR consistently achieves the highest accuracy across all settings.
MedQA
BioASQ
MMLU-Med
Retriever
Std RAG
HyDE
CHR
Std RAG
HyDE
CHR
Std RAG
HyDE
CHR
Avg.
MedCPT
36.1
44.6
51.5
41.8
51.3
63.1
36.8
47.2
50.4
47.0
BGE-large-en-v1.5
41.8
52.4
52.8
41.8
53.1
60.2
52.3
54.5
56.6
51.7
E5-large-v2
40.2
50.6
52.3
45.5
40.5
62.6
51.7
55.2
57.7
50.7
BM25
40.1
51.5
53.3
46.8
65.2
63.4
57.9
59.3
59.9
55.3
Table 2. Retriever-agnosticism. Accuracy (%) on three medical QA benchmarks with Llama-3-8B-Instruct as the answer generator and Qwen2.5-72B-Instruct as the hypothesis generator, sweeping four retrievers spanning dense (MedCPT, BGE, E5) and sparse (BM25) families. Bold marks the best method within each (retriever, dataset) triple. The Avg. column averages all nine method–benchmark scores within each retriever row. CHR wins on MedQA and MMLU-Med across all four retrievers and on BioASQ for three of four retrievers; the only exception is BM25, where HyDE is slightly higher.
Retriever
λ schedule
MedQA
BioASQ
MMLU
Avg.
MedCPT
Fixed ( λ=1.0 )
51.5
63.1
50.4
55.0
Adaptive (linear)
51.7
69.4
56.1
59.1
Adaptive (threshold)
50.3
70.4
57.2
59.3
Adaptive (sigmoid)
53.3
68.8
57.2
59.8
BGE
Fixed ( λ=1.0 )
52.8
60.2
56.6
56.5
Adaptive (linear)
55.5
65.7
58.4
59.9
Table 3. Adaptive contrastive weighting versus fixed λ . Accuracy (%) with Llama-3-8B-Instruct as the answer generator and Qwen2.5-72B-Instruct as the hypothesis generator. All three adaptive schedules outperform the fixed weight on average for both retrievers, with a best-schedule improvement of +4.8 pp on MedCPT and +3.4 pp on BGE.
Dataset
n
Zero Overlap
Mean Overlap
MMLU-Med
172
78.5%
0.25
MedQA
249
80.3%
0.25
BioASQ
166
99.4%
0.01
Combined
587
85.2%
0.18
Table 4. Retrieval divergence analysis for cases where CHR succeeds but HyDE fails ( n=587 ). Zero Overlap indicates the percentage of cases with no shared documents in top-5 results. Mean Overlap is the average overlap ratio of top-5 results between CHR and HyDE.
Method
LLM Calls
Output Tokens
Token Reduction
Standard RAG
0
0
N/A
Query2Doc
1
∼ 512
5.7 ×
CSQE
1 + retrieval
∼ 480
6.1 ×
ThinkQE
≥ 3
∼ 1,500
2.0 ×
HyDE
∼ 8
∼ 2,926
1.0 ×
CHR (Ours)
1
∼ 303
9.7 ×
Table 5. Query-expansion cost per sample, averaged across MedQA and MMLU-Med. Output token counts are estimated from generated text length. Token Reduction is relative to HyDE (most expensive). CHR achieves the lowest cost while consistently achieving the highest accuracy.
H− Quality
n
Correct
Accuracy
Excellent
9
6
66.7%
Good
30
12
40.0%
Poor
8
3
37.5%
Table 6. CHR accuracy stratified by physician-rated mimic hypothesis quality. Samples where H+ was factually incorrect are excluded ( n=47 ).
Figure 3. Case study showing why the negative (mimic) hypothesis H− is essential for discriminative retrieval. In the left box, bolded text highlights misleading co-occurrences of antivirals with parkinsonian symptoms (as side effects, not treatments), which led to the incorrect answer. In the right box, bolded text highlights the key sentences identifying amantadine as an antiviral repurposed for Parkinson’s disease, directly leading to the correct answer.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4. Additional case study from MedQA (oncology domain). In the left box, bolded text highlights how retrieved documents focus on tamoxifen-related gynecologic bleeding, the dominant complication in the tamoxifen safety literature, which led to the incorrect answer. In the right box, bolded text highlights evidence describing tamoxifen’s pro-coagulant effects via hepatic estrogen agonism, directly supporting deep venous thrombosis as the correct answer.
Figure 5. Sensitivity of CHR accuracy to the contrastive weight λ on MedQA (Gemma-2-9B-It). The dashed red lines indicate the robust plateau region λ∈[0.6,1.2] where CHR consistently outperforms both baselines. Performance peaks at λ=1.0 .
Question (MedQA #16): A 7-year-old boy with seasonal allergic asthma is being considered for an experimental therapy targeting a mediator of antibody class switching. Which mediator is described?
Options: (A) IL-2 (B) IL-10 (C) IL-13 (D) IL-4 ✓
H+ : IL-4 is the key cytokine driving Th2 differentiation and B-cell class switching to IgE .
H− : IL-13 overlaps functionally with IL-4 but does not directly drive B-cell class switching to IgE .
CHR top evidence: (1) Co-occurring IL-4 and IL-13 with IgE switching. (2) “ IL-13, another switch factor for IgE …” (3) “Both IL13 and IL4 are capable of inducing class switching…” HyDE top evidence: (1) “B cells are switched to IgE by IL-4 …” (2) Th2-focused evidence emphasizing IL-4 -targeted interventions. (3) Receptor-level discussion (IL-4/IL-13 axis) with clearer IL-4 role.
Analysis: Strong IL-4/IL-13 co-occurrence causes partial overlap between target and mimic directions; subtracting λH− can therefore suppress IL-4-relevant evidence and make CHR’s context more ambiguous than HyDE’s.
Appendix
Table 7. Failure case of semantic co-occurrence collapse. CHR correctly identifies IL-13 as the mimic for IL-4, but heavy corpus co-occurrence of these cytokines in the same Th2 pathway causes the contrastive subtraction to suppress some target-relevant evidence.
Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by grounding generation in external, non-parametric knowledge. However, when a task requires choosing among competing options, simply grounding generation in broadly relevant context is often insufficient to drive the final decision. Existing RAG methods typically rely on a single initial query, which often favors topical relevance over decision-relevant evidence, and therefore retrieves background information that can fail to discriminate among answer options. To address this issue, here we propose Hypothesis-Conditioned Query Rewriting (HCQR), a training-free pre-retrieval framework that reorients RAG from topic-oriented retrieval to evidence-oriented retrieval. HCQR first derives a lightweight working hypothesis from the input question and candidate options, and then rewrites retrieval into three targeted queries that seek evidence to: (1) support the hypothesis, (2) distinguish it from competing alternatives, and (3) verify salient clues in the question. This approach enables context retrieval that is more directly aligned with answer selection, allowing the generator to confirm or overturn the initial hypothesis based on the retrieved evidence. Experiments on MedQA and MMLU-Med show that HCQR consistently outperforms single-query RAG and re-rank/filter baselines, improving average accuracy over Simple RAG by 5.9 and 3.6 points, respectively. Code is available at https://anonymous.4open.science/r/HCQR-1C2E.
Hangeol Chang, Changsun Lee, Seungjoon Rho +2
Graduate School of AI, KAIST, Republic of Korea · School of Electrical Engineering, KAIST, Republic of Korea · Department of Industrial Engineering, Yonsei University, Republic of Korea
In medical multiple-choice question answering (MCQA), Retrieval-Augmented Generation (RAG) can supplement the domain knowledge of language models (LMs). However, since vanilla RAG indiscriminately utilizes retrieved documents, it can degrade LM performance. To address this, we propose MedJudgeRAG. Our framework represents retrieved documents as a dynamic knowledge graph (KG) composed of entities and relations. For each option, the model judges an evidence verdict from the retrieved documents and the KG. Based on the verdict combination, the model determines a knowledge utilization strategy to reason toward the final answer. These capabilities are trained via supervised fine-tuning using structured reasoning traces generated by a teacher LM. The training employs a weighted cross-entropy loss that differentially weights the KG and reasoning segments. Experiments on two medical MCQA benchmarks demonstrate that MedJudgeRAG consistently outperforms both vanilla RAG and parametric baselines. Furthermore, ablation analysis reveals that the dynamic KG is more effective as graph-conditioned supervision at training time than as an explicit output at inference time. Our code is available at https://github.com/hyu-amllab/medjudgerag, and the generated reasoning traces are released at https://huggingface.co/datasets/youarethewon/medjudgerag.
Seongwon Seo, Seung Hwan Cho, Young-Min Kim
Department of Industrial Data Engineering, Hanyang University, Seoul, South Korea · School of Interdisciplinary Industrial Studies, Hanyang University, Seoul, South Korea
Large language models (LLMs) have shown promising performance across a wide range of biomedical applications, including medical question answering (QA), yet they remain prone to hallucinations and outdated knowledge. Although retrieval-augmented generation (RAG) can alleviate this issue by incorporating external documents, there still exist two fundamental limitations. First, medical knowledge is often fragmented across documents, while most RAG methods rely on a single retrieval path, which makes it challenging to jointly preserve fine-grained semantic information and structured global associations. Second, static retrieval strategies are typically insufficient to support deep reasoning that is important in complex medical QA. In this paper, we present a dual-path retrieval framework with an iterative retrieval-reasoning mechanism termed "Hybrid-IR" for complex medical QA. The proposed Hybrid-IR integrates graph-based retrieval for exploration of structured knowledge and dense retrieval for fine-grained semantic matching. Moreover, the reasoning trajectory can be progressively refined through an iterative retrieve-reason loop. Experiments on three widely used medical QA benchmarks demonstrate the effectiveness of our Hybrid-IR.
Sheng Wan, Jiahui Zhang, Zicheng Zhao +1
College of Artificial Intelligence, Nanjing Agricultural University, Nanjing, China · School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China