Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrieval

Hangeol Chang; Changsun Lee; Seungjoon Rho; Junho Yeo; Jong Chul Ye

cs.CLMar 19, 2026

Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrieval

Authors: Hangeol Chang, Changsun Lee, Seungjoon Rho, Junho Yeo, Jong Chul Ye

Organizations: Graduate School of AI, KAIST, Republic of Korea · School of Electrical Engineering, KAIST, Republic of Korea · Department of Industrial Engineering, Yonsei University, Republic of Korea

Abstract

Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by grounding generation in external, non-parametric knowledge. However, when a task requires choosing among competing options, simply grounding generation in broadly relevant context is often insufficient to drive the final decision. Existing RAG methods typically rely on a single initial query, which often favors topical relevance over decision-relevant evidence, and therefore retrieves background information that can fail to discriminate among answer options. To address this issue, here we propose Hypothesis-Conditioned Query Rewriting (HCQR), a training-free pre-retrieval framework that reorients RAG from topic-oriented retrieval to evidence-oriented retrieval. HCQR first derives a lightweight working hypothesis from the input question and candidate options, and then rewrites retrieval into three targeted queries that seek evidence to: (1) support the hypothesis, (2) distinguish it from competing alternatives, and (3) verify salient clues in the question. This approach enables context retrieval that is more directly aligned with answer selection, allowing the generator to confirm or overturn the initial hypothesis based on the retrieved evidence. Experiments on MedQA and MMLU-Med show that HCQR consistently outperforms single-query RAG and re-rank/filter baselines, improving average accuracy over Simple RAG by 5.9 and 3.6 points, respectively. Code is available at https://anonymous.4open.science/r/HCQR-1C2E.

Figures & tables

Figure 1: HCQR pipeline. A hypothesis formulator proposes a lightweight working hypothesis together with the evidence expected under that hypothesis. The query rewriter turns this state into three verification-oriented queries. The working hypothesis steers retrieval but is not shown to the generator by default.

	MedQA					MMLU-Med
Method	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average
CoT	58.0	69.0	72.8	84.1	71.0	66.3	75.0	82.9	90.2	78.6
Simple RAG	55.6	66.8	71.3	82.7	69.1	64.5	74.7	81.8	88.9	77.5
Rerank-RAG	56.8	67.3	72.1	83.3	69.9	67.1	75.8	82.3	90.1	78.8
Rewriting	58.9	67.5	73.1	83.3	70.7	67.0	74.7	82.5	88.8	78.2
HyDE	61.0	69.3	71.4	82.9	71.1	67.8	76.5	82.6	89.4	79.1

Table 1: Accuracy (%) on MedQA and MMLU-Med for HCQR and baseline methods under the shared 15-document final-context budget. HCQR achieves the best performance in every reported dataset-model cell.

	MedQA					MMLU-Med
Method	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average
Simple RAG	30.0	30.0	30.0	30.0	30.0	48.7	48.7	48.7	48.7	48.7
Rerank-RAG	44.1	44.1	44.1	44.1	44.1	56.8	56.8	56.8	56.8	56.8
Rewriting	45.2	51.2	62.2	65.6	56.1	51.4	53.7	56.7	59.1	55.3
HyDE	51.6	54.5	52.8	54.6	53.4	43.0	46.2	44.3	46.3	44.9
MAIN-RAG	23.7	26.4	24.7	26.7	25.4	35.6	36.7	33.7	36.5	35.6

Table 2: Decision-Useful Rate ( Entailed + Useful ) on MedQA and MMLU-Med for HCQR and baseline methods. HCQR yields the highest rate of decision-useful retrieved contexts across all reported dataset-model cells.

Method	Entailed	Useful	Not Useful
Simple RAG	14.3	25.1	60.7
Rerank-RAG	19.9	30.6	49.5
Rewriting	19.6	36.1	44.3
HyDE	16.6	32.5	50.8
MAIN-RAG	11.6	18.9	69.5
HCQR (ours)	34.7	41.5	23.8

Table 3: Average context-utility distribution across MedQA and MMLU-Med (%). Values are averaged over the two datasets and four model configurations. Useful abbreviates Useful-but-Not-Entailed .

Label	MedQA	MMLU-Med
Entailed	94.4 / +8.2	95.4 / +5.1
Useful	77.9 / +4.2	83.9 / +1.3
Not Useful	57.6 / -5.8	70.2 / -3.2

Table 4: Utility-conditioned accuracy and same-subset gain over no-retrieval CoT. Results are averaged over two datasets, four models, and six methods under the configurations of Table 2 . Each cell is reported as accuracy /

\Delta_{\mathrm{CoT}}

Method	MedQA	MMLU-Med
Rewriting	70.7	78.2
Rewriting $+$ options	71.3	79.2
HCQR $-$ options	73.1	79.9
HCQR	75.0	81.1

Table 5: Diagnostic control for answer-option exposure.

Method	MedQA	MMLU-Med
HCQR	75.0	81.1
HCQR $-$ $q_{1}$ (Support)	72.4	79.0
HCQR $-$ $q_{2}$ (Distinction)	73.2	80.1
HCQR $-$ $q_{3}$ (Key Features)	73.0	80.5

Table 6: HCQR query-component ablation.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

	MedQA					MMLU-Med
Method	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average
Rewriting	58.9	67.5	73.1	83.3	70.7	67.0	74.7	82.5	88.8	78.2
Rewriting $+$ options	57.0	68.3	75.8	84.1	71.3	67.5	77.2	83.7	88.4	79.2
HCQR $-$ options	61.1	71.6	75.3	84.5	73.1	69.1	77.3	83.7	89.4	79.9
HCQR (ours)	63.4	73.0	77.9	85.8	75.0	68.6	79.7	85.4	90.6	81.1

Appendix

Table 7: Full model-by-model accuracy values underlying the dataset-level averages reported in Table 5 .

	MedQA					MMLU-Med
Method	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average
HCQR (ours)	63.4	73.0	77.9	85.8	75.0	68.6	79.7	85.4	90.6	81.1
HCQR $-$ $q_{1}$	62.1	70.1	73.8	83.5	72.4	66.9	77.1	83.0	88.9	79.0
HCQR $-$ $q_{2}$	61.2	71.5	74.0	86.3	73.2	69.2	78.7	83.4	89.0	80.1
HCQR $-$ $q_{3}$	61.3	70.9	74.3	85.6	73.0	68.2	77.8	86.0	90.1	80.5

Appendix

Table 8: Full model-by-model accuracy values underlying the dataset-level averages reported in Table 6 .

	MedQA					MMLU-Med
Method	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average
Rewriting	45.2	51.2	62.2	65.6	56.1	51.4	53.7	56.7	59.1	55.3
Rewriting $+$ options	51.1	62.9	71.8	76.4	65.6	56.8	61.3	64.8	66.3	62.3
HCQR $-$ options	61.0	70.5	70.5	80.5	70.6	57.0	61.1	64.2	67.5	62.4
HCQR (ours)	73.4	81.8	83.2	89.8	82.1	65.8	69.1	72.5	73.7	70.3

Appendix

Table 9: Additional DUR results for the option-exposure control. DUR denotes Entailed + Useful .

	MedQA					MMLU-Med
Method	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Average
HCQR (ours)	73.4	81.8	83.2	89.8	82.1	65.8	69.1	72.5	73.7	70.3
HCQR $-$ $q_{1}$	55.0	62.1	66.5	70.9	63.6	48.5	52.3	55.2	55.8	53.0
HCQR $-$ $q_{2}$	60.8	71.7	70.1	80.4	70.8	53.5	57.4	61.8	64.2	59.2
HCQR $-$ $q_{3}$	63.1	73.0	74.2	82.6	73.2	51.6	57.0	60.8	63.0	58.1

Appendix

Table 10: Additional DUR results for the HCQR query-role ablation. DUR denotes Entailed + Useful .

		MedQA					MMLU-Med
Method	Label	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Avg.	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Avg.
Simple RAG	Entailed	6.0	6.0	6.0	6.0	6.0	22.5	22.5	22.5	22.5	22.5
	Useful	24.0	24.0	24.0	24.0	24.0	26.2	26.2	26.2	26.2	26.2
	Not Useful	70.0	70.0	70.0	70.0	70.0	51.3	51.3	51.3	51.3	51.3
Rerank-RAG	Entailed	10.7	10.7	10.7	10.7	10.7	29.1	29.1	29.1	29.1	29.1
	Useful	33.4	33.4	33.4	33.4	33.4	27.7	27.7	27.7	27.7	27.7

Appendix

Table 11: Full context-utility breakdown underlying Table 3 . For each method, we report the rates (%) of Entailed , Useful-but-Not-Entailed ( Useful ), and Not Useful contexts across all four model configurations and their average.

		MedQA					MMLU-Med
Method	Label	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Avg.	Llama3.2 3B	Llama3.1 8B	Qwen3 4B	Qwen3 30B	Avg.
Simple RAG	Entailed	93.5 / +9.1	93.5 / +9.1	93.5 / +9.1	93.5 / +9.1	93.5 / +9.1	97.1 / +6.1	97.1 / +6.1	97.1 / +6.1	97.1 / +6.1	97.1 / +6.1
	Useful	85.6 / +5.9	85.6 / +5.9	85.6 / +5.9	85.6 / +5.9	85.6 / +5.9	88.1 / +0.7	88.1 / +0.7	88.1 / +0.7	88.1 / +0.7	88.1 / +0.7
	Not Useful	64.5 / -4.9	64.5 / -4.9	64.5 / -4.9	64.5 / -4.9	64.5 / -4.9	71.9 / -5.2	71.9 / -5.2	71.9 / -5.2	71.9 / -5.2	71.9 / -5.2
Rerank-RAG	Entailed	97.8 / +8.1	97.8 / +8.1	97.8 / +8.1	97.8 / +8.1	97.8 / +8.1	95.6 / +6.0	95.6 / +6.0	95.6 / +6.0	95.6 / +6.0	95.6 / +6.0
	Useful	81.6 / +5.2	81.6 / +5.2	81.6 / +5.2	81.6 / +5.2	81.6 / +5.2	82.1 / -2.0	82.1 / -2.0	82.1 / -2.0	82.1 / -2.0	82.1 / -2.0

Appendix

Table 12: Full model-by-model values underlying Table 4 . Each cell is reported as accuracy /

\Delta_{\mathrm{CoT}}

Panel of Figure 2 — Figure 2: Prompt used for the context-utility judge.

Panel of Figure 3 — Figure 3: Prompt used for the no-retrieval CoT baseline.

Panel of Figure 6 — Figure 6: Prompt used for generating hypothetical documents in the HyDE baseline.

Panel of Figure 9 — Figure 9: Prompt used for the HCQR hypothesis-formulation stage.

Panel of Figure 11 — Figure 11: Prompt used for the HCQR query-rewriting stage.

Panel of Figure 13 — Figure 13: Representative MedQA example used to illustrate the end-to-end HCQR workflow.

Panel of Figure 14 — Figure 14: Illustrative provisional output of the Hypothesis Formulator. The working hypothesis is used to steer retrieval and may later be confirmed or overturned by the retrieved evidence.

Panel of Figure 15 — Figure 15: Three intent-aligned queries generated from the provisional hypothesis: answer support, alternative discrimination, and clue verification.

Panel of Figure 16 — Figure 16: Representative passages retrieved by the generated queries. The retrieved set can contain both supporting and corrective evidence, allowing the final generator to revise the provisional hypothesis when necessary.

Panel of Figure 17 — Figure 17: Raw generator output for the representative MedQA example. The output shows that the final generator answers from the retrieved evidence and the original question rather than from the provisional hypothesis alone, and in this case correctly arrives at the gold answer after revising the initial guess. The text is reproduced verbatim except for formatting.

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 792 papers

Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering
Apr 6, 2026Byeolhee Kim, Min-Kyung Kim, Young-Hak Kim +1Medical QA Retrieval-Augmented Generation
Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea · Department of Computer Science, Yonsei University, Seoul, Republic of Korea · INMED DATA, Seoul, Republic of Korea +1
Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation
Apr 19, 2026Jiaang Li, Zhendong Mao, Quan Wang +2Retrieval-Augmented Generation Text Style Transfer
University of Science and Technology of China · Beijing University of Posts and Telecommunications
Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines
Apr 24, 2026Negar Arabzadeh, Andrew Drozdov, Michael Bendersky +1Retrieval-Augmented Generation RAG Evaluation
UC Berkeley · Berkeley, CA, United States · Databricks +1

Apr 6, 2026J/K move · Enter open · S save

Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering

Byeolhee Kim, Min-Kyung Kim, Young-Hak Kim, Tae-Joon Jeon

Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea · Department of Computer Science, Yonsei University, Seoul, Republic of Korea · INMED DATA, Seoul, Republic of Korea+1

Medical QA Retrieval-Augmented Generation

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) grounds large language models in external medical knowledge, yet standard retrievers frequently surface hard negatives that are semantically close to the query but describe clinically distinct conditions. While existing query-expansion methods improve query representation to mitigate ambiguity, they typically focus on enriching target-relevant semantics without an explicit mechanism to selectively suppress specific, clinically plausible hard negatives. This leaves the system prone to retrieving plausible mimics that overshadow the actual diagnosis, particularly when such mimics are dominant within the corpus. We propose Contrastive Hypothesis Retrieval (CHR), a framework inspired by the process of clinical differential diagnosis. CHR generates a target hypothesis

H^+

for the likely correct answer and a mimic hypothesis

H^-

for the most plausible incorrect alternative, then scores documents by promoting

H^+

-aligned evidence while penalizing

H^-

-aligned content. Across three medical QA benchmarks and three answer generators, CHR outperforms all five baselines in every configuration, with improvements of up to 10.4 percentage points over the next-best method. On the

n=587

pooled cases where CHR answers correctly while embedded hypothetical-document query expansion does not, 85.2% have no shared documents between the top-5 retrieval lists of CHR and of that baseline, consistent with substantive retrieval redirection rather than light re-ranking of the same candidates. By explicitly modeling what to avoid alongside what to find, CHR bridges clinical reasoning with retrieval mechanism design and offers a practical path to reducing hard-negative contamination in medical RAG systems.

Abstract

Retrieval-Augmented Generation

Explore similar work

Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering

Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation

Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines