Multi-Faceted Self-Consistent Preference Alignment for Query Rewriting in Conversational Search

Zhiyu Cao; Peifeng Li; Qiaoming Zhu

cs.CLApr 8, 2026

Multi-Faceted Self-Consistent Preference Alignment for Query Rewriting in Conversational Search

Authors: Zhiyu Cao, Peifeng Li, Qiaoming Zhu

Organizations: School of Computer Science and Technology, Soochow University, Suzhou, China

Abstract

Conversational Query Rewriting (CQR) aims to rewrite ambiguous queries to achieve more efficient conversational search. Early studies have predominantly focused on the rewriting in isolation, ignoring the feedback from query rewrite, passage retrieval and response generation in the rewriting process. To address this issue, we propose Multi-Faceted Self-Consistent Preference Aligned CQR (MSPA-CQR). Specifically, we first construct self-consistent preference alignment data from three dimensions (rewriting, retrieval, and response) to generate more diverse rewritten queries. Then we propose prefix guided multi-faceted direct preference optimization to learn preference information from three different dimensions. The experimental results show that our MSPA-CQR is effective in both in- and out-of-distribution scenarios.

Figures & tables

Figure 1: An example of CQR. Green and blue represent the key information for retrieval that was omitted in

\text{Q}_{3}

Figure 2: An overview of MSPA-CQR, including the two stages of Multi-Faceted Preference Data Construction and Prefix-Guided Multi-Faceted Preference Optimization.

Figure 3: Kendall’s Tau correlation between the consistency score ranking lists of different preferences on TopiOCQA (left) and QReCC (right).

Type	System	Backbone	TopiOCQA				QReCC
Type	System	Backbone	MRR	NDCG	R@10	R@100	MRR	NDCG	R@10	R@100
	w/o Feedback
Sparse (BM25)	EDIRCS	T5-base	-	-	-	-	41.2	-	62.7	90.2
	LLM-Aided	ChatGPT	-	-	-	-	49.4	46.5	67.1	88.2
	LLM4CS	ChatGPT	27.9	26.4	48.4	71.1	51.6	49.3	75.3	92.6
	CHIQ	LLaMA2-7B	25.6	23.5	44.7	-	54.3	51.9	78.5	-

Table 1: Evaluation results of various retrieval system types on the test sets of TopiOCQA and QReCC, where all reported values of the baselines originated from their papers.

Rewrite	Retrieval	Response	MRR	NDCG	R@10	R@100
$\checkmark$	$\checkmark$	$\checkmark$	41.4	39.5	63.5	77.4
$\checkmark$	$\checkmark$		40.4	38.9	62.1	77.1
$\checkmark$		$\checkmark$	40.2	38.5	60.5	76.6
	$\checkmark$	$\checkmark$	39.6	37.8	60.4	76.4
$\checkmark$			38.1	36.7	57.8	74.7
	$\checkmark$		37.3	35.8	56.7	73.5

Table 2: Ablation study under the preference guidance of rewriting, retrieval, response, and the combinations.

	TopiOCQA
Variant	MRR	NDCG	R@10	R@100
MSPA-CQR ( Ours )	41.4	39.5	63.5	77.4
w/o. Prefix Guidance	39.2	36.8	60.7	74.3
w/o. MDPO	29.1	27.8	48.6	65.8
w/o. NLI	40.8	38.7	62.7	76.8
w/o. Query Expansion	37.4	36.3	61.6	75.8

Table 3: Ablation study for each component of MSPA-CQR and different LLMs.

Figure 4: A comparison of the linguistic features of rewritten queries generated under three preference guides. “N-grams div.” refers to the diversity of n-grams (

\text{n}=2

), and “Edit dist.” refers to the Levenshtein distance.

Figure 5: The intersection size of passages obtained through dense retrieval using rewritten queries guided by different preferences on TopiOCQA (left) and QReCC (right).

Dialogue Context

Q: What did Ned Kelly do?

A: An Australian bushranger, outlaw, gang leader and convicted police murderer.

Q: What happened after he was locked up for the night?

A: While he was escorted by four policemen, he absconded and ran, taking refuge in a shoemaker’s shop.

Q: Did it say what happened in the shoemakers shop?

Gold Passage : On 18 September 1877 in Benalla, Kelly , while drunk…he absconded and ran, taking refuge in a shoemaker’s shop . The police and the shop owner tried to handcuff him but failed…

Table 4: A case study on TopiOCQA, where “Rank” refers to the ranking of the retrieved gold passage, and “Not Found” means that there is no gold passage in the top-100 retrieved passages.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Split	#Conv.	#Turns(Qry.)	#Collection
TopiOCQA	Train	3,509	45,450	25M
TopiOCQA	Test	205	2,514	25M
QReCC	Train	10,823	63,501	54M
QReCC	Test	2,775	16,451	54M
CAsT-19	Test	50	479	38M
CAsT-20	Test	25	208	38M

Appendix

Table 5: Statistics of conversational search datasets.

System	Backbone	CAsT-19			CAsT-20			CAsT-21
System	Backbone	MRR	N@3	R@10	MRR	N@3	R@10	MRR	N@3	R@10
E5-Mistral	Mistral-7B	62.2	31.3	9.5	22.0	15.4	8.4	48.2	32.5	20.5
HyDE	ChatGPT-3.5	55.6	39.2	10.0	44.8	29.3	16.9	-	-	-
Query2doc	ChatGPT-3.5	58.8	42.4	11.6	48.6	32.5	17.3	-	-	-
InstructorR	ChatGPT-3.5	61.2	46.6	10.4	43.7	29.6	8.3	46.7	32.5	18.4
LLM4CS	ChatGPT-3.5	70.4	46.8	11.7	58.6	41.5	19.3	66.1	46.9	24.4

Appendix

Table 6: Zero-shot retrieval performances under the dense retrieval (ANCE).

	TopiOCQA
Variant	MRR	NDCG	R@10	R@100
MSPA-CQR ( Ours )	41.4	39.5	63.5	77.4
w/o. Length Penalty Term	41.1	39.0	62.8	76.9
w/ passages	41.8	40.2	64.1	77.6
MSPA-CQR (CombGMNZ)	40.5	38.8	62.6	76.8
MSPA-CQR (Log_ISR)	40.7	39.1	62.4	76.5

Appendix

Table 7: Ablation experiments of different modules in the MSPA-CQR.

Panel of Figure 6 — Figure 6: The prompt used to construct the response preference data based on retrieval.

	Response Generation	Sparse Retrieval	Dense Retrieval
TopiOCQA	0.74	0.82	0.77
QReCC	0.79	0.84	0.81

Appendix

Table 8: Kendall’s Tau correlation between self-consistency scores and downstream task performance.

Figure 7: The trend of R@100 on TopiOCQA with the change in the number of sampled candidate rewritten queries.

Figure 8: The trend of MRR on TopiOCQA with the change in the number of sampled candidate rewritten queries.

	TopiOCQA
Different Values of T	MRR	NDCG	R@10	R@100
MSPA-CQR (T = 10)	40.7	38.4	62.7	76.8
MSPA-CQR (T = 50)	41.6	39.8	64.1	78.2
MSPA-CQR (T = 100)	41.4	39.5	63.5	77.4

Appendix

Table 9: Analysis of setting different numbers of retrieved passages on TopiOCQA.

Method	ROUGE 1	ROUGE L	BertScore
LLM4CS	29.20	27.78	85.88
RETPO	26.01	24.39	85.74
MSPA-CQR	31.97	29.85	86.72

Appendix

Table 10: Performance analysis of end-to-end question answering using the MSPA-CQR and baseline methods.

	TopiOCQA
Variant	MRR	NDCG	R@10	R@100
MSPA-CQR ( Llama-2-7b-hf )	41.4	39.5	63.5	77.4
MSPA-CQR ( Llama-3.1-8B-Instruct )	40.5	38.8	62.6	76.8
MSPA-CQR ( Llama-3.2-3B-Instruct )	40.7	39.1	62.4	76.5

Appendix

Table 11: Analysis of using different backbones in MSPA-CQR.

	TopiOCQA
Variant	MRR	NDCG	R@10	R@100	LT ${}_{\text{rewrite}}$	LT ${}_{\text{retrieval}}$
MSPA-CQR	30.6	29.5	51.9	75.2	1.32	0.00079
MSPA-CQR (merge)	29.4	28.1	50.1	74.0	0.69	0.00034
RETPO	28.3	26.5	48.3	73.1	0.57	0.00031

Appendix

Table 12: Analysis of the trade-off between performance and efficiency.

Panel of Figure 9 — Figure 9: A case of merging multiple rewritten queries into a single query.

Dialogue Context

Q: What is quoll?

A: They are carnivorous marsupials.

Q: Where are they found?

A: Australia and New Guinea.

Q: Where did the latter get it’s name from?

A: UNANSWERABLE.

Appendix

Table 13: A case study on TopiOCQA.

Dialogue Context

Q: What happened in 1977 to Journey the band?

A: Journey’s album sales did not improve and Columbia Records requested that they change their musical style and add a frontman, with whom keyboardist Gregg Rolie could share lead vocal duties.

Q: Why did they change their musical direction?

A: Journey hired Robert Fleischman and made the transition to a more popular style, akin to that of Foreigner and Boston.

Q: What was Robert’s role with the band?

A: Lead singer.

Appendix

Table 14: A case study on QReCC.

Dialogue Context

Q: Where did the uk come in eurovision song contest?

A: The United Kingdom came in 26th place in the Eurovision Song Contest 2019.

Q: What song did the country perform?

A: The United Kingdom’s selected song for the Eurovision Song Contest 2019 was Bigger than Us by Michael Rice.

Q: How many times has the uk participated in the contest?

A: The United Kingdom has participated in the Eurovision Song Contest 62 times.

Appendix

Table 15: A case study on QReCC.

Query Reformulation

Query reformulation transforms user questions into retrieval-ready queries to improve search, evidence coverage, and downstream answers.

See all 42 papers

CompCQR: Compositional Query Generation for Training-Free Conversational Search
Sep 13, 2026Yunah Jang, Kang-il Lee, Joongbo Shin +1Conversational Search Query Reformulation
Dept. of ECE, Seoul National University · LG AI Research
WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search
Oct 12, 2026Yingtao Ren, Aolin Li, Cheng Cheng +6Query Reformulation Personalized Language Model Alignment
Weixin Group, Tencent China
Not All Queries Need Rewriting: When Prompt-Only LLM Refinement Helps and Hurts Dense Retrieval
Mar 2, 2026Varun KotteQuery Reformulation Dense Retrieval
Adobe

Sep 13, 2026J/K move · Enter open · S save

CompCQR: Compositional Query Generation for Training-Free Conversational Search

Yunah Jang, Kang-il Lee, Joongbo Shin, Kyomin Jung

Dept. of ECE, Seoul National University · LG AI Research

Conversational Search Query Reformulation

Open paper AskPDF arXiv

Multi-turn interactions with LLMs are becoming increasingly common in information-seeking scenarios. However, user queries are often ambiguous and context-dependent, making them ill-suited for direct use as retriever queries. Conversational query reformulation (CQR) addresses this issue by rewriting the current utterance into a stand-alone query grounded in the dialogue history. Recent LLM-based CQR approaches achieve strong performance; however, their repeated LLM invocations and misalignment with downstream retrievers remain challenges. In this work, we begin from the observation that retrievers are highly sensitive to content ordering: simply reordering the same content can lead to changes in retrieval coverage and performance. Based on this, we propose a novel training-free method that generates a very large number of queries with minimal LLM usage by compositionally combining a small set of atomic components. We further apply LLM reasoning to construct a high-quality document set that balances precision and recall while capturing the user's core intent. Our framework generalizes across both open- and closed-source LLMs as well as dense and sparse retrievers. It achieves strong performance on four widely used conversational benchmarks, with up to 22.5% relative MRR improvement over the previous state-of-the-art baseline with far fewer LLM calls.

Abstract

Query Reformulation

Explore similar work

CompCQR: Compositional Query Generation for Training-Free Conversational Search

WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search

Not All Queries Need Rewriting: When Prompt-Only LLM Refinement Helps and Hurts Dense Retrieval