cs.CLSep 13, 2026
SaveCompCQR: Compositional Query Generation for Training-Free Conversational Search
Organizations: Dept. of ECE, Seoul National University · LG AI Research
Abstract
Multi-turn interactions with LLMs are becoming increasingly common in information-seeking scenarios. However, user queries are often ambiguous and context-dependent, making them ill-suited for direct use as retriever queries. Conversational query reformulation (CQR) addresses this issue by rewriting the current utterance into a stand-alone query grounded in the dialogue history. Recent LLM-based CQR approaches achieve strong performance; however, their repeated LLM invocations and misalignment with downstream retrievers remain challenges. In this work, we begin from the observation that retrievers are highly sensitive to content ordering: simply reordering the same content can lead to changes in retrieval coverage and performance. Based on this, we propose a novel training-free method that generates a very large number of queries with minimal LLM usage by compositionally combining a small set of atomic components. We further apply LLM reasoning to construct a high-quality document set that balances precision and recall while capturing the user's core intent. Our framework generalizes across both open- and closed-source LLMs as well as dense and sparse retrievers. It achieves strong performance on four widely used conversational benchmarks, with up to 22.5% relative MRR improvement over the previous state-of-the-art baseline with far fewer LLM calls.
Figures & tables
Figure 1: Comparison of MRR across different FLOPs. On TopiOCQA Adlakha et al. (2022) , we conduct a comparison of two prompting-based CQR methods: CompCQR (ours) and LLM4CS Mao et al. (2023b) .
Figure 2: Comparison of query rewriting methods. We compare the average and upper-bound retrieval performance across 325 query candidates generated by compositional and sampling-based query rewriting methods, where IID denotes independent sampling at different temperatures, Seq. sequential sampling, and Comp. compositional query rewriting.
Figure 3: CompCQR Framework Overview. CompCQR first extracts query components with two LLM calls, generates compositional query candidates, aggregates retrieved documents via RRF scoring, and finally performs LLM-based document selection.
| #Q | LS | RRF | Mean | ||
| MRR | NDCG | MRR | NDCG | ||
| 5 | ✗ | 36.0 | 34.6 | 34.2 | 33.1 |
| ✓ | 39.6 (+3.6) | 38.5 (+3.9) | 38.1 (+3.9) | 37.5 (+4.4) | |
| 325 | ✗ | 37.8 | 36.5 | 36.8 | 35.8 |
| ✓ | 42.3 (+4.5) | 41.5 (+5.0) | 39.4 (+2.6) | 38.9 (+3.1) | |
Table 5: Comparison with different aggregation strategies. We compare mean and RRF aggregation with and without the LLM selection (LS) setting on the TopiOCQA dataset. Each method is evaluated using 5 and 325 queries. The values in parentheses indicate the performance gain when LS is applied.
| Method | # Cands. | End-to-End (s) | MRR |
| LLM4CS | 5 | 4.05 | 34.6 |
| LLM4CS | 16 | 5.51 | 35.4 |
| LLM4CS | 325 | 29.90 | 40.5 |
| CompCQR | 5 | 15.67 | 40.9 |
| CompCQR | 325 | 17.41 | 42.3 |
Table 6: End-to-end wall-clock latency and MRR on TopiOCQA across different numbers of candidates.
| Step | Description | Time | Fraction |
| 0 | Rewrite | 2.93 s | 17.0% |
| 1 | Follow-up Generation | 1.80 s | 10.3% |
| 2 | Variation Generation | 0.19 ms | 0.0% |
| 3 | Retrieval + Sorting | 1.83 s | 10.5% |
| 4 | LLM-based Selection | 10.85 s | 62.3% |
| LLM Total | 15.58 s | 89.5% |
Table 7: End-to-end latency breakdown of CompCQR on TopiOCQA.
| Method | Retriever | MRR | NDCG@3 |
| LLM4CS | GTR | 42.6 | 41.7 |
| CompCQR | GTR | 47.2 (+4.6) | 45.9 (+4.2) |
| LLM4CS | Contriever | 46.1 | 45.1 |
| CompCQR | Contriever | 48.9 (+2.8) | 47.6 (+2.5) |
Table 8: Results with additional dense retrievers . We report GTR and Contriever results on our framework.
Figure 4: Per-turn performance comparison. We compare per-turn MRR with LLM4CS ( 2023b ) .
| Component | Notation | Dense ( ) | Sparse ( ) |
| Original Query | 1.13 | 0.41 | |
| Rewrite | 9.98 | 6.27 | |
| Rewrite Answer | 16.20 | 14.54 | |
| Followup Query | 5.13 | 3.86 | |
| Followup Answer | 7.13 | 6.32 |
Table 9: Impact of components by Shapley value. Shapley value analysis under dense and sparse retrievers, where all values are scaled by 100.
Figure 5: Position sensitivity of the component. We report the change in MRR relative to the mean performance of queries sharing the same component set, grouped by the position of in the query.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: RRF Softening . We test the impact of softening parameter s on RRF ranking.
| Model | K | Accuracy | MRR | R@5 |
| GPT-4.1-mini | 5 | 66.4% | 41.5 | 49.7 |
| 10 | 56.1% | 42.0 | 52.3 | |
| 20 | 53.2% | 44.3 | 54.6 | |
| 30 | 49.9% | 44.6 | 54.0 | |
| 40 | 49.4% | 45.1 | 54.6 | |
| 50 | 47.2% | 44.7 | 54.7 |
Table 11: LLM best document selection accuracy. We report the accuracy of LLMs on best document selection.
| QR Model | Selection Model | Accuracy | MRR |
| GPT-4.1-mini | Llama-3.1-8B | 48.3% | 43.7 ( -5.0 ) |
| Llama-3.1-8B | GPT-4.1-mini | 52.1% | 46.4 ( +4.1 ) |
Table 12: Impact of selection models. We analyze the effects of the QR model and the document selection model by using different models for each component. The and indicate the performance relative to using the original QR model as the selection model.
| QReCC | TopiOCQA | CAsT-19 | CAsT-20 | |
| # Dialogues | 2775 | 205 | 50 | 25 |
| # Turns | 8209 | 2514 | 479 | 208 |
| # Collections | 54M | 25M | 38M | 38M |
Table 13: Statistics of conversational search test datasets: QReCC, TopiOCQA, CAsT-19, and CAsT-20.
Figure 7: Upper-bound MRR with accumulated queries. We report upperbound of MRR on TopiOCQA.
| Retriever | Top-10 Jaccard Distance |
| ANCE | 28.0% |
| GTR | 22.6% |
| Contriever | 23.2% |
Table 16: Top-10 Jaccard Distance under query-component reordering.
| Method | LLM | MRR | NDCG | R@100 |
| LLM4CS | Qwen2.5-72B | 43.4 | 42.2 | 82.2 |
| CompCQR | Qwen2.5-72B | 47.4 | 46.6 | 83.4 |
| LLM4CS | Llama-3.3-70B | 45.3 | 44.5 | 83.6 |
| CompCQR | Llama-3.3-70B | 48.6 | 47.5 | 84.0 |
Table 17: Retrieval performance with large open-source LLMs. We report results on TopiOCQA using the ANCE retriever.
Figure 8: Topic Shift Performance. We test the framework performance in topic-shift (TS) and topic-concentrated (TC) turns.
| Method | LLM Calls | TFLOPs | MRR |
| CHIQ-Fusion | 6 | - | 38.0 |
| LLM4CS (k=5) | 5 | 69 | 34.6 |
| LLM4CS (k=16) | 16 | 114 | 35.4 |
| CompCQR (w/o LS) | 2 | 74 | 37.8 |
| CompCQR | 3 | 109 | 42.3 |
Table 18: Computational Cost. We compare the number of LLM calls, FLOPs per query, and corresponding TopiOCQA MRR across different methods. We report both CompCQR without LLM selection and the full framework.
| Original Query: Describe core, aggregator, and edge switches. | ||
| Category | Document | LLM Passage Analysis |
| RRF 1st-ranked | “This makes a large core switch a logical way to handle traffic passing between … Top-of-rack switches are designed slightly differently from user edge switches …” | Passage 1 describes core switches and their roles, including their switching capacity and position in the network. … with some details on data center use. |
| LLM-selected (Rank: 3) | “… an additional layer of switching, called the distribution layer, aggregates the edge switches . … The aggregation layer connects to the uplinks of the edge switches and is uplinked to the … network core .” | Passage 3 explicitly describes the distribution and aggregation layers, explaining how aggregation layers connect edge switches and aggregate uplinks. … including the rationale behind the layers and design considerations. |
| Reason for Selection | Passage 3 provides a clear and direct explanation of core, aggregator (aggregation), and edge switches … making it the most suitable to answer the query comprehensively. | |
Table 19: Case study of a hard negative in RRF top-1 ranking. Although the top-ranked RRF document is a hard negative, the LLM correctly identifies the relevant gold document that was originally ranked third.
| Component | Content | Rank |
| Original Query | Regarding the ancient kingdom above; was any innovation made that time? | 6 |
| Rewrite | What innovations were made during the time of the ancient kingdom of Macedon? | 1 |
| Rewrite Answer | …One notable innovation was the development of the phalanx formation, a tactical … | 36 |
| Follow-up Query | What were some of the notable innovations and achievements …particularly during the reign of Alexander the Great? | 7 |
| Follow-up Answer | …particularly during the reign of Alexander the Great, was known for several notable innovations and achievements … | |
| CompCQR (RRF only) | 8 |
Table 20: A failure case from TopiOCQA. We report the gold-passage rank for each component in the rightmost column. The generated components are truncated for readability.
| Method | TopiOCQA | QReCC |
| Conv-ANCE Xiong et al. (2020) | 20.5 | 45.6 |
| ConvDR Yu et al. (2021) | 26.4 | 35.7 |
| Conv-SPLADE Formal et al. (2021) | 29.5 | 46.6 |
| InstructOR-ANCE Jin et al. (2023) | 23.7 | 40.5 |
| LeCoRE Mao et al. (2023c) | 32.0 | 51.1 |
| ConvAug Chen et al. (2024) | 33.3 | 50.4 |
Table 21: Comparison with CDR Methods. We report results on TopiOCQA, and QReCC with NDCG@3.
More in this topic
Conversational Search
Conversational search uses dialogue to interpret evolving information needs, retrieve relevant material, and help people refine or verify results.
See all 23 papers