RealRoute: Dynamic Query Routing System via Retrieve-then-Verify Paradigm

Jiahe Liu; Qinkai Yu; Jingcheng Niu; Xi Zhu; Zirui He; Zhen Xiang; Fan Yang; Jinman Zhao

cs.IRMar 2, 2026

RealRoute: Dynamic Query Routing System via Retrieve-then-Verify Paradigm

Authors: Jiahe Liu, Qinkai Yu, Jingcheng Niu, Xi Zhu, Zirui He, Zhen Xiang, Fan Yang, Jinman Zhao

Organizations: Technical University of Denmark · University of Exeter · University of Toronto · Rutgers University ·

Abstract

Despite the success of Retrieval-Augmented Generation (RAG) in grounding LLMs with external knowledge, its application over heterogeneous sources (e.g., private databases, global corpora, and APIs) remains a significant challenge. Existing approaches typically employ an LLM-as-a-Router to dispatch decomposed sub-queries to specific sources in a predictive manner. However, this "LLM-as-a-Router" strategy relies heavily on the semantic meaning of different data sources, often leading to routing errors when source boundaries are ambiguous. In this work, we introduce RealRoute System, a framework that shifts the paradigm from predictive routing to a robust Retrieve-then-Verify mechanism. RealRoute ensures evidence completeness through parallel, source-agnostic retrieval, followed by a dynamic verifier that cross-checks the results and synthesizes a factually grounded answer. Our demonstration allows users to visualize the real-time "re-routing" process and inspect the verification chain across multiple knowledge silos. Experiments show that RealRoute significantly outperforms predictive baselines in the multi-hop Rag reasoning task. The RealRoute system is released as an open-source toolkit with a user-friendly web interface. The code is available at the URL: https://github.com/Joseph1951210/RealRoute.

Figures & tables

System	Multi-Hop	Multi-Source Retrieval	Cross-Domain	Knowledge Check
Naive RAG Lewis et al. (2020)	✗	✗	✗	✗
HippoRAG Jimenez Gutierrez et al. (2024)	✓	✗	✗	✗
GraphRag system Edge et al. (2024)	✗	✓	✗	✗
Deepsieve system Guo et al. (2026)	✓	✓	✗	✗
Real Route	✓	✓	✓	✓

Table 1: Comparison of Real Route with representative RAG systems across key system capabilities, including multi-hop reasoning, multi-source retrieval, cross-domain generalization, and built-in knowledge verification.

Figure 1: Overall framework of the multi-source retrieval and evidence check pipeline. Phase I performs context-aware parallel retrieval by decomposing the input query into dependency-aware subqueries, predicting preferred sources, and retrieving top-k candidates from heterogeneous knowledge bases (e.g., local corpus, global corpus, BioASQ, SciQ..) to form a unified candidate pool. Phase II applies dynamic evidence selection via an adaptive token budget (cap) allocated per source, followed by score-based filtering to retain the most informative evidence. Phase III fuses the selected subquery answers within a structured prompt to produce the final output.

Model	Setting	Method	EM	F1	Avg Tokens
GPT-4o	2-source	Hard Routing (Baseline)	49.30	61.70	3926.6
		Real Route (Ours)	53.67	66.93	2711.5
	3-source	Hard Routing (Baseline)	52.33	63.90	1881.6
		Real Route (Ours)	53.00	64.61	2553.6
	4-source	Hard Routing (Baseline)	56.67	68.08	1941.2
		Real Route (Ours)	56.67	67.09	2651.3

Table 2: Overall question answering performance. Hard Routing serves as the baseline. The best results within each setting are highlighted in bold , and our proposed method is emphasized with a shaded background.

Paradigm	EM	F1	#Tokens
Direct	36.2	28.0	55.5
IR-CoT Trivedi et al. (2023b)	30.8	22.4	481.9
ReAct Yao et al. (2022)	39.6	32.2	9795.1
ReWOO Xu et al. (2023)	40.1	30.4	1986.2
Reflexion Shinn et al. (2023)	46.7	62.5	37893.0
DeepSieve Guo et al. (2026)	49.3	61.7	3926.6

Table 3: Comparison across QA reasoning paradigms on HotPotQA. All results are reported under the same benchmark configuration. The best results within each setting are highlighted in bold , and our proposed method is emphasized with a shaded background.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 2 — Figure 2: The user interface of the experiment: (a) shows the initial configuration, while (b) details the custom source settings.

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 766 papers

Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation
Mar 10, 2026Eeham Khan, Luis Rodriguez, Marc QueudotRAG Citation Faithfulness Retrieval-Augmented Generation
Concordia University, Montréal, QC, Canada · Centre de recherche informatique de Montréal (CRIM), Montréal, QC, Canada
Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking
May 31, 2026Fachrina Dewi Puspitasari, Chaoning Zhang, Jiaquan Zhang +6Multi-Hop QA Document Chunking
School of Computer Science and Engineering, University of Electronic Science and Technology of China · Massachusetts General Hospital, Harvard University · Nota AI
Retrieval is Cheap, Show Me the Code: Executable Multi-Hop Reasoning for Retrieval-Augmented Generation
May 13, 2026Jiashuo Sun, Jimeng Shi, Yixuan Xie +10Multi-Hop QA Retrieval-Augmented Generation
University of Illinois Urbana-Champaign · Hong Kong University of Science and Technology · Texas A&M University +1

Mar 10, 2026J/K move · Enter open · S save

Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation

Eeham Khan, Luis Rodriguez, Marc Queudot

Concordia University, Montréal, QC, Canada · Centre de recherche informatique de Montréal (CRIM), Montréal, QC, Canada

RAG Citation Faithfulness Retrieval-Augmented Generation RAG Robustness Faithfulness of Language Model Explanations Biomedical QA Retrieval-Augmented ICL

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) significantly improves the factuality of Large Language Models (LLMs), yet standard pipelines often lack mechanisms to verify inter- mediate reasoning, leaving them vulnerable to hallucinations in high-stakes domains. To address this, we propose a domain-specific RAG framework that integrates explicit rea- soning and faithfulness verification. Our architecture augments standard retrieval with neural query rewriting, BGE-based cross-encoder reranking, and a rationale generation module that grounds sub-claims in specific evidence spans. We further introduce an eight-category verification taxonomy that enables fine-grained assessment of rationale faithfulness, distinguishing between explicit and implicit support patterns to facilitate structured error diagnosis. We evaluate this framework on the BioASQ and PubMedQA benchmarks, specifically analyzing the impact of dynamic in-context learning and rerank- ing under constrained token budgets. Experiments demonstrate that explicit rationale generation improves accuracy over vanilla RAG baselines, while dynamic demonstration selection combined with robust reranking yields further gains in few-shot settings. Using Llama-3-8B-Instruct, our approach achieves 89.1% on BioASQ-Y/N and 73.0% on Pub- MedQA, competitive with systems using significantly larger models. Additionally, we perform a pilot study combining human expert assessment with LLM-based verification to explore how explicit rationale generation improves system transparency and enables more detailed diagnosis of retrieval failures in biomedical question answering.

Abstract

Retrieval-Augmented Generation

Explore similar work

Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation

Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking

Retrieval is Cheap, Show Me the Code: Executable Multi-Hop Reasoning for Retrieval-Augmented Generation