Entropic Claim Resolution: Uncertainty-Driven Evidence Selection for RAG

Davide Di Gioia

cs.AIMar 30, 2026

Entropic Claim Resolution: Uncertainty-Driven Evidence Selection for RAG

Authors: Davide Di Gioia

Abstract

Current Retrieval-Augmented Generation (RAG) systems predominantly rely on relevance-based dense retrieval, sequentially fetching documents to maximize semantic similarity with the query. However, in knowledge-intensive and real-world scenarios characterized by conflicting evidence or fundamental query ambiguity, relevance alone is insufficient for resolving epistemic uncertainty. We introduce Entropic Claim Resolution (ECR), a novel inference-time algorithm that reframes RAG reasoning as entropy minimization over competing semantic answer hypotheses. Unlike action-driven agentic frameworks (e.g., ReAct) or fixed-pipeline RAG architectures, ECR sequentially selects atomic evidence claims by maximizing Expected Entropy Reduction (EER), a decision-theoretic criterion for the value of information. The process dynamically terminates when the system reaches a mathematically defined state of epistemic sufficiency (H <= epsilon, subject to epistemic coherence). We integrate ECR into a production-grade multi-strategy retrieval pipeline (CSGR++) and analyze its theoretical properties. Our framework provides a rigorous foundation for uncertainty-aware evidence selection, shifting the paradigm from retrieving what is most relevant to retrieving what is most discriminative.

Figures & tables

Panel of Figure 2 — Figure 2: System overview: Entropic Claim Resolution (ECR) operates as an inference-time controller between competitive retrieval and answer synthesis. Given a retrieved claim set, ECR sequentially selects evidence to minimize hypothesis entropy and terminates when epistemic sufficiency is reached.

Subsystem	Role in the pipeline
AutoAdaptAgent + SchemaFeedbackLoop	Schema inference with dry-run calibration
GenericStore (EAV SQLite)	Item/attribute persistence; safe SELECT -only SQL execution
EmbeddingProvider + VectorIndex	3-tier embeddings; LanceDB/NumPy backends; ID-filtered cosine search
ClaimExtractor + ClaimStore	Atomic claims + relations + temporal keys + dynamic confidence
EnsembleRetriever	Competitive retrieval + RRF fusion (Eq. 9 )
EntropicClaimResolver	ECR loop: entropy, EER selection (Eq. 4 )

Table 1: Key subsystems implemented in our system that support ECR and the full end-to-end pipeline.

Policy	Claims	$H_{final}$	$\Delta H$ /claim	Collapse	$2^{H_{final}}$	Redund.	HypCondRed.	SrcEnt
Retrieval-only	$15.0\pm 0.0$	$1.585\pm 0.000$	$0.0000\pm 0.0000$	$16.0\pm 0.0$	$3.000\pm 0.000$	$0.684\pm 0.119$	$0.662\pm 0.110$	$0.342\pm 0.510$
ECR	$5.0\pm 0.0$	$0.2129\pm 0.0000$	$0.2744\pm 0.0000$	$5.0\pm 0.0$	$1.159\pm 0.000$	$0.672\pm 0.125$	$0.672\pm 0.125$	$0.276\pm 0.443$
Random	$5.0\pm 0.0$	$1.243\pm 0.289$	$0.0684\pm 0.0577$	$6.0\pm 0.0$	$2.411\pm 0.437$	$0.658\pm 0.118$	$0.653\pm 0.123$	$0.354\pm 0.527$

Table 2: Claims-only evaluation (80 cases, seed=7). “Claims” is the number of evaluated claims.

H_{final}

is final answer-hypothesis entropy in bits. “Collapse” is claims-to-collapse (first step where

H\leq\epsilon=0.3

; else budget

+1

). “Redund.” is claim redundancy, “HypCondRed.” is hypothesis-conditioned claim redundancy, and “SrcEnt” is source entropy (diagnostic diversity metrics).

Policy	$H_{final}$	$\Delta H$ /claim	Collapse	$2^{H_{final}}$	TraceVar	HypCondRed.
Retrieval-only	$1.585\pm 0.000$	$0.0000\pm 0.0000$	$16.00\pm 0.00$	$3.000\pm 0.000$	$0.002987\pm 0.000000$	$0.6619\pm 0.0000$
ECR	$0.2129\pm 0.0000$	$0.2744\pm 0.0000$	$5.00\pm 0.00$	$1.159\pm 0.000$	$0.262859\pm 0.000000$	$0.6719\pm 0.0000$
Random	$1.2628\pm 0.0265$	$0.0644\pm 0.0053$	$5.995\pm 0.006$	$2.436\pm 0.044$	$0.03210\pm 0.00374$	$0.6401\pm 0.0075$

Table 3: Multi-seed robustness (seeds 0–4): mean

\pm

std over seeds of the seed-level mean metrics. Only the random baseline changes across seeds in this frozen setup. “HypCondRed.” is hypothesis-conditioned claim redundancy.

Panel of Figure 3 — Figure 3: Schematic entropy trajectories consistent with the measured endpoints: ECR reaches $H\leq\epsilon$ quickly, whereas relevance-only and random baselines typically remain above $\epsilon$ at matched claim budgets.

Method	Avg. Claims Used	Exact Match (EM) $\uparrow$	Token F1 $\uparrow$	Evidence Faithfulness $\uparrow$
Baseline RAG	19.87	0.313	0.459	0.639
Random Control	19.87	0.207	0.307	0.427
ECR (ours)	19.68	0.297	0.450	0.626

Table 4: End-to-End Evaluation on HotpotQA-Style Multi-Hop QA (300 Questions). All methods use the same retriever and language model; only the inference-time evidence selection policy differs. ECR substantially outperforms random selection while maintaining performance comparable to a strong relevance-based baseline.

Method	EM (No Noise)	Faith (No Noise)	EM (40% Noise)	Faith (40% Noise)
Baseline RAG	0.323	0.660	0.167	0.345
ECR (ours)	0.307	0.657	0.163	0.331

Table 5: Robustness ablation on HotpotQA-style evaluation (300 questions) with noise injected after retrieval and before evidence selection . “40% Noise” replaces 40% of retrieved candidate claims with unrelated (potentially contradictory) claims sampled from a noise pool. Only baseline relevance-based RAG and ECR are evaluated; the retriever, LLM, prompts, and decoding are unchanged. (Performance is bounded above when ground-truth evidence is removed.)

Method	$\boldsymbol{\alpha}$	EM	OverconfErr	AmbExp	Mean $H$	Stop Reason
Baseline RAG	0.0	0.0067	0.9933	0.0000	–	fixed_budget (300/300)
Baseline RAG	0.3	0.0067	0.9933	0.0000	–	fixed_budget (300/300)
Baseline RAG	0.5	0.0067	0.9933	0.0000	–	fixed_budget (300/300)
ECR (ours)	0.0	0.0000	0.9900	0.0100	0.226	epistemic_sufficiency (297/300)
ECR (ours)	0.3	0.0067	0.0000	1.0000	1.496	unresolved_conflict (300/300)
ECR (ours)	0.5	0.0067	0.0000	1.0000	1.458	unresolved_conflict (300/300)

Table 6: Offline contradiction-injection ablation (300 questions). Paired contradictions are injected into the candidate claim pool at rate

\alpha

after retrieval and before evidence selection. EM is reported only as a sanity anchor under a deterministic offline answerer; the key signals are Ambiguity Exposure and Overconfident Error (epistemic collapse). ECR exhibits a phase transition from epistemic sufficiency to principled non-convergence as contradictions accumulate, while baseline RAG remains uniformly overconfident. Counts indicate number of runs terminating for each reason.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure A.1: Ambiguity exposure as a function of the coherence weight $\lambda_{\mathrm{conflict}}$ under structured contradiction injection. Empirically, ambiguity exposure exhibits a sharp phase transition: for $\alpha=0.5$ , exposure jumps from 0 to 1 for any tested $\lambda>0$ , while remaining 0 for $\alpha\leq 0.3$ across all tested settings.

$\boldsymbol{\lambda}$	$\boldsymbol{\alpha=0.0}$		$\boldsymbol{\alpha=0.3}$		$\boldsymbol{\alpha=0.5}$
	MeanClaims	Mean $H$	MeanClaims	Mean $H$	MeanClaims	Mean $H$
0.00	5.04	0.226	5.06	0.226	5.08	0.226
0.01	5.04	0.226	25.83	1.496	29.81	1.458
0.025	5.04	0.226	25.83	1.496	29.81	1.458
0.05	5.04	0.226	25.83	1.496	29.81	1.458
0.10	5.04	0.226	25.83	1.496	29.81	1.458

Appendix

Table A.1:

\lambda

-sweep summary statistics (offline, deterministic). Values are aggregated over 300 questions.

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 792 papers

ERA: Evidence-based Reliability Alignment for Honest Retrieval-Augmented Generation
Feb 24, 2026Sunguk Shin, Meeyoung Cha, Byung-Jun Lee +1RAG Robustness LLM Abstention
Korea University, Seoul, South Korea · MPI-SP, Bochum, Germany · KAIST, Daejeon, South Korea +1
EvidentialRAG: Quantifying and Mitigating Information Conflict in Multi-Source Retrieval-Augmented Generation via Evidential Deep Learning
Jul 11, 2026S M Asif Hossain, Ruksat Khan Shayoni, M. F. MridhaRetrieval-Augmented Generation Evidential Deep Learning
aSchool of Computing, Wichita State University, Kansas, USA · bDepartment of Computer Science, American International University-Bangladesh, Dhaka, Bangladesh
Uncertainty-Aware Hybrid Retrieval for Long-Document RAG
Jun 11, 2026Hoin Jung, Xiaoqian WangRetrieval-Augmented Generation Information Retrieval
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907

Feb 24, 2026J/K move · Enter open · S save

ERA: Evidence-based Reliability Alignment for Honest Retrieval-Augmented Generation

Sunguk Shin, Meeyoung Cha, Byung-Jun Lee, Sungwon Park

Korea University, Seoul, South Korea · MPI-SP, Bochum, Germany · KAIST, Daejeon, South Korea+1

RAG Robustness LLM Abstention Knowledge Conflicts in Language Models Confidence Estimation in Language Models Epistemic Uncertainty

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) grounds language models in factual evidence but introduces critical challenges regarding knowledge conflicts between internalized parameters and retrieved information. However, existing reliability methods, typically relying on scalar confidence, fail to explicitly distinguish between epistemic uncertainty and inherent data ambiguity in such hybrid scenarios. In this paper, we propose a new framework called ERA (Evidence-based Reliability Alignment) to enhance abstention behavior in RAG systems by shifting confidence estimation from scalar probabilities to explicit evidence distributions. Our method consists of two main components: (1) Contextual Evidence Quantification, which models internal and external knowledge as independent belief masses via the Dirichlet distribution, and (2) Quantifying Knowledge Conflict, which leverages Dempster-Shafer Theory (DST) to rigorously measure the geometric discordance between information sources. These components are used to disentangle epistemic uncertainty from aleatoric uncertainty and modulate the optimization objective based on detected conflicts. Experiments on standard benchmarks and a curated generalization dataset demonstrate that our approach significantly outperforms baselines, optimizing the trade-off between answer coverage and abstention with superior calibration.

Abstract

Retrieval-Augmented Generation

Explore similar work

ERA: Evidence-based Reliability Alignment for Honest Retrieval-Augmented Generation

EvidentialRAG: Quantifying and Mitigating Information Conflict in Multi-Source Retrieval-Augmented Generation via Evidential Deep Learning

Uncertainty-Aware Hybrid Retrieval for Long-Document RAG