cs.CLNov 2, 2026

EviCast: Forecasting Retrieval Gains Through Shared Response Calibration

Authors: Yibo Wang, Jisheng Dang, Bimei Wang, Jiongpei Hou, Jingwen Zhao, Wenjun Ma, Fei Shen, Juncheng Li, 

Organizations: Lanzhou University, Lanzhou, China. · Nanjing University of Science and Technology, Nanjing, China. · Zhejiang University, Hangzhou, China. · the School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230009, China. · 

Abstract

Retrieval-augmented generation (RAG) extends large language models with external evidence, yet relevant evidence does not ensure that a generator can use it. Indiscriminate retrieval can degrade task performance while adding retrieval and long-context inference costs. Existing methods neither determine before large-scale execution whether a generator, task distribution, and retriever configuration is worth deploying nor diagnose multiple candidate retrievers under a shared budget. We introduce EviCast, a framework that forecasts retrieval value through interventional evidence-state decomposition. EviCast measures a generator's response to controlled evidence states once and combines this shared profile with the offline evidence-state distribution of each candidate retriever, yielding the end-to-end net utility and its 95% lower confidence bound without executing every RAG configuration. A state-refinement analysis characterizes when its binary and topology-aware instantiations are exact and when shared calibration is statistically more efficient than per-retriever calibration. In frozen evaluations, EviCast correctly decides all 14 HotpotQA retrieval configurations with a forecast error of 2.75 F1 points while reducing diagnostic generation calls by 80%, and it attains the lowest forecast error among seven predictors in the low-budget regime. Its topology-aware instantiation, EviCast-T, captures the complementarity between terminal and bridge evidence, correctly deciding all 16 configurations on MuSiQue and HotpotQA and 23 of 24 after prospective transfer to a third dataset. These results show that retrieval deployment depends on both evidence availability and the generator's ability to convert evidence structure into stable net utility. Code is available at https://github.com/YiboWang2002/EviCast.

Figures & tables

Explore similar work

CardsList
  1. Know Before You Fetch: When Does Confidence-Based Retrieval-Budget Allocation Pay Off?

    Jun 29, 2026Zhe Dong, Fang Qin, Manish ShahRetrieval-Augmented GenerationSelective Prediction

  2. Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG

    Aug 26, 2026Utshab Kumar Ghosh, Debayan Mukhopadhyay, Shubham ChatterjeeRAG EvaluationIR Evaluation

  3. Two-Sided Conformal Evidence Sets for Retrieval-Augmented Generation

    Nov 7, 2026Ben Jenkins, Mihaela CardeiEvidence SelectionRAG Evaluation