cs.AINov 17, 2026

Synergizing Evidence Retrieval and Utilization for Self-Improving Multimodal Scientific Document Reasoning

Authors: Frank Nie, Ethan B. Liu, Yuan Zhu, Jindong Han

Organizations: Shandong University, China

Abstract

Scientific document reasoning requires an agent to acquire and integrate evidence distributed across text, figures, and tables. Retrieval can identify relevant sources while missing complementary support, and answer-level feedback alone does not reveal whether an error stems from evidence acquisition or evidence use. We present an agentic Retriever--Perceiver--Reasoner framework for multimodal scientific document reasoning. The Retriever sequentially selects source-aligned evidence under a context budget. Given the selected evidence and question, the Perceiver constructs a question-conditioned latent ontology, an implicit organization of multimodal evidence in a shared continuous space. The Reasoner combines this latent guidance with the original sources to produce grounded answers. We train the three components through diagnostic-guided co-evolution: alternating updates optimize them with a shared answering reward, while Retrieval Potential Feedback (RPF) and Latent Efficacy Feedback (LEF) identify questions that need better retrieval or latent guidance and direct targeted resampling. Across four benchmarks and seven evaluation settings, our framework obtains average scores of 69.9 and 75.0 with 7B and 8B Reasoner backbones, exceeding the strongest baseline by 4.0 and 9.1 points, respectively.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Hybrid Retriever Evolution for Multimodal Document Reasoning Agents

    Jun 28, 2026Bohan Yao, Shruthan Radhakrishna, Vikas YadavMultimodal RAGMultimodal QA

  2. SciMDR: Advancing Scientific Multimodal Document Reasoning

    Mar 12, 2026Ziyu Chen, Yilun Zhao, Chengye Wang +3Multimodal ReasoningMultimodal QA