cs.IROct 30, 2026

Defining Page-Centric Visual Document Retrieval and Evaluating Existing Benchmarks

Authors: Chongzhi Zhang, Yuqi Zuo, Aixin Sun

Organizations: Nanyang Technological University Singapore, Singapore

Abstract

In modern information retrieval (IR), documents are increasingly encountered as structured sequences of visually rich, paginated units, rather than as flat text. Example documents encountered in our daily life include Word files, PDFs, and slide decks. Free text-centric retrieval systems either ignore this structure entirely or destroy it through arbitrary passage chunking. In this paper, we formulate page-centric Visual Document Retrieval (VDR) with pages as the fundamental indexing units and document context available for assessing page relevance. The formulation distinguishes two retrieval granularities, page and document, and three retrieval paradigms: independent-item, evidence-set, and iterative retrieval. We then assess 42 benchmarks using eight criteria covering corpus characteristics, query construction, relevance annotation, and ranking support. Our analysis shows two main findings on current VDR research. First, evaluation remains concentrated on page-level independent-item retrieval, with fewer resources supporting document-level, evidence-set, and iterative settings. Second, queries originating from real user information needs independently of target corpus content and graded relevance judgments are uncommon, while the coverage and verification of relevance judgments vary substantially. Both findings reveal gaps between the capabilities applications require and those existing evaluation protocols can establish. We provide benchmark selection guidance and identify priorities for future benchmark design, emphasizing alignment between task requirements, relevance semantics, and evaluation protocols.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval

    Oct 12, 2026Simon Ott, Yufan Chen, Ruiping Liu +6Visual Document RetrievalLong-Context Retrieval

  2. VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval

    Aug 18, 2026Lexiang Hu, Yanzhao Zhang, Mingxin Li +5Visual Document RetrievalAgentic Retrieval

  3. Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval

    Feb 23, 2026Yibo Yan, Jiahao Huo, Guanbo Feng +12Multimodal RAGMultimodal IR