cs.CVNov 19, 2026

Beyond Entity Recognition: Vision-Centric Knowledge Visual Question Answering

Authors: Jiatong Ma, Yuchen Liu, Longteng Guo, Yichen Yuan, Yuze Wu, Dongze Hao, Ru Zhen, Yanhao Zhang, 

Organizations: Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.

Abstract

We propose VC-VQA, a large-scale benchmark for vision-centric knowledge reasoning, aiming to systematically evaluate the capabilities of Multimodal Large Language Models (MLLMs) in fine-grained visual knowledge acquisition and complex multi-stage reasoning. Unlike existing benchmarks that primarily treat visual information as entity recognition and rely on text-based external knowledge, VC-VQA explicitly models the role of visual information by considering both its source (internal vs. external images) and stage of involvement (pre-retrieval vs. post-retrieval). Specifically, VC-VQA requires models to retrieve external visual evidence, identify visual clues before retrieval, and localize, verify, or match retrieved knowledge against the internal image. We further construct a controlled multimodal knowledge base with traceable textual and visual evidence for each sample. We evaluate 21 representative MLLMs under four settings: no external knowledge, with provided evidence, knowledge base retrieval, and open-web search. Experimental results lead to three key findings: current models still struggle with vision-centric knowledge reasoning; fine-grained visual knowledge acquisition remains the primary bottleneck; and agentic retrieval highlights the value of multi-step planning and cross-modal knowledge integration. Overall, VC-VQA provides a more challenging benchmark for evaluating vision-centric knowledge reasoning in MLLMs. Our code and dataset are available at https://github.com/CASIA-IVA-Lab/VC-VQA.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. M3^3-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering

    Apr 28, 2026Jiatong Ma, Longteng Guo, Yuchen Liu +4Multimodal UnderstandingVisual Question Answering

  2. Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

    Apr 1, 2026Marco Morini, Sara Sarto, Marcella Cornia +2Knowledge-Based Visual Question AnsweringVLM Inference