Knowledge-Based Visual Question Answering

Also known as VQA

Overview

Knowledge-based visual question answering studies questions that require combining what is visible in an image with facts from sources such as Wikipedia, knowledge graphs, documents, or a model’s internal knowledge. Common approaches retrieve images or text, identify entities, and use multimodal language models or search agents to select evidence and reason over it; other methods organize evidence as graphs or highlight relevant regions and passages. Training ranges from supervised retrieval and answer generation to reinforcement learning for multi-step search, while benchmarks test encyclopedic questions, multi-hop reasoning, and specialized settings such as medicine or navigation. A central challenge is ensuring that models use relevant, trustworthy evidence rather than noisy or conflicting retrievals, and that evaluations measure grounded reasoning rather than answer accuracy alone.

Latest papers

CardsList

45 papers1 paper in the last four weeks, down 80% on the four weeks before. 0.0% of all new papers.

  1. Beyond Entity Recognition: Vision-Centric Knowledge Visual Question Answering

    Nov 19, 2026Jiatong Ma, Yuchen Liu, Longteng Guo +7Knowledge-Based Visual Question AnsweringMLLM Reasoning

  2. SAGE-VQA: A Training-Free, Self-Evolving Agentic Framework with Grounded Evidence for Knowledge-Based Visual Question Answering

    Nov 15, 2026Ruirui Shen, Jianqi Gao, Xiangfeng Luo +5Knowledge-Based Visual Question AnsweringReasoning for VLMs

  3. VisualWorldBench: A Fine-Grained Multi-Task Benchmark for Evaluating Visual World Knowledge in MLLMs

    Oct 14, 2026Chen Chen, Zheming Liang, Jiaqi WangVLM EvaluationKnowledge-Based Visual Question Answering

  4. A doctrine-grounded visual question answering dataset for Tactical Combat Casualty Care

    Oct 5, 2026Junseob Kim, Jade Chng, Ayman Ali +5Medical Visual Question AnsweringVideo Question Answering

  5. Question-Specific Knowledge Graphs for Efficient Visual Reasoning

    Sep 28, 2026Ting-Chih Chen, Emile van Krieken, Shujian Yu +1VLM ReasoningKnowledge-Based Visual Question Answering

  6. SAFE-G: Structure-aware Faithful Evidence-guided Generation for Knowledge-based Visual Question Answering

    Aug 22, 2026Long Shu, Shuochen Liu, Wei Chen +4Knowledge-Based Visual Question AnsweringMultimodal RAG

  7. Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering

    Aug 19, 2026Hangrui Xu, Zhengxian Wu, Yunyao Yu +6Knowledge-Based Visual Question AnsweringVision-Language Retrieval

  8. Boosting Knowledge-based Visual Question Answering with Structured Context Reasoning

    Aug 17, 2026Qiyou Liu, Yong Zhang, Jianjie Luo +2Knowledge-Based Visual Question AnsweringVLM Reasoning

  9. JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis

    Aug 12, 2026Ran Li, Huiguo He, Jiahuan Cao +3VLM EvaluationVLM Reasoning

  10. Bayesian Data Reweighting Improves Multimodal Retrieval for Knowledge-Based Visual Question Answering

    Aug 3, 2026Jingchen Sun, Shaobo Han, Ruiyi Zhang +5Multimodal IRContrastive Learning

  11. UniHEAR: Unified Heterogeneous-Source Attentive Retrieval for Knowledge-Based Visual Question Answering

    Aug 2, 2026Ganzhong Luo, Yang Ren, Hanyong Wang +2Multimodal IRMultimodal Reranking

  12. Distilling Answer Set Programming Theories from Large Language Models

    Jul 30, 2026Nelson Higuera Ruiz, Markus Hofmarcher, Claudiu Leoveanu-CondreiLogic ProgrammingLLM-Based Program Synthesis

  13. Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering

    Jul 28, 2026Noor Islam S. Mohammad, Uluğ BayazıtMultimodal RAGEfficient Multimodal Inference

  14. CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA

    Jul 23, 2026Hanseok Oh, Parishad BehnamGhader, Benno Krojer +4VLM EvaluationMultimodal RAG

  15. WasteAssistant: Regulation-Guided Visual Question Answering Framework for Intelligent Waste Segregation and Sustainable Managemen

    Jul 12, 2026Khush Kataruka, Harshit Maurya, Anuja Vats +3Visual Question AnsweringKnowledge-Based Visual Question Answering

  16. MMAgent-R2^2: Learning to Rerank and Reject for Agentic mRAG

    Jul 8, 2026Tao Zhang, Ziqi Zhang, Zongyang Ma +7Multimodal RAGAdaptive Retrieval

  17. Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmenting

    Jun 30, 2026Qian Ma, S M Rayeed, Charles V. Stewart +2VLM EvaluationVisual Question Answering

  18. ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

    Jun 26, 2026ZhengXian Wu, Hangrui Xu, Kai Shi +8Multimodal RAGMultimodal Search Agents

  19. Ground Then Rank: Revisiting Knowledge-Based VQA with Training-Free Entity Identification

    Jun 22, 2026Qian Ma, Qiong Wu, Zhengyi Zhou +1Multimodal RAGVisual Question Answering

  20. VL-MemKnG: Hybrid Memory with a Spatio-Temporal Knowledge Graph for Question Answering over Long Egocentric Navigation Trajectories

    Jun 15, 2026Svetlana Lukina, Mohamad Al Mdfaa, Gloria Haro +2Egocentric VQAVideo Question Answering

  21. Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering

    Jun 15, 2026Jieyuan Liu, Jianyang Gu, Shijie Chen +2VLM EvaluationMultimodal RAG

  22. Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation

    May 5, 2026Quanxing Xu, Ling Zhou, Xian Zhong +3Multimodal RAGVisual Question Answering

  23. M3^3-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering

    Apr 28, 2026Jiatong Ma, Longteng Guo, Yuchen Liu +4Multimodal UnderstandingVisual Question Answering

  24. BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering

    Apr 24, 2026Jinghong Chen, Jingbiao Mei, Guangyu Yang +1Visual Question AnsweringRetrieval-Augmented Generation

  25. KIRA: Knowledge-Intensive Image Retrieval and Reasoning Architecture for Specialized Visual Domains

    Apr 18, 2026Parthaw Goswami, Jaynto Goswami DeepMultimodal RAGVisual Reasoning