cs.LGNov 19, 2026

FG-GraphBench: Benchmarking and Enhancing Fine-Grained Visual Understanding with Multimodal Scene Graphs

Authors: Sicheng Liu, Xunkai Li, Ziyu Han, Xingnan Zhang, Ru Zhang, Rong-Hua Li

Organizations: Department of Computer Science, Beijing Institute of Technology · School of Airspace Science and Engineering, Shandong University

Abstract

Fine-grained visual understanding requires multimodal large language models (MLLMs) to recognize small visual entities and understand their relationships. Scene graphs offer a natural way to represent this information, organizing visual entities as nodes and their relationships as edges. However, we identify two challenges: existing scene graph benchmarks provide insufficient coverage of fine-grained entities and the relationships among them, and existing methods for fine-grained visual understanding have yet to effectively leverage such graph-structured information in MLLMs. To address these challenges, we introduce Fine-Grained Multimodal Scene Graph Benchmark (FG-GraphBench) and FiGRA. FG-GraphBench is a benchmark comprising three complementary tasks: fine-grained visual question answering (FG-QA), image captioning (FG-Caption), and image description generation (FG-Description). We develop a two-stage construction pipeline that discovers and grounds fine-grained entities and establishes their relations to construct the fine-grained multimodal scene graphs (MSGs) in FG-GraphBench. Building on these MSGs, we propose FiGRA, a Fine-grained Graph Representation Augmentation method for MLLMs. FiGRA adopts a two-stage training strategy. It encodes multimodal entity and relation features as continuous graph tokens, and integrates them into the MLLM. Experiments show that FiGRA significantly outperforms strong baselines across three tasks of FG-GraphBench. Further analyses confirm the effectiveness of FG-GraphBench and highlight the importance of preserving fine-grained entity and relation information for visual understanding.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models

    Jul 7, 2026Zhiwei Yang, Yuanchen Wu, Nan Zhang +3Multimodal Large Language ModelsMultimodal CoT Reasoning

  2. GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models

    Aug 7, 2026Yuanfu Sun, Yuanhang Ren, Kang Li +5VLM EvaluationVisual Reasoning

  3. GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning

    Mar 9, 2026Jiajin Liu, Dongzhe Fan, Chuanhao Ji +2Multimodal Graph LearningMultimodal Model Evaluation