cs.CLAug 2, 2026

LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning

Authors: Ziyan Xiao, Yinghao Zhu, Wenting Zhang, Heaju Kim, Lequan Yu

Organizations: The University of Hong Kong

Abstract

Multimodal large language models (MLLMs) are rapidly evolving with expanded context windows and stronger reasoning capabilities, enabling multi-chart understanding and multi-step inference. These abilities are increasingly important as MLLMs are adopted in complex agentic tasks. However, existing benchmarks largely emphasize single-chart perception, while simple chart-to-chart connections are insufficient to evaluate these capabilities. To capture multi-chart complexity while ensuring consistency and validity, we design a synthesis pipeline supported by latent graphs. Building on this pipeline, we introduce LongChart, a benchmark whose VQA sets contain an average of 6.5 images and 31.2 questions. We evaluate 10 state-of-the-art MLLMs and examine three factors that influence performance: reasoning patterns, auxiliary tools, and robustness to image perturbations. Our results show that MLLM accuracy decreases and varies substantially as computational complexity increases, highlighting directions for future research in multi-chart reasoning.

Explore similar work

CardsList
  1. Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts

    Apr 23, 2026Azher Ahmed Efat, Seok Hwan Song, Wallapak TavanapongChartMultimodal Query

  2. ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation

    Sep 21, 2026Lijian Wu, Henry Hengyuan Zhao, Zijian Zhang +3