cs.CLNov 17, 2026

MM-ReportBench: A Comprehensive Benchmark for Multimodal Report Generation from Tables

Authors: Teng Lin, Nan Tang

Organizations: DSA Thrust, HKUST(GZ), Guangzhou, China

Abstract

The ability to generate multimodal reports integrating fluent narratives with faithful visualizations from tabular data, has become a cornerstone capability for modern vision-language models in practical domains such as finance, healthcare, and policy making. However, existing benchmarks are limited by single-modality focus, simplistic metrics, novelty evaluation, and visualization quality. We address these limitations with MM-ReportBench, a multimodal report evaluation benchmark covering 386 tasks across 185 real-world tables. Each task is paired with an expert-curated reference report featuring standardized chapter structures and verifiable key insight points, constructed through a hybrid pipeline of multimodal reports generation based on Monte Carlo Tree Search and rigorous human quality control. For robust evaluation, we design a dual-judge evaluation framework: Text Judge Model assesses textual quality along structural completeness, grounded insight verification with numerical fact-checking against source tables, and novelty analysis that distinguishes genuine analytical discovery from trivial paraphrasing; Visualization Judge Model evaluates generated visualizations across data fidelity, expressiveness, aesthetics, and technical correctness. They also evaluate cross-modal consistency. Extensive experiments on 20 state-of-the-art models reveal critical weaknesses: even the strongest model achieves only 68.9 overall score, with numerical hallucination affecting 38.5% of reports and genuinely novel insights remaining rare. Our ablation studies further demonstrate that single-judge evaluation consistently inflates scores by 8-12 points compared to dual-judge assessment, confirming that rigorous and reliable benchmarking demands multi-faceted, complementary evaluation probes.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Monte Carlo Tree Search for Table-to-Multimodal Report Generation

    Aug 4, 2026Teng Lin, Zhiyang Zhang, Yuyu Luo +1Data VisualizationMultimodal Generation

  2. TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexity

    May 7, 2026Zheyuan Yang, Liqiang Shang, Junjie Chen +6Multimodal RobustnessMultimodal Reasoning

  3. TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

    Jun 8, 2026Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee +2Document UnderstandingTable QA