Chart QA

QA: Question Answering

Chart question answering asks models to interpret plotted data and answer questions about values, trends, comparisons, and relationships, combining visual perception with reasoning. Systems use OCR and axis or legend grounding alongside multimodal language models; newer methods add chart structure representations, visual tools, executable code, and reinforcement learning with verifiable rewards. Evaluation covers synthetic and real-world charts, including scientific and financial figures, multi-chart and interactive tasks, and tests of evidence localization and precise value extraction. Despite gains from prompting and specialized training, models still struggle with dense or misleading charts, unfamiliar formats, complex multi-step reasoning, exact numerical recovery, and grounding answers in visible evidence.

Latest papers

33 papers5 papers in the last four weeks, up 25% on the four weeks before. 0.0% of all new papers.

Improving VLMs on Scientific Chart Analysis via Autonomous Evolution
Oct 26, 2026Abhinav Sudhakar Dubey, Vidushi Goel, Kiran Rana +2Chart QA VLM Evaluation
University of California, Santa Cruz · Airbnb Inc.
Code-Interactive Visual Reasoning for Financial Candlestick Understanding
Oct 12, 2026Chenxi Zhang, Youwei Pang, Xiaoqi Zhao +4Chart QA Chart Understanding
Wuhan University · HiThink Research · Nanyang Technological University +1
Do you "see" what I "see"? A Multi-panel Visual Question Answer Dataset for Large Language Model Chart Analysis
Oct 9, 2026Daniel J. Evans, J. P. Naiman, JooYoung Seo +1Scientific Figure Understanding Chart QA
School of Information Sciences, University of Illinois, Urbana-Champaign, 61820, USA
ChartBmkAgent: Harness-Governed Multi-Agent Construction of Chart QA Benchmarks from Sparse Error-Taxonomy Specifications
Oct 6, 2026Langxi Huang, Pingping Zhang, Lanyun Zhu +4Synthetic Benchmark Generation LLM Evaluation
City University of Hong Kong · Tongji University · Hong Kong University of Science and Technology
PlotGround: Grounding Plot Digitization in Real Scientific Figures and Their Source Data
Oct 5, 2026Yaohui Zhang, Binxu Li, Haoyi Duan +7Scientific Visualization Chart QA
Generative Expert Labs, Inc · Stanford University · Princeton University
When Does an Image Determine the Answer? Benchmarking Visual Answerability across Charts and Scenes
Sep 28, 2026Sungguk Cha, Mintae Kim, Youngsub Han +2VLM Evaluation Visual Question Answering
LG Uplus, Seoul, South Korea
Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth
Sep 12, 2026Tianhao Niu, Qingfu Zhu, Wanxiang CheChart QA VLM Interpretability
Research Center for Social Computing and Interactive Robotics Harbin Institute of Technology, China
Charts Are Beyond Pixels: Probing for Layer-Wise Chart Understanding and Editing
Sep 8, 2026Xiaochuan Zhong, Yifan Hou, Chenxi Pang +1VLM Evaluation Chart QA
DeepDelta Lab, School of Artificial Intelligence, Shanghai Jiao Tong University · ETH Zurich · Google DeepMind
Structure-Token Evidence-Anchored Reasoning for Scientific Chart Understanding
Sep 7, 2026Alberlucia Rafael Soarez, Camila Ferreira, Daniel Kim +2Chart Understanding Chart QA
University of Brasilia
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
Sep 2, 2026Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park +5Multimodal Large Language Models Chart QA
University of Wisconsin-Madison · Massachusetts Institute of Technology · Stanford University +2
ChartProbe: A Diagnostic Study on Visual Reasoning through Perception, Grounding, and Simple Reasoning
Aug 13, 2026Mahsa Khoshnoodi, Sarah Adel BargalChart Understanding Chart QA
Department of Computer Science, Georgetown University
Chartography: A Benchmark for Professional Chart Understanding
Aug 11, 2026Suhaas Garre, Chris Mutty, Sushant Mehta +1Data Visualization Chart QA
Surge AI*
CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning
Aug 3, 2026Xuehang Guo, Pingyue Zhang, Ruiyi Zhang +6Data Visualization Chart QA
1William & Mary · 2Northwestern University · 3Adobe +1
LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning
Aug 2, 2026Ziyan Xiao, Yinghao Zhu, Wenting Zhang +2Data Visualization Visual Reasoning
The University of Hong Kong
AgentFinVQA: A Deployable Multi-Agent Pipeline for Auditable Financial Chart QA
Jun 18, 2026Aravind Narayanan, Shaina RazaLLM Answer Verification Financial QA
Vector Institute
SAFE-Cascade: Cost-Adaptive Vision-Language Routing for Chart Question Answering
Jun 17, 2026Ayush Dwivedi, Qixin Wang, Ashvi Soni +5Multimodal Understanding Efficient VLM Inference
Walmart USA · University of Arkansas USA
HakushoBench: A Japanese Chart and Table VQA Benchmark from Governmental White Papers
May 31, 2026Issa Sugiura, Shuhei Kurita, Yusuke Oda +1VLM Evaluation Data Visualization
Institute of Science Tokyo · NII LLMC · NII
Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models
May 26, 2026Yifan Jiang, Dae Yon Hwang, Jesse C. Cresswell +1VLM Evaluation Data Visualization
University of Waterloo · Vector Institute · Layer 6 AI
ChartAct: A Benchmark for Dynamic Chart Understanding
May 26, 2026Muye Huang, Lin Wu, Lingling Zhang +5Data Visualization Chart QA
School of Computer Science and Technology, Xi’an Jiaotong University · 2MOE KLNN Lab, Xi’an Jiaotong University
PlotPick: AI-powered batch extraction of numerical data from scientific figures
May 7, 2026Tommy CarstensenData Visualization Vision-Language Models
Copenhagen Research Centre for Biological and Precision Psychiatry, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark
Hierarchical Visual Agent: Managing Contexts in Joint Image-Text Space for Advanced Chart Reasoning
May 5, 2026Qihua Dong, Ruozhen He, Junwen Chen +4Data Visualization Chart QA
1Northeastern University · 2Rice University · 3Amazon AGI
Generating Statistical Charts with Validation-Driven LLM Workflows
May 1, 2026Pavlin G. Poličar, Andraž Pevcin, Blaž ZupanData Visualization Multimodal Large Language Models
University of Ljubljana · Faculty of Computer and Information Science · Veˇcna pot 113, 1000 Ljubljana, Slovenia
DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams
Apr 28, 2026Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Gaurav Najpande +4VLM Evaluation Visual Reasoning
Arizona State University · Adobe Research · University of Maryland
Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts
Apr 23, 2026Azher Ahmed Efat, Seok Hwan Song, Wallapak TavanapongData Visualization Multimodal Large Language Models
Department of Computer Science, Iowa State University, Ames, Iowa, USA
Beyond Pixels: Introspective and Interactive Grounding for Visualization Agents
Apr 22, 2026Yiyang Lu, Woong Shin, Ahmad Maroof Karimi +3Scientific Visualization Data Visualization
William & Mary · Oak Ridge National Laboratory
Chart-RL: Policy Optimization Reinforcement Learning for Enhanced Visual Reasoning in Chart Question Answering with Vision Language Models
Apr 3, 2026Yunfei Bai, Amit Dhanda, Shekhar JainChart QA VLM Reasoning
Amazon
CharTool: Tool-Integrated Visual Reasoning for Chart Understanding
Apr 3, 2026Situo Zhang, Yifan Zhang, Zichen Zhu +6Chart Understanding Chart QA
X-LANCE Lab, Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China · Shanghai Innovation Institution, Shanghai, China +1
Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering
Mar 30, 2026Yanjie Zhang, Yafei Li, Rui Sheng +5VLM Robustness Chart QA
HKUST, Hong Kong SAR, China · HKUST(GZ), Guangzhou, China
ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding
Mar 28, 2026Jovana Kondic, Pengyuan Li, Dhiraj Joshi +24Chart Understanding Data Visualization
MIT · IBM Research · MIT-IBM Watson AI Lab +1
VisDoT : Enhancing Visual Reasoning through Human-Like Interpretation Grounding and Decomposition of Thought
Mar 12, 2026Eunsoo Lee, Jeongwoo Lee, Minki Hong +2VLM Reasoning Visually Grounded Reasoning
Department of Computer Science and Artificial Intelligence, Dongguk University · Department of Electronics and Electrical Engineering, Dongguk University

Oct 26, 2026J/K move · Enter open · S save

Improving VLMs on Scientific Chart Analysis via Autonomous Evolution

Abhinav Sudhakar Dubey, Vidushi Goel, Kiran Rana, Shikhar Mathur+1

University of California, Santa Cruz · Airbnb Inc.

Chart QA VLM Evaluation

Open paper AskPDF arXiv

Understanding and operating with scientific charts is a standard approach in data analysis, and while humans are good at interpreting such charts, open vision-language models (VLMs) are known to perform poorly when it comes to question answering based on such data types. Motivated by an initial observation that many of their errors are due to inexact answering, we raise the following natural question: how much accuracy can we recover without changing the model at all? In this paper, we study this question by searching over inference programs and the code around a "frozen" version of the model. Using ideas from autonomous evolution (famously used in AlphaEvolve), we first start from a baseline inference program, and in each round we show an LLM judge the code together with a few failures on training questions, and ask for an improving evolutionary move. As usual in autonomous evolution approaches, the candidate programs are kept in a small pool ranked by training accuracy, and the next edit is performed on top-scorers from this pool. We run this approach on three VLMs and four popular benchmarks on scientific charts, and we measure the fraction of descriptive questions whose answer exactly matches the ground truth. The accuracy improves in all experimental settings. On average, it rises from 16.6% with the starting program to 41.3% with the evolved one, and the largest single increase is 55.2 percentage points. As a side benefit, our evolved program is 3.8x faster than the starting program, because short answers are easier to grade and faster to generate. Our results demonstrate that a large part of the observed errors arise due to the inference pipeline, and that optimizing it can improve the "frozen" VLM without the need to retrain it.