Chart QA

QA: Question Answering

Chart question answering asks models to interpret plotted data and answer questions about values, trends, comparisons, and relationships, combining visual perception with reasoning. Systems use OCR and axis or legend grounding alongside multimodal language models; newer methods add chart structure representations, visual tools, executable code, and reinforcement learning with verifiable rewards. Evaluation covers synthetic and real-world charts, including scientific and financial figures, multi-chart and interactive tasks, and tests of evidence localization and precise value extraction. Despite gains from prompting and specialized training, models still struggle with dense or misleading charts, unfamiliar formats, complex multi-step reasoning, exact numerical recovery, and grounding answers in visible evidence.

Latest papers

33 papers5 papers in the last four weeks, up 25% on the four weeks before. 0.0% of all new papers.

Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewards
Mar 7, 2026Xin Zhang, Xingyu Li, Rongguang Wang +4Chart QA RL for VLMs
Oracle AI
Evaluating Prompting Strategies for Chart Question Answering with Large Language Models
Mar 3, 2026Ruthuparna Naikar, Ying ZhuLLM Prompting Chart QA
Georgia State University, Atlanta GA, USA
ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation
Jan 19, 2026Jesus-German Ortiz-Barajas, Jonathan Tonglet, Vivek Gupta +1Data Visualization Adversarial Attacks on VLMs
INSAIT · KU Leuven · Arizona State University +2

Mar 7, 2026J/K move · Enter open · S save

Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewards

Xin Zhang, Xingyu Li, Rongguang Wang, Ruizhong Miao+3

Oracle AI

Chart QA RL for VLMs Reinforcement Learning with Verifiable Rewards VLM Robustness

Open paper AskPDF arXiv

Accurate chart comprehension represents a critical challenge in advancing multimodal learning systems, as extensive information is compressed into structured visual representations. However, existing vision-language models (VLMs) frequently struggle to generalize on unseen charts because it requires abstract, symbolic, and quantitative reasoning over structured visual representations. In this work, we introduce Chart-RL, an effective reinforcement learning (RL) method that employs mathematically verifiable rewards to enhance chart question answering in VLMs. Our experiments demonstrate that Chart-RL consistently outperforms supervised fine-tuning (SFT) across different chart understanding benchmarks, achieving relative improvements of 16.7% on MutlChartQA, and 11.5% on ChartInsights. We conduct robustness analysis, where Chart-RL achieves enhanced performance in 18 of 25 perturbed chart categories, demonstrating strong consistency and reasoning capability across visual variations. Furthermore, we demonstrate that task difficulty and inherent complexity are more critical than data quantity in RL training. For instance, Chart-RL trained on merely 10 complex chart-query examples significantly outperforms models trained on over 6,000 simple examples. Additionally, training on challenging reasoning tasks not only improves in-domain generalization relative to simpler tasks, but also facilitate strong transfer to out-of-domain visual mathematical problems.

Latest papers

Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewards

Evaluating Prompting Strategies for Chart Question Answering with Large Language Models

ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation