Evaluating Prompting Strategies for Chart Question Answering with Large Language Models

Ruthuparna Naikar; Ying Zhu

cs.CLMar 3, 2026

Evaluating Prompting Strategies for Chart Question Answering with Large Language Models

Authors: Ruthuparna Naikar, Ying Zhu

Organizations: Georgia State University, Atlanta GA, USA

Abstract

Prompting strategies affect LLM reasoning performance, but their role in chart-based QA remains underexplored. We present a systematic evaluation of four widely used prompting paradigms (Zero-Shot, Few-Shot, Zero-Shot Chain-of-Thought, and Few-Shot Chain-of-Thought) across GPT-3.5, GPT-4, and GPT-4o on the ChartQA dataset. Our framework operates exclusively on structured chart data, isolating prompt structure as the only experimental variable, and evaluates performance using two metrics: Accuracy and Exact Match. Results from 1,200 diverse ChartQA samples show that Few-Shot Chain-of-Thought prompting consistently yields the highest accuracy (up to 78.2%), particularly on reasoning-intensive questions, while Few-Shot prompting improves format adherence. Zero-Shot performs well only with high-capacity models on simpler tasks. These findings provide actionable guidance for selecting prompting strategies in structured data reasoning tasks, with implications for both efficiency and accuracy in real-world applications.

Figures & tables

Figure 1: This diagram illustrates the workflow of chart-based question answering with LLMs. Structured chart data and natural language questions are transformed into prompts, which are submitted to an instruction-tuned LLM to generate answers.

Panel of Figure 2 — Figure 2: Sample data visualization and its serialized JSON representation.

Figure 3: Generalized prompt template with components.

Component	Content
Instruction	Use the provided table to answer the question. Respond only with the final answer after “ Answer: ”. Round numbers to two decimals.
Exemplar 1	Table: Year–GDP (trillion USD): 2018–2.1, 2019–2.3. Q: What is the GDP growth from 2018 to 2019? Answer: 0.2
Exemplar 2	Table: Country–Literacy Rate (%): India–74, China–96. Q: Which country has a higher literacy rate? Answer: China
Exemplar 3	Table: Year–Death Rate (Under-5): 2000–100, 2010–60. Q: What is the difference in death rate between 2000 and 2010? Answer: 40
Target Question (Data + Question)	Table: Age Group–Death Rate: Under-5–170.2, 15–49–95.7, 70+–610.5. Q: Which age group has the lowest death rate? Model Output (GPT-4o): 15–49 Expected Answer: 15–49

Table 1: Example of Few-Shot prompting with structured chart data and model output.

Component	Content
Instruction	Use the provided table to answer the question. Respond only with the final answer after “ Answer: ”. Round numbers to two decimals.
Target Question (Data + Question)	Table: Age Group–Death Rate: Under-5–170.2, 15–49–95.7, 70+–610.5. Q: Which age group has the lowest death rate? Model Output (GPT-4o): 15–49 Expected Answer: 15–49

Table 2: Example of Zero-Shot prompting with structured chart data and model output.

Prompting Strategy	GPT-3.5	GPT-4	GPT-4o	Average
ZSP	68.4	71.2	69.8	69.8
FSP	72.1	74.5	70.3	72.3
ZS-CoT	70.9	73.6	72.1	72.2
FS-CoT	75.8	76.9	78.2	77.0
Average	71.8	74.1	72.6

Table 3: Accuracy (%) for each prompting strategy and model (N=1,200 evaluation items)

Prompting Strategy	GPT-3.5	GPT-4	GPT-4o	Average
ZSP	60.9	62.1	59.1	60.7
FSP	65.2	60.4	68.5	64.7
ZS-CoT	62.3	61.2	60.7	61.4
FS-CoT	58.0	57.2	58.4	57.9
Average	61.6	60.2	61.7

Table 4: Exact Match (%) for each prompting strategy and model (N=1,200 evaluation items)

Accuracy
Prompting	Arithmetic	Comparative	Boolean	Direct Retrieval
ZSP	62.3	70.8	72.4	75.6
FSP	66.7	73.5	73.9	77.2
ZS-CoT	64.1	72.6	74.2	76.8
FS-CoT	74.5	77.9	76.3	81.1
Exact Match

Table 5: Accuracy and Exact Match (%) by reasoning category (N = 300 per category, macro-averaged across models).

LLM Prompting

Large language model prompting studies how instructions, examples, context, and formats steer model behavior at inference time.

See all 429 papers

CharTool: Tool-Integrated Visual Reasoning for Chart Understanding
Apr 3, 2026Situo Zhang, Yifan Zhang, Zichen Zhu +6Chart Understanding Chart QA
X-LANCE Lab, Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China · Shanghai Innovation Institution, Shanghai, China +1
Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models
May 26, 2026Yifan Jiang, Dae Yon Hwang, Jesse C. Cresswell +1VLM Evaluation Data Visualization
University of Waterloo · Vector Institute · Layer 6 AI
CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning
Aug 3, 2026Xuehang Guo, Pingyue Zhang, Ruiyi Zhang +6Data Visualization Chart QA
1William & Mary · 2Northwestern University · 3Adobe +1

Apr 3, 2026J/K move · Enter open · S save

CharTool: Tool-Integrated Visual Reasoning for Chart Understanding

Situo Zhang, Yifan Zhang, Zichen Zhu, Da Ma+5

X-LANCE Lab, Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China · Shanghai Innovation Institution, Shanghai, China+1

Chart Understanding Chart QA Tool Use in VLMs Numerical Reasoning in Language Models

Open paper AskPDF arXiv

Charts are ubiquitous in scientific and financial literature for presenting structured data. However, chart reasoning remains challenging for multimodal large language models (MLLMs) due to the lack of high-quality training data, as well as the need for fine-grained visual grounding and precise numerical computation. To address these challenges, we first propose DuoChart, a scalable dual-source data pipeline that combines synthesized charts with real-world charts to construct diverse, high-quality chart training data. We then introduce CharTool, which equips MLLMs with external tools, including image cropping for localized visual perception and code-based computation for accurate numerical reasoning. Through agentic reinforcement learning on DuoChart, CharTool learns tool-integrated reasoning grounded in chart content. Extensive experiments on six chart benchmarks show that our method consistently improves over strong MLLM baselines across model scales. Notably, CharTool-7B outperforms the base model by +8.0% on CharXiv (Reasoning) and +9.78% on ChartQAPro, while achieving competitive performance with substantially larger or proprietary models. Moreover, CharTool demonstrates positive generalization to out-of-domain visual math reasoning benchmarks.

Abstract

LLM Prompting

Explore similar work

CharTool: Tool-Integrated Visual Reasoning for Chart Understanding

Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models

CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning