Misleading charts can lead vision-language models to conclusions that conflict with underlying data. Existing work focuses on inference-time strategies, while training-time supervision remains less explored. We present VizDAGuide, a DAG-structured supervision method for misleading-chart question answering. The method represents intermediate reasoning states and their dependencies as a directed acyclic graph, assigns supervision weights according to their roles, and serializes the graph into a unified training target for open-ended and multiple-choice QA with single-pass inference. Experiments on two benchmarks across Qwen3-VL and InternVL3.5 models show improvements over a content- and weight-matched Linear CoT-SFT baseline on both answer formats. Process-level evaluation shows gains in grounding and operation execution, while ablations indicate that dependency structure and weighted supervision are most effective when combined. These results support DAG-structured weighted supervision for misleading-chart question answering.
Figures & tables
Figure 1: An example multiple-choice question (MCQ) illustrating misleading-chart reasoning. The reversed axis makes the chart visually suggest an increasing trend despite decreasing underlying values.
Figure 2: Overview of VizDAGuide.
Open-ended QA
MCQ
Setting
FAA ↑
RA ↑
Acc . ↑
MCR ↓
Qwen3-VL-4B
Base
34.98
45.03
62.73
17.61
+ Answer-only-SFT
36.02
47.07
72.05
4.77
+ Linear CoT-SFT
56.25
68.28
71.93
15.45
+ U-DAG
55.31
67.89
70.68
15.91
Table 1: Comparison of supervision variants across VLM backbones.
Open-ended QA
Supervision
Gnd. ↑
Op. ↑
Corr. ↑
Pac. ↑
FC ↓
Qwen3-VL-4B
Linear CoT-SFT
57.49
68.21
88.64
48.99
11.10
U-DAG
57.57
67.99
86.66
49.72
2.67
VizDAGuide (Ours)
59.44
70.07
84.90
50.53
1.81
Qwen3-VL-8B
Table 2: Process-level results on open-ended and multiple-choice QA. Gnd., Op., Corr., Pac., FC, and Suff. denote Grounding, Operation, Correction, Process Accuracy, Clean FC, and Sufficiency.
Corr.
Answer Error
Task
Process
4B
8B
4B
8B
OEQ
Grounding
0.447
0.439
9.94
8.68
Operation
0.757
0.756
100.00
100.00
Correction
0.105
0.085
18.44
18.88
MCQ
Grounding
0.294
0.212
57.14
48.57
Operation
0.174
0.129
20.66
22.13
Table 3: Process–answer relation. Corr. measures process–answer correctness correlation; Answer Error is the final-answer error rate when only the corresponding process is incorrect.
Task
Misleader Type
Process
4B
8B
OEQ
Stacked misuse
Operation
0.963
0.960
Axis-range misuse
Operation
0.937
0.946
Dual-axis misuse
Operation
0.930
0.910
3D effect
Operation
0.919
0.909
Log-scale misuse
Operation
0.898
0.904
MCQ
Small size
Sufficiency
1.000
1.000
Table 4: Top-5 misleader types ranked by average process–answer correlation across 4B and 8B. Process denotes the highest-correlated process.
Figure 3: Fine-grained comparison between the Base model and VizDAGuide across question types, chart types, and misleading conditions.
Figure 4: Distribution of the earliest identified error stage in VizDAGuide predictions on open-ended and multiple-choice QA.
(a) Open-ended QA
Model
RA ↑
Gnd. ↑
Op. ↑
Corr. ↑
Pac. ↑
FC ↓
GPT-5.5
64.42
59.34
55.36
63.16
36.54
4.62
Gemini 3 Flash
64.28
59.45
55.86
61.48
37.12
3.76
Kimi-K3
63.83
57.78
55.18
55.98
34.57
1.57
Claude Opus 4.8
63.11
48.47
31.71
52.58
15.50
1.62
Gemini 2.5 Pro
61.63
50.71
51.69
58.18
29.11
6.14
Table 5: Reference comparison with general-purpose models on open-ended and multiple-choice misleading chart QA. External models use a unified JSON evaluation protocol, whereas our models use their native structured-output setting; the results are intended for contextual comparison rather than strict ranking.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
α/β/γ
0.25/0.35/0.40
τ
0.8
wmin/wmax
0.5/2.0
Appendix
Table 6: Hyperparameters used for DAG-guided weight assignment.
Setting
Open-ended
MCQ
LoRA rank / scaling
32 / 64
32 / 64
LoRA dropout
0.05
0.05
Learning rate
5×10−5
3×10−5
Schedule
Constant
Cosine
Warmup ratio
0
0.03
Weight decay
0
0.01
Appendix
Table 7: Training settings for open-ended and multiple-choice QA. The same task-specific configuration is used across model backbones.
Model
Comparator
Metric
Δ [95% CI]
p
Open-ended QA
Qwen3-VL-4B
Linear CoT
FAA
+1.29 [0.45, 2.12]
0.0020
Qwen3-VL-4B
Linear CoT
RA
+1.69 [0.82, 2.55]
0.0001
Qwen3-VL-8B
Linear CoT
FAA
+2.43 [1.63, 3.23]
0.0001
Qwen3-VL-8B
Linear CoT
RA
+2.42 [1.57, 3.25]
0.0001
InternVL3.5-4B
Linear CoT
FAA
+1.14 [0.33, 1.92]
0.0070
Appendix
Table 8: Paired significance analysis for the main comparisons in Table 1 . Differences are reported as VizDAGuide minus the corresponding comparator in percentage points. Confidence intervals use paired chart-group bootstrap, and p -values use two-sided chart-group permutation tests. The reported p -values are not adjusted for multiple comparisons. A value of 0.0001 is the minimum attainable permutation estimate rounded to four decimal places.
Figure 5: Qualitative comparison on a chart with a nonlinear scale. Both methods identify the misleading scale, but VizDAGuide correctly computes the vote ratio (25%), whereas Linear CoT returns an incorrect answer (200%).
Figure 6: Qualitative comparison on a pie chart with misleading slice proportions. Both methods identify the labeled market shares, but VizDAGuide returns the correct ratio (6:1), whereas Linear CoT selects an incorrect answer (4:1).
Figure 7: Qualitative comparison on a bar chart with a truncated axis. Both methods identify the truncated scale and the relevant values, but VizDAGuide returns the correct ratio (50%), whereas Linear CoT selects an incorrect answer (25%).
Figure 8: Qualitative comparison on a truncated-axis chart. VizDAGuide computes the correct ticket-sales ratio (50%) from the numerical values, whereas Linear CoT selects an incorrect answer (20%).
Figure 9: Qualitative comparison on a chart with redundant visual encoding. VizDAGuide identifies Brand B as having the largest market share (40%), whereas Linear CoT incorrectly selects Others.
Figure 10: Qualitative comparison on a chart with a misleading title. Despite the title suggesting a population decline, the plotted values increase. VizDAGuide identifies the correct trend, whereas Linear CoT predicts a decrease.
Figure 11: Qualitative comparison on a revenue chart with a misleading title. VizDAGuide identifies the increasing trend from the plotted values, whereas Linear CoT incorrectly concludes that the available information is insufficient.
Figure 12: Qualitative comparison of overall trend interpretation. VizDAGuide identifies a generally increasing trend despite a decrease in the final interval, whereas Linear CoT selects the insufficient-information option.
Figure 13: Open-ended QA comparison on a chart with an inverted axis. VizDAGuide identifies the correct minimum value (4,000,000), whereas Linear CoT returns 38,000,000.
Figure 14: Open-ended QA comparison on a 3D bar chart. VizDAGuide identifies the correct minimum CPIA rating (3.5), whereas Linear CoT returns 3 after incorrectly reading the chart values.
Figure 15: Open-ended QA comparison on a logarithmic-axis chart. VizDAGuide grounds the relevant values and computes the correct difference (1), whereas Linear CoT returns 2.
Figure 16: Open-ended QA comparison on an inverted-axis line chart. VizDAGuide correctly identifies the increasing numerical trend, whereas Linear CoT predicts the opposite trend.