cs.CVNov 2, 2026

VizDAGuide: Misleading Chart Question Answering in Compact Vision Language Models via DAG-Structured Supervision

Authors: Haiyang Zhang, Sicheng Song, Zixin Chen, Jie Gu, Huiya Ou, Chenhui Li, Changbo Wang

Organizations: East China Normal University · The Hong Kong University of Science and Technology

Abstract

Misleading charts can lead vision-language models to conclusions that conflict with underlying data. Existing work focuses on inference-time strategies, while training-time supervision remains less explored. We present VizDAGuide, a DAG-structured supervision method for misleading-chart question answering. The method represents intermediate reasoning states and their dependencies as a directed acyclic graph, assigns supervision weights according to their roles, and serializes the graph into a unified training target for open-ended and multiple-choice QA with single-pass inference. Experiments on two benchmarks across Qwen3-VL and InternVL3.5 models show improvements over a content- and weight-matched Linear CoT-SFT baseline on both answer formats. Process-level evaluation shows gains in grounding and operation execution, while ablations indicate that dependency structure and weighted supervision are most effective when combined. These results support DAG-structured weighted supervision for misleading-chart question answering.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models

    May 26, 2026Yifan Jiang, Dae Yon Hwang, Jesse C. Cresswell +1VLM EvaluationData Visualization

  2. ChartProbe: A Diagnostic Study on Visual Reasoning through Perception, Grounding, and Simple Reasoning

    Aug 13, 2026Mahsa Khoshnoodi, Sarah Adel BargalChart UnderstandingChart QA

  3. Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering

    Mar 30, 2026Yanjie Zhang, Yafei Li, Rui Sheng +5VLM RobustnessChart QA