cs.CVMar 3, 2026

Chart Deep Research in LVLMs via Parallel Relative Policy Optimization

Authors: Jiajin Tang, Gaoyang, Wenjie Wang, Sibei Yang, Xing Chen

Organizations: ByteDance · ShanghaiTech University · School of Computer Science and Engineering, Sun Yat-sen University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

With the rapid advancement of data science, charts have evolved from simple numerical presentation tools to essential instruments for insight discovery and decision-making support. However, current chart data intelligence exhibits significant limitations in deep research capabilities, with existing methods predominantly addressing shallow tasks such as visual recognition or factual question-answering, rather than the complex reasoning and high-level data analysis that deep research requires. This limitation stems from two primary technical bottlenecks: at the training level, existing post-training techniques exhibit deficiencies in handling multi-dimensional reward signal interference and heterogeneous data gradient conflicts, preventing models from achieving balanced development across multiple capability dimensions; at the evaluation level, current methods remain limited to factual retrieval and basic computation, failing to assess end-to-end analytic reasoning and other deep research capabilities. To address the training challenge, we propose PRPO, which performs parallel optimization across reward dimensions and capability partitioning across data types, effectively disentangling conflicts between heterogeneous data and multi-dimensional reward signals while ensuring optimization stability. For the evaluation challenge, we construct MCDR-Bench based on the ``error uniqueness principle," transforming subjective generation assessment into objective error identification through controllable error injection, enabling quantifiable evaluation of deep research capabilities. Experimental validation confirms that the proposed PRPO and MCDR-Bench jointly establish a unified framework that systematically advances chart deep research through enhanced collaborative training and objective evaluation.

Figures & tables

Figure 1: Optimization trajectories of GRPO vs. PRPO. Left: under multi-dimensional rewards, GRPO suffers from signal interference and limited exploration. Right: PRPO decomposes optimization across reward dimensions and data types, reducing interference and enabling specialized learning, which yields more effective exploration and near-optimal performance.
Figure 2: Multi-agent annotation process for multimodal chart deep research. The process consists of five stages: (1) Background Acquisition : retrieving domain-specific knowledge; (2) Fact Extraction : extracting atomic data elements; (3) Relationship Construction : modeling topological and logical connections; (4) Deep Research Report Generation : synthesizing comprehensive reports; (5) Forecast/Plan : proposing strategic recommendations. Human filtering ensures quality control throughout the process. The five specialized expert agents are represented by .
Figure 3: Demonstration of GRPO and our PRPO. PRPO unifies Reward-PRPO and Data-PRPO, partitioning data into capability-based groups and decomposing rewards across dimensions. This approach addresses multi-dimensional reward conflicts and data optimization conflicts, enabling balanced training across complex tasks.
ModelBGFERLDRF/POverallMean
GPT-4o27.1721.8941.0147.4560.0035.8338.89
GPT-4.134.6819.5072.2577.1179.2250.8755.61
Claude-3.7 Sonnet68.7857.2789.4585.0286.9774.9677.08
Gemini-2.5-Pro81.2187.3191.4493.7892.9889.2989.34
ChartGemma-3B7.535.027.519.2514.298.148.29
Qwen2.5-VL-3B-Instruct1.880.813.0610.8312.584.965.69
Table 1: Performance comparison on MCDR-Bench across different splits. We evaluate both GRPO and our PRPO under two prompt settings: direct answer generation and chain-of-thought reasoning (Think). The Think setting encourages models to provide step-by-step reasoning before generating final answers, which is particularly important for complex chart analysis tasks.
ModelFactoidMCQConv.FactChkHypo.OverallMean
GPT-4o35.7646.7234.7545.4928.9137.6738.33
Claude-3.5 Sonnet38.8451.4044.5355.6045.4843.5846.57
Gemini-Flash-2.043.4360.2840.2567.6224.4746.8547.15
ChartGemma-3B6.860.016.001.226.536.846.12
TinyChart-3B8.527.0017.4633.1916.0613.2516.45
ChartInstruct-LLama2-7B7.090.03.770.06.914.883.55
Table 2: Performance comparison on ChartQAPRO across different splits.
Table 6
DataScore
Level134.89
Level239.79
Level341.02
Level450.32
Level532.30
Low Level32.17
Table 5: Data Ablation Results.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Comparison of multi-dimensional reward processing between GRPO and PRPO. (1) Original multi-dimensional rewards for the same sample. (2a) GRPO aggregates rewards into a single scalar value, resulting in low variance and weak signal strength, which diminishes the distinct optimization signals from each reward dimension. (3a) GRPO’s advantage calculation further exhibits weak discrimination due to the loss of variability in aggregated rewards. (2b) PRPO independently processes each reward dimension, preserving signal integrity. (3b) PRPO’s dimension-specific advantage calculation achieves higher discrimination, effectively capturing the unique contributions of each reward type and enabling more effective optimization.
Figure 5: Comparison of reward values (left) and response lengths (right) during training for Qwen2.5-VL-3B (top) and Qwen2.5-VL-7B (bottom) models under different reward strategies.
Figure 6: Accuracy reward trajectories during training for different optimization strategies. The orange curve represents GRPO, while the blue curve represents our PRPO. PRPO demonstrates faster growth in early training, stable improvement in the middle phase, and higher peak rewards in later stages, consistently outperforming GRPO. This indicates that PRPO’s parallel optimization across reward dimensions and data types effectively mitigates reward sparsity while enhancing exploration capability and optimization efficiency.
Figure 7: The pie chart illustrates the proportional distribution of error types across the five evaluation stages in the MCDR-Bench dataset, ensuring comprehensive coverage of diverse capabilities from background knowledge integration to strategic forecasting and planning.
Figure 8: Illustration of a background knowledge error case from the MCDR-Bench.
Figure 9: Illustration of a fact extraction error case from the MCDR-Bench.
Figure 10: Illustration of a relationship constriction error case from the MCDR-Bench.
Figure 11: Illustration of a deep report error case from the MCDR-Bench.
Figure 12: Illustration of a forecast and plan error case from the MCDR-Bench.
Figure 13: Qualitative Results.
More in this topic

Chart Understanding

Chart understanding enables AI systems to read visualizations, recover their data, answer questions, and reason about what charts show.

See all 59 papers

Explore similar work

CardsList
May 3, 2026

Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts

Multimodal large language models (MLLMs) have shown considerable potential in chart understanding and reasoning tasks. However, they still struggle with high information density (HID) charts characterized by multiple subplots, legends, and dense annotations due to three major challenges: (1) limited fine-grained perception results in the omission of critical visual cues; (2) redundant or noisy visual information undermines the performance of multimodal reasoning; (3) lack of adaptive deep reasoning relative to the amount of visual information. To tackle these challenges, we present a novel focus-driven fine-grained chart reasoning model, Chart-FR1, to improve perception, focusing efficiency, and adaptive deep reasoning on HID charts. Specifically, we propose Focus-CoT, a visual focusing chain-of-thought that enhances fine-grained perception by explicitly linking reasoning steps to key visual cues, such as local image regions and OCR signals. Building on this, we introduce Focus-GRPO, a focus-driven reinforcement learning algorithm with an information-efficiency reward that compresses redundant visual information for efficient focusing, and an adaptive KL penalty mechanism that enables flexible control over reasoning depth as more visual cues are discovered. Furthermore, to fill the gap in benchmarks for HID charts, we build HID-Chart, a challenging benchmark with an information-density metric designed to evaluate fine-grained chart reasoning capabilities. Extensive experiments on multiple chart benchmarks demonstrate that Chart-FR1 outperforms state-of-the-art MLLMs in chart understanding and reasoning. Code is available at https://github.com/phkhub/Chart-FR1.

Hongkun Pan, Yuwei Wu, Wanyi Hong +8

Zhejiang University · State Key Lab of CAD&CG, Zhejiang University · HiThink Research

Data VisualizationMultimodal CoT ReasoningChart UnderstandingFine-Grained Visual PerceptionRL for VLMs
AskPDF
Apr 3, 2026

CharTool: Tool-Integrated Visual Reasoning for Chart Understanding

Charts are ubiquitous in scientific and financial literature for presenting structured data. However, chart reasoning remains challenging for multimodal large language models (MLLMs) due to the lack of high-quality training data, as well as the need for fine-grained visual grounding and precise numerical computation. To address these challenges, we first propose DuoChart, a scalable dual-source data pipeline that combines synthesized charts with real-world charts to construct diverse, high-quality chart training data. We then introduce CharTool, which equips MLLMs with external tools, including image cropping for localized visual perception and code-based computation for accurate numerical reasoning. Through agentic reinforcement learning on DuoChart, CharTool learns tool-integrated reasoning grounded in chart content. Extensive experiments on six chart benchmarks show that our method consistently improves over strong MLLM baselines across model scales. Notably, CharTool-7B outperforms the base model by +8.0% on CharXiv (Reasoning) and +9.78% on ChartQAPro, while achieving competitive performance with substantially larger or proprietary models. Moreover, CharTool demonstrates positive generalization to out-of-domain visual math reasoning benchmarks.

Situo Zhang, Yifan Zhang, Zichen Zhu +6

X-LANCE Lab, Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China · Shanghai Innovation Institution, Shanghai, China +1

Chart UnderstandingChart QATool Use in VLMsNumerical Reasoning in Language Models
AskPDF
Aug 3, 2026

CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

Chart question answering (CQA) requires multimodal large language models (MLLMs) to integrate visual comprehension with logical reasoning, yet current models struggle with accurate visual grounding and coherent reasoning chains. While extrinsic chain-of-thought prompting and visual cues significantly improve performance, current MLLMs lack intrinsic visual grounded reasoning capabilities, leading to inaccurate perception and reasoning disconnected from visual evidence. To address these limitations, we propose CURV, a curriculum learning framework that develops intrinsic visual reasoning capabilities by reformulating CQA as multi-step visual grounded reasoning, where each step coordinates logical reasoning with dynamic visual grounding through spatial attention concentration. To assist model learning, we further introduce CCQA, a three-level curriculum dataset with scalable synthetic generation across diverse chart types and reasoning patterns. Our curriculum systematically progresses from basic single-operation reasoning to complex multi-chart compositional tasks. Experiments demonstrate that CURV achieves up to ↑20.50%\uparrow20.50\% improvements over baselines and is generalizable to real-world benchmarks (up to ↑12.30%\uparrow12.30\%) and out-of-domain multimodal reasoning tasks (up to ↑10.20%\uparrow10.20\%), validating the effectiveness of internalizing visual reasoning with dynamic grounding for enhanced chart understanding capabilities. Code is available at: https://xhguo7.github.io/CURV/.

Xuehang Guo, Pingyue Zhang, Ruiyi Zhang +6

1William & Mary · 2Northwestern University · 3Adobe +1

Data VisualizationChart QAMultimodal CoT ReasoningVisually Grounded ReasoningCurriculum Learning
AskPDF