Chart Deep Research in LVLMs via Parallel Relative Policy Optimization

Jiajin Tang; Gaoyang; Wenjie Wang; Sibei Yang; Xing Chen

cs.CVMar 3, 2026

Chart Deep Research in LVLMs via Parallel Relative Policy Optimization

Authors: Jiajin Tang, Gaoyang, Wenjie Wang, Sibei Yang, Xing Chen

Organizations: ByteDance · ShanghaiTech University · School of Computer Science and Engineering, Sun Yat-sen University

Abstract

With the rapid advancement of data science, charts have evolved from simple numerical presentation tools to essential instruments for insight discovery and decision-making support. However, current chart data intelligence exhibits significant limitations in deep research capabilities, with existing methods predominantly addressing shallow tasks such as visual recognition or factual question-answering, rather than the complex reasoning and high-level data analysis that deep research requires. This limitation stems from two primary technical bottlenecks: at the training level, existing post-training techniques exhibit deficiencies in handling multi-dimensional reward signal interference and heterogeneous data gradient conflicts, preventing models from achieving balanced development across multiple capability dimensions; at the evaluation level, current methods remain limited to factual retrieval and basic computation, failing to assess end-to-end analytic reasoning and other deep research capabilities. To address the training challenge, we propose PRPO, which performs parallel optimization across reward dimensions and capability partitioning across data types, effectively disentangling conflicts between heterogeneous data and multi-dimensional reward signals while ensuring optimization stability. For the evaluation challenge, we construct MCDR-Bench based on the ``error uniqueness principle," transforming subjective generation assessment into objective error identification through controllable error injection, enabling quantifiable evaluation of deep research capabilities. Experimental validation confirms that the proposed PRPO and MCDR-Bench jointly establish a unified framework that systematically advances chart deep research through enhanced collaborative training and objective evaluation.

Figures & tables

Figure 1: Optimization trajectories of GRPO vs. PRPO. Left: under multi-dimensional rewards, GRPO suffers from signal interference and limited exploration. Right: PRPO decomposes optimization across reward dimensions and data types, reducing interference and enabling specialized learning, which yields more effective exploration and near-optimal performance.

Figure 2: Multi-agent annotation process for multimodal chart deep research. The process consists of five stages: (1) Background Acquisition : retrieving domain-specific knowledge; (2) Fact Extraction : extracting atomic data elements; (3) Relationship Construction : modeling topological and logical connections; (4) Deep Research Report Generation : synthesizing comprehensive reports; (5) Forecast/Plan : proposing strategic recommendations. Human filtering ensures quality control throughout the process. The five specialized expert agents are represented by .

Figure 3: Demonstration of GRPO and our PRPO. PRPO unifies Reward-PRPO and Data-PRPO, partitioning data into capability-based groups and decomposing rewards across dimensions. This approach addresses multi-dimensional reward conflicts and data optimization conflicts, enabling balanced training across complex tasks.

Model	BG	FE	RL	DR	F/P	Overall	Mean
GPT-4o	27.17	21.89	41.01	47.45	60.00	35.83	38.89
GPT-4.1	34.68	19.50	72.25	77.11	79.22	50.87	55.61
Claude-3.7 Sonnet	68.78	57.27	89.45	85.02	86.97	74.96	77.08
Gemini-2.5-Pro	81.21	87.31	91.44	93.78	92.98	89.29	89.34
ChartGemma-3B	7.53	5.02	7.51	9.25	14.29	8.14	8.29
Qwen2.5-VL-3B-Instruct	1.88	0.81	3.06	10.83	12.58	4.96	5.69

Table 1: Performance comparison on MCDR-Bench across different splits. We evaluate both GRPO and our PRPO under two prompt settings: direct answer generation and chain-of-thought reasoning (Think). The Think setting encourages models to provide step-by-step reasoning before generating final answers, which is particularly important for complex chart analysis tasks.

Model	Factoid	MCQ	Conv.	FactChk	Hypo.	Overall	Mean
GPT-4o	35.76	46.72	34.75	45.49	28.91	37.67	38.33
Claude-3.5 Sonnet	38.84	51.40	44.53	55.60	45.48	43.58	46.57
Gemini-Flash-2.0	43.43	60.28	40.25	67.62	24.47	46.85	47.15
ChartGemma-3B	6.86	0.0	16.00	1.22	6.53	6.84	6.12
TinyChart-3B	8.52	7.00	17.46	33.19	16.06	13.25	16.45
ChartInstruct-LLama2-7B	7.09	0.0	3.77	0.0	6.91	4.88	3.55

Table 2: Performance comparison on ChartQAPRO across different splits.

Table 6

Data	Score
Level1	34.89
Level2	39.79
Level3	41.02
Level4	50.32
Level5	32.30
Low Level	32.17

Table 5: Data Ablation Results.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Comparison of multi-dimensional reward processing between GRPO and PRPO. (1) Original multi-dimensional rewards for the same sample. (2a) GRPO aggregates rewards into a single scalar value, resulting in low variance and weak signal strength, which diminishes the distinct optimization signals from each reward dimension. (3a) GRPO’s advantage calculation further exhibits weak discrimination due to the loss of variability in aggregated rewards. (2b) PRPO independently processes each reward dimension, preserving signal integrity. (3b) PRPO’s dimension-specific advantage calculation achieves higher discrimination, effectively capturing the unique contributions of each reward type and enabling more effective optimization.

Figure 5: Comparison of reward values (left) and response lengths (right) during training for Qwen2.5-VL-3B (top) and Qwen2.5-VL-7B (bottom) models under different reward strategies.

Figure 6: Accuracy reward trajectories during training for different optimization strategies. The orange curve represents GRPO, while the blue curve represents our PRPO. PRPO demonstrates faster growth in early training, stable improvement in the middle phase, and higher peak rewards in later stages, consistently outperforming GRPO. This indicates that PRPO’s parallel optimization across reward dimensions and data types effectively mitigates reward sparsity while enhancing exploration capability and optimization efficiency.

Figure 7: The pie chart illustrates the proportional distribution of error types across the five evaluation stages in the MCDR-Bench dataset, ensuring comprehensive coverage of diverse capabilities from background knowledge integration to strategic forecasting and planning.

Figure 8: Illustration of a background knowledge error case from the MCDR-Bench.

Figure 9: Illustration of a fact extraction error case from the MCDR-Bench.

Figure 10: Illustration of a relationship constriction error case from the MCDR-Bench.

Figure 11: Illustration of a deep report error case from the MCDR-Bench.

Figure 12: Illustration of a forecast and plan error case from the MCDR-Bench.

Chart Understanding

Chart understanding enables AI systems to read visualizations, recover their data, answer questions, and reason about what charts show.

See all 59 papers

Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts
May 3, 2026Hongkun Pan, Yuwei Wu, Wanyi Hong +8Data Visualization Multimodal CoT Reasoning
Zhejiang University · State Key Lab of CAD&CG, Zhejiang University · HiThink Research
CharTool: Tool-Integrated Visual Reasoning for Chart Understanding
Apr 3, 2026Situo Zhang, Yifan Zhang, Zichen Zhu +6Chart Understanding Chart QA
X-LANCE Lab, Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China · Shanghai Innovation Institution, Shanghai, China +1
CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning
Aug 3, 2026Xuehang Guo, Pingyue Zhang, Ruiyi Zhang +6Data Visualization Chart QA
1William & Mary · 2Northwestern University · 3Adobe +1

May 3, 2026J/K move · Enter open · S save

Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts

Hongkun Pan, Yuwei Wu, Wanyi Hong, Shenghui Hu+7

Zhejiang University · State Key Lab of CAD&CG, Zhejiang University · HiThink Research

Data Visualization Multimodal CoT Reasoning Chart Understanding Fine-Grained Visual Perception RL for VLMs

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have shown considerable potential in chart understanding and reasoning tasks. However, they still struggle with high information density (HID) charts characterized by multiple subplots, legends, and dense annotations due to three major challenges: (1) limited fine-grained perception results in the omission of critical visual cues; (2) redundant or noisy visual information undermines the performance of multimodal reasoning; (3) lack of adaptive deep reasoning relative to the amount of visual information. To tackle these challenges, we present a novel focus-driven fine-grained chart reasoning model, Chart-FR1, to improve perception, focusing efficiency, and adaptive deep reasoning on HID charts. Specifically, we propose Focus-CoT, a visual focusing chain-of-thought that enhances fine-grained perception by explicitly linking reasoning steps to key visual cues, such as local image regions and OCR signals. Building on this, we introduce Focus-GRPO, a focus-driven reinforcement learning algorithm with an information-efficiency reward that compresses redundant visual information for efficient focusing, and an adaptive KL penalty mechanism that enables flexible control over reasoning depth as more visual cues are discovered. Furthermore, to fill the gap in benchmarks for HID charts, we build HID-Chart, a challenging benchmark with an information-density metric designed to evaluate fine-grained chart reasoning capabilities. Extensive experiments on multiple chart benchmarks demonstrate that Chart-FR1 outperforms state-of-the-art MLLMs in chart understanding and reasoning. Code is available at https://github.com/phkhub/Chart-FR1.

Abstract

Chart Understanding

Explore similar work

Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts

CharTool: Tool-Integrated Visual Reasoning for Chart Understanding

CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning