REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models

Yuanbang Liu; Chenxi Ruan; Yihan Hou; Qiong Luo; Wei Zeng

cs.CVAug 18, 2026

REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models

Authors: Yuanbang Liu, Chenxi Ruan, Yihan Hou, Qiong Luo, Wei Zeng

Organizations: HKUST(GZ)

Abstract

Chart editing requires inferring and modifying visualization code from a reference chart image based on an editing instruction, challenging fine-grained visual reasoning, instruction following, and executable code synthesis capabilities of MLLMs. Large reasoning models (LRMs) with extended Chain-of-Thought (CoT) reasoning are suitable for tackling such complex multimodal tasks. However, our preliminary study reveals an inverted-U'' relationship between reasoning length and chart-editing performance: Excessive reasoning often leads to overthinking,'' where models drift toward hallucinated visual details or get stuck in redundant reasoning loops. To address the gap, we introduce REChart, a two-stage training framework that provides process-level supervision over intermediate reasoning steps, improving both editing fidelity and reasoning efficiency. First, we synthesize 200k high-quality reasoning trajectories for supervised fine-tuning from a large image-instruction-code pool, using a role-specialized agentic Reason-Score-Refine workflow that iteratively refine the chart code toward higher quality. Second, we optimize the model via reinforcement learning with two complementary rewards: a fidelity reward evaluating code correctness, visual fidelity, and structural consistency, and an efficiency reward that assigns each rollout a random thinking budget, truncates the reasoning process, and credits the final reasoning segment according to its contribution to the output. On the ChartEdit and ChartMIMIC benchmarks, our model achieves state-of-the-art chart-editing performance among open-source models of comparable scale, while mitigating overthinking and reducing average reasoning token usage by 79.0% under a maximum thinking budget of 16,384 tokens compared with the base model.

Figures & tables

Category	Key Editing Aspects
Direct Reproduction	No-edit baseline: reconstruction of the input chart from image to code.
Content Manipulation	Data filtering, aggregation, point/series addition or removal, and trend-line overlays.
Style & Aesthetics	Modifications to backgrounds, grids, markers, colormaps, and visual effects.
Layout Edits	Subplot composition, legend and colorbar positioning, spacing, and figure sizing.
Chart Transformation	Chart type swapping, orientation changes, and coordinate system transformations.
Axes & Annotations	Title and label content and typography, axis scaling, ticks, and data annotations.

Table 1: Taxonomy of chart-editing tasks across six functional categories.

Panel of Figure 1 — Figure 1: Reasoning scaling on ChartEdit. (a) Performance follows an inverted-U trend with reasoning length. (b) Error patterns shift toward hallucination and redundancy as the budget grows.

Figure 2: The data collection and annotation pipeline consists of three stages: (1) Chart Collection and Filtering; (2) Chart-Code Pair Generation; and (3) Chart Editing Augmentation.

Figure 3: Multi-agent pipeline for generating the reasoning path.

Figure 4: Overview of our RL framework guided by an episode-level efficiency reward and a multi-dimensional fidelity reward.

Proprietary Models
Model	# Params.	ChartEdit w/o Code				ChartMIMIC Customized
Model	# Params.	Exec.	Code	Chart	Overall	Exec.	Low	High	Overall
GPT-4o ( Hurst et al., 2024 )	-	91.5	60.0	79.9	70.0	96.5	82.1	84.3	83.2
Claude-3.7-Sonnet ( Anthropic, 2025 )	-	96.3	73.4	87.5	80.5	80.6	67.4	78.1	72.3
Gemini-2.5-Pro ( Comanici et al., 2025 )	-	95.3	82.9	89.2	86.1	91.7	81.3	83.5	82.4
Open-Source Models

Table 2: Performance on ChartEdit w/o Code and ChartMIMIC (customized mimic) benchmarks.

Model	Exec.	Low	High	Overall
GPT-4o ( Hurst et al., 2024 )	93.2	79.0	83.5	81.2
GeminiProVision ( Team et al., 2023 )	68.2	53.8	53.3	53.6
Claude-3-opus ( Anthropic, 2024 )	83.3	60.5	60.1	60.3
InternVL2-8B ( Chen et al., 2024 )	61.8	34.4	38.9	36.6
LLaVA-Mistral-7B ( Liu et al., 2023c )	59.7	20.7	21.3	21.0
Qwen3-VL-32B-Thinking ( Bai et al., 2025a )	82.8	65.2	72.2	68.7

Table 3: Results on ChartMimic Direct Mimic task.

Variant	ChartEdit		ChartMIMIC Customized		ChartMIMIC Direct
Variant	Exec.	Overall	Exec.	Overall	Exec.	Overall
Base	79.15	67.30	77.61	61.50	75.00	58.40
SFT	87.76	73.40	80.78	63.90	84.06	67.80
+ Base GRPO	95.66	79.60	89.61	70.66	94.94	77.59
+ Budgeted GRPO	95.87	80.24	86.67	68.76	96.00	78.96
+ Efficiency Reward	96.65	79.71	88.61	70.09	94.72	77.15

Table 4: Ablation studies on training and reward design. We start from the base model, add SFT, then apply base GRPO, budgeted rollout training, and the efficiency reward.

Variant	ChartEdit	ChartMIMIC Direct	ChartMIMIC Customized
Base	6521.12	9575.09	8889.43
+ SFT	2790.66	2267.25	2251.27
+ Base GRPO	3184.46	2320.49	2285.46
+ Budgeted GRPO	3040.28	1997.75	1995.39
+ Efficiency Reward	2005.18	1727.69	1515.22

Table 5: Average reasoning token usage under a maximum thinking budget of 16,384 tokens, starting from the base model and cumulatively adding SFT and RL variants.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Prompt for terminating the reasoning phase within the given thinking budget and generating the final output.

Panel of Figure 6 — Figure 6: Additional analysis of reasoning budget sensitivity. (a) The optimal thinking budget differs across model scales and capabilities, suggesting that stronger models can often compress useful reasoning into shorter chains. (b) For Qwen3-VL-8B, the preferred budget also varies with task difficulty: harder editing tasks benefit from longer reasoning before overthinking begins.

Variant	Visual Hallucination	Edit Intent Drift	Details Missing
Base	125	16	73
+ SFT	105	13	66
+ RL	84	9	69

Appendix

Table 6: Reasoning error counts on the ChartEdit human set before and after training.

Chart Type	Ratio (%)	Chart Type	Ratio (%)
line	39.74	area	4.70
scatters	14.97	graph	3.52
bar	8.70	contour	3.19
heatmap	8.50	hist	3.15
errorbar	6.99	density	2.12
box	1.43	quiver	0.77

Appendix

Table 7: Distribution of chart types in the Chart Edit dataset (by percentage).

Model	Number of Charts	Proportion
Qwen3-VL-32B-Instruct	118,577	22.74%
Qwen3-VL-253B-A22B	187,907	36.04%
Gemini 3 Flash	214,854	41.21%
Total	521,338	100%

Appendix

Table 8: Statistics of visualization code generation models and their respective contribution to the final dataset. The table shows the number of chart-code pairs generated by each model and their proportion in the total dataset (N=521,338).

Task Type	Ratio (%)	Task	Ratio (%)
Axes & Annotations	27.11	Layout & Spatial Organization	10.02
Content Manipulation & Data Logic	20.68	Style & Aesthetics	12.68
Chart Transformation	11.61	direct	17.91

Appendix

Table 9: Distribution of edit types in the Chart Edit dataset (by percentage).

Stage	Count
arXiv papers (2024)	244,029
Papers under CC BY 4.0 license	100,180
Figures extracted from L a T e X sources	854,484
Chart images after classification	622,672
Chart-code pairs after cascade filtering	521,338
Editing samples accepted by audit	567,049

Appendix

Table 10: Pipeline statistics for image-instruction-code triplet construction. Each row reports the count of items remaining after the corresponding stage.

Panel of Figure 7 — Figure 7: Prompt for code reproduction based on the given image from the arXiv papers.

Panel of Figure 8 — Figure 8: Prompt for evaluating chart-code pairs during dataset construction. The prompt instructs the MLLM (Multimodal Large Language Model) assessor to check quality across four key dimensions: readability, completeness, visual integrity, and consistency. Each dimension is scored 1-10, with a binary pass/fail status. This quality filtering ensures the dataset contains only high-fidelity image-code pairs.

Panel of Figure 9 — Figure 9: Prompt for generating chart editing instructions, which defines the role, inputs, task, domain-specific categories, and structured output format. This prompt is used to guide a multimodal LLM in producing high-quality instruction-code pairs for fine-tuning.

Panel of Figure 10 — Figure 10: System prompt for the quality audit of a visual editing process. The prompt defines the auditor’s role, required inputs, evaluation criteria, with a detailed 1-25 scoring scale, and a strict JSON output format. The auditor must assess faithfulness, integrity, linguistic quality, and visual legibility of the edit.

Agent	Symbol	Backbone
Planner	$\mathcal{P}$	Qwen3-VL-32B-Instruct
Reasoner	$\mathcal{T}$	Qwen3-VL-8B-Thinking
Generator	$\mathcal{G}$	Qwen3-VL-8B-Thinking
Critic	$\mathcal{C}$	Qwen3-VL-32B-Instruct
Finalizer	$\mathcal{F}$	Qwen3-VL-32B-Instruct

Appendix

Table 11: Backbone model assignment for each agent in the trajectory synthesis pipeline.

Planner	Critic	Pass@3	Pass@6	Pass@9	Avg. Token
✓	✓	24.75%	35.17%	42.58%	2086.33
	✓	24.00%	35.75%	42.58%	2416.21
✓		16.58%	22.00%	24.75%	2200.22
		15.33%	19.75%	23.42%	2300.86

Appendix

Table 12: Ablation of the Reason-Score-Refine synthesis pipeline. We evaluate whether the planner and critic are both needed for effective trajectory construction.

Module	Hardware	Throughput
Planner	1 $\times$ A800	3500/h
Reasoner	1 $\times$ H100	4000/h
Generator	1 $\times$ H100	4000/h
Scorer	64 Core GPU + 512GB Memory	9000/h
Critic	1 $\times$ A800	18000/h
Finalizer	1 $\times$ A800	9000/h

Appendix

Table 13: Estimated throughput and compute footprint of the SFT synthesis pipeline.

Task Type	Total Input	Filtered	Final Output	Pass Rate	Avg. Round Num
Edit Task	562,485	383,867	178,618	31.76%	5.39
Direct Task	75,275	45,957	29,318	38.95%	5.42
Total	637,760	429,824	207,936	32.60%	5.40

Appendix

Table 14: Statistics of the SFT data synthesis pipeline after filtering.

SFT Dataset Size	ChartEdit w/o Code		ChartMIMIC Direct
	Exec. Rate	Overall	Exec. Rate	Overall
base	79.15	67.30	75.00	58.45
50k	83.56	68.76	79.06	61.75
100k	84.48	69.12	80.72	63.58
150k	86.98	71.01	82.11	65.05
full	87.76	73.43	84.06	67.78

Appendix

Table 15: Effect of SFT dataset size on downstream performance.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 356 papers

Apr 3, 2026

CharTool: Tool-Integrated Visual Reasoning for Chart Understanding

Charts are ubiquitous in scientific and financial literature for presenting structured data. However, chart reasoning remains challenging for multimodal large language models (MLLMs) due to the lack of high-quality training data, as well as the need for fine-grained visual grounding and precise numerical computation. To address these challenges, we first propose DuoChart, a scalable dual-source data pipeline that combines synthesized charts with real-world charts to construct diverse, high-quality chart training data. We then introduce CharTool, which equips MLLMs with external tools, including image cropping for localized visual perception and code-based computation for accurate numerical reasoning. Through agentic reinforcement learning on DuoChart, CharTool learns tool-integrated reasoning grounded in chart content. Extensive experiments on six chart benchmarks show that our method consistently improves over strong MLLM baselines across model scales. Notably, CharTool-7B outperforms the base model by +8.0% on CharXiv (Reasoning) and +9.78% on ChartQAPro, while achieving competitive performance with substantially larger or proprietary models. Moreover, CharTool demonstrates positive generalization to out-of-domain visual math reasoning benchmarks.

Situo Zhang, Yifan Zhang, Zichen Zhu +6

X-LANCE Lab, Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China · Shanghai Innovation Institution, Shanghai, China +1

Chart Understanding Chart QA Tool Use in VLMs Numerical Reasoning in Language Models

Ask PDF

May 22, 2026

ETCHR: Editing To Clarify and Harness Reasoning

Multimodal Large Language Models have advanced visual reasoning, yet a purely textual chain of thought remains a bottleneck for questions that require fine-grained focus or view transformations. The ''think with images'' paradigm narrows this gap, but existing approaches are either constrained by fixed predefined toolkits or produce noisy intermediate images from unified multimodal methods. We pursue a third option: using a dedicated image editing model and decouple it with an understanding model. However, off-the-shelf image editors fail as reasoning assistants with two complementary gaps: a language-side gap, where editors trained as passive instruction-followers cannot map an abstract question to an appropriate visual transformation, and a generation-side gap, where edit correctness degrades as reasoning depth grows. Guided by this analysis, we introduce ETCHR (Editing To Clarify and Harness Reasoning), a question-conditioned, reasoning-aware image editor decoupled from the downstream understanding model and trained with a two-stage recipe targeted at the two gaps: Reasoning Imitation via supervised fine-tuning on edit trajectories, followed by Reasoning Enhancement with VLM-derived rewards for edit correctness and downstream reasoning accuracy. Since the editor is decoupled, ETCHR plugs into different open- and closed-source MLLMs in a training-free manner. Across five task families (fine-grained perception, chart understanding, logic reasoning, jigsaw restoration, and 3D understanding), ETCHR raises average Pass@1 from 55.95 to 60.77 (+4.82) with Qwen3-VL-8B, from 65.08 to 70.55 (+5.47) with Gemini-3.1-Flash-Lite, and from 76.55 to 81.16 (+4.61) with the 1T-parameter MoE model Kimi K2.5.

Beichen Zhang, Yuhong Liu, Jinsong Li +3

The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University +2

Tool Use in VLMs Multimodal Reasoning Multimodal CoT Reasoning Visually Grounded Reasoning Multimodal Foundation Models Image Editing+1

Ask PDF

Sep 21, 2026

ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation

Building strong chart-to-code systems increasingly relies on reinforcement learning, whose effectiveness depends critically on the quality of the reward signal. Large Multimodal Models (LMMs) play a natural critical role in jointly assessing chart visual appearance and task requirements. They are therefore increasingly used as visual critics and reward models, yet their reliability as judges remains largely unexplored. To this end, we introduce ChartJudgeBench, a diagnostic vision-language benchmark for assessing LMM judges in chart-to-code workflows. It includes 1,003 Chart Perception Alignment (CPA) instances for pairwise chart comparison and 650 Chart Reasoning Judgment (CRJ) instances for binary Accept/Reject verification in Chart Reproduction and Chart Editing. Together, these tasks emulate the core judging decisions required in agentic refinement and RL-based chart optimization. Our evaluation of strong LMMs reveals four systematic limitations: (i) positional bias in pairwise comparison, (ii) a strong tendency to overpredict Accept, (iii) difficulty in matching visual styles and aesthetics, and (iv) an unexpected leniency bias in RL-trained models. These findings show that current LMM judges require explicit reliability validation before being used as critics or reward models in chart-to-code optimization. The code and data are available on ChartJudgeBench.

Lijian Wu, Henry Hengyuan Zhao, Zijian Zhang +3

CSU-JPG, Central South University · National University of Singapore

VLM Evaluation LLM Evaluation LLM-as-a-Judge Chart-to-Code Generation Code Generation LLM Reliability+1

Ask PDF

Abstract

Efficient Language Model Reasoning

Explore similar work

CharTool: Tool-Integrated Visual Reasoning for Chart Understanding

ETCHR: Editing To Clarify and Harness Reasoning

ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation