MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction

Zitian Tang; Xu Zhang; Jianbo Yuan; Yang Zou; Varad Gunjal; Songyao Jiang; Davide Modolo

cs.AIApr 2, 2026

MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction

Authors: Zitian Tang, Xu Zhang, Jianbo Yuan, Yang Zou, Varad Gunjal, Songyao Jiang, Davide Modolo

Organizations: Brown University · Amazon AGI

Abstract

Multimodal Large Language Models (MLLMs) have recently demonstrated promising capabilities in multimodal coding tasks such as chart-to-code generation. However, existing methods primarily rely on supervised fine-tuning (SFT), which requires the model to learn code patterns through chart-code pairs but does not expose the model to a code execution environment. Moreover, while self-correction through execution feedback offers a potential route to improve coding quality, even state-of-the-art MLLMs have been shown to struggle with effective self-correction. In this work, we introduce MM-ReCoder, a chart-to-code generation model trained with reinforcement learning (RL) and equipped with self-correction ability. We propose a two-stage multi-turn self-correction RL strategy based on Group Relative Policy Optimization (GRPO). The first stage enhances the model's self-correction ability via rolling out a shared first turn, while the second stage improves the coding capability with full-trajectory optimization. MM-ReCoder learns to produce more accurate and executable code through the interaction with the environment and by iteratively correcting its own outputs. Our results on three chart-to-code benchmarks demonstrate the state-of-the-art performance of MM-ReCoder.

Figures & tables

Figure 1: In multi-turn Chart2Code, the model takes the execution result from its first-turn code and revises the code accordingly. Although existing MLLMs improve evaluation scores between the two turns, the gains mainly come from increased code executability. When we restrict the analysis to cases where the first-turn code is already executable, existing models show a negative improvement, whereas our model demonstrates a positive one.

Figure 2: Training pipeline of MM-ReCoder. We first conduct two stages of cold start: (a) we first train the model on ground truth chart-code pairs with SFT, then (b) we construct self-correction data with Qwen3VL-235B [ 31 ] , filter the successful ones, and train our model on the filtered data. After cold start, we conduct two stages of reinforcement learning: (c) we first enhance the model’s self-correction capability in the second turn via shared-first-turn optimization, then (d) we optimize the two turns jointly to improve the coding ability.

Figure 3: Result of a model trained solely with rule-based reward. The model receives a full rule-based reward though the texts are overlapped. But the model-based reward can penalize this chart.

Model	# Turns	ChartMimic			Plot2Code			ChartX
		Exec.Rate	Low-Level	High-Level	Pass Rate	Text-Match	Rating	GPT-score
		Exec.Rate	Low-Level	(GPT-4o)	Pass Rate	Text-Match	(GPT-4V/4o)	(GPT-4/4o)
Full score	-	100	100	100	100	100	10	5
Proprietary
GeminiProVision	1	68.2	53.8	53.3	68.2	53.6	3.69 / -	-
Claude-3-opus	1	83.3	60.5	60.1	84.1	57.5	3.80 / -	-

Table 1: Evaluation results on the Chart2Code task. MM-ReCoder achieves the best low-level score on ChartMimic and Text-match score on Plot2Code. On the other evaluation metrics, MM-ReCoder outperforms models of comparable size significantly.

Strategy	$\gamma$	$\eta$	First turn		Second turn		Avg. low-level improvement on rendered charts	Percentage of improved samples	Percentage of degraded samples	Percentage of repeated code
Strategy	$\gamma$	$\eta$	Exec.Rate	Low-Level	Exec.Rate	Low-Level	Avg. low-level improvement on rendered charts	Percentage of improved samples	Percentage of degraded samples	Percentage of repeated code
	0	0	91.8	81.8	95.2	83.9	0.21	3.4	2.6	46.9
Full-trajectory	0	0.1	64.5	66.7	96.0	84.3	10.11	87.5	2.3	0.3
	0.5	0.1	96.5	85.0	97.5	85.5	-0.03	2.2	2.2	55.6
Turn-wise	-	-	94.5	82.7	96.8	83.4	0.05	2.7	1.6	63.0
Shared-first-turn	-	-	91.7	79.8	95.8	82.6	0.72	14.4	8.4	16.8

Table 2: Comparison of RL strategies for self-correction. Our two-stage strategy enables self-correction capability while the others cannot.

Model	Low-level			High-level
Model	Average improve	Improved $\%$	Degraded $\%$	Average improve	Improved $\%$ @0.1	Degraded $\%$ @0.1
Qwen2.5-VL-7B	-0.36	8.1	13.9	-0.85	8.9	10.7
Qwen2.5-VL-72B	0.22	15.9	14.5	0.20	11.2	10.2
Qwen3-VL-8B	-1.03	6.0	14.5	-0.70	6.8	8.3
Qwen3-VL-235B-A22B	-0.26	13.4	13.5	0.27	5.4	4.8
GPT-4o	0.85	22.4	12.0	1.76	11.2	4.9

Table 3: Self-correction capability of MM-ReCoder and baselines.

Turn	Exec. Rate	Low- level	High- level	Low-level		High-level
Turn	Exec. Rate	Low- level	High- level	Avg. improve	Improved%	Avg. improve	Improved%@0.1
1	92.5	83.5	81.2	-	-	-	-
2	96.5	84.8	84.7	0.30	7.3	0.89	7.1
3	96.5	85.3	84.8	0.12	3.8	0.22	3.7
4	97.5	86.5	84.9	0.10	2.6	-0.1	2.6
5	98.3	86.4	85.5	-0.12	2.4	0.26	3.7

Table 4: MM-ReCoder iteratively self-corrects for multiple turns.

Strategy	First turn low-level	Second turn low-level	Avg. improvement on rendered charts	Percentage of repeated code
Qwen2.5-VL-7B	56.2	57.2	-0.36	10.9
+ Single-turn cold start	79.1	78.9	-0.10	81.5
+ Multi-turn cold start	75.2	73.5	-0.54	2.3
+ RL	83.5	84.8	0.30	2.2

Table 5: Performance after each stage. Cold start improves coding ability, but cannot directly enable self-correction. Multi-turn cold start recovers the model’s multi-turn ability but degrades the performance because the data in this stage is not ground truth.

Reward weights			ChartMimic
Format	Rule-based	Model-based	Exec.Rate	Low-level	High-level
0.1	0.9	0.0	95.0	84.8	78.6
0.1	0.8	0.1	95.0	84.3	83.7
0.1	0.6	0.3	94.3	84.1	83.7
0.1	0.4	0.5	95.5	84.2	85.3
0.1	0.2	0.7	93.3	83.9	83.8

Table 6: Ablation on RL reward weights. Model training and inference are single-turn without self-correction.

Panel of Figure 4 — Figure 4: Qualitative results comparing ground truth, $1^{st}$ -round generation, and $2^{nd}$ -round self-correction (left to right).

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure A1 : Training curves of MM-ReCoder. Curves on the training set are smoothed with a window size of 20 steps. We evaluate the model on ChartMimic every 40 steps. Note that the value of the rule-based reward can be slightly different from the reported model performance on low-level score because the latter is evaluated under the official evaluation codebase of ChartMimic.

Strategy	$\gamma$	$\eta$	First turn		Second turn		Avg. low-level improvement on rendered charts	Percentage of improved samples	Percentage of degraded samples	Percentage of repeated code
Strategy	$\gamma$	$\eta$	Exec.Rate	Low-Level	Exec.Rate	Low-Level	Avg. low-level improvement on rendered charts	Percentage of improved samples	Percentage of degraded samples	Percentage of repeated code
Full-trajectory	0	0	91.8	81.8	95.2	83.9	0.21	3.4	2.6	46.9
	0	0.1	64.5	66.7	96.0	84.3	10.11	87.5	2.3	0.3
Full-trajectory	0	0.3	79.0	73.7	94.5	83.2	4.57	77.1	8.3	0.0
	0	0.5	71.8	72.5	96.2	77.7	3.13	77.7	10.2	0.0
	0.5	0.1	96.5	85.0	97.5	85.5	-0.03	2.2	2.2	55.6

Appendix

Table A1: Ablation on the hyperparameters in full-trajectory optimization. All experiments are conducted without model-based reward .

Strategy	$\gamma$	$\eta$	First turn		Second turn		Avg. low-level improvement on rendered charts	Percentage of improved samples	Percentage of degraded samples	Percentage of repeated code
Strategy	$\gamma$	$\eta$	Exec.Rate	Low-Level	Exec.Rate	Low-Level	Avg. low-level improvement on rendered charts	Percentage of improved samples	Percentage of degraded samples	Percentage of repeated code
Full-trajectory	0	0	91.8	81.8	95.2	83.9	0.21	3.4	2.6	46.9
Full-trajectory	0.5	0.1	96.5	85.0	97.5	85.5	-0.03	2.2	2.2	55.6
Full-trajectory for 2 epochs	0	0	92.3	79.0	97.0	81.8	0.16	4.5	2.5	54.0
Full-trajectory for 2 epochs	0.5	0.1	94.3	82.2	94.2	82.3	0.07	2.7	1.6	87.7
Shared-first-turn	-	-	91.7	79.8	95.8	82.6	0.72	14.4	8.4	16.8

Appendix

Table A2: Ablation on the RL stage combinations. All experiments are conducted without model-based reward .

Cold Start	Performance on ChartMimic after RL
Cold Start	First turn low-level	Second turn low-level	Average improvement on rendered charts	Percentage of repeated code
None	83.0	84.9	0.53	36.3
Single-turn cold start	82.3	84.8	0.02	13.1
Single-turn cold start + multi-turn cold start	83.7	86.0	0.55	21.6

Appendix

Table A3: Ablation studies on cold start. We train the base model with different cold start recipes and the same RL strategy (full-trajectory + shared-first-turn). We evaluate their performance on ChartMimic after RL. Experiments are conducted without model-based reward .

Reward Model	ChartMimic
Reward Model	Exec.Rate	Low-level	High-level
None	95.0	84.8	78.6
Qwen2.5-VL-7B	95.3	81.0	82.6
Qwen2.5-VL-72B	95.0	84.3	83.7

Appendix

Table A4 : Ablation on the reward model under the single-turn RL setting . Qwen2.5-VL-7B as the reward model is able to improve the high-level score on ChartMimic, but replacing Qwen2.5-VL-7B with Qwen2.5-VL-72B can further boost the model.

	Win	Tie	Loss
Ours v.s. ChartCoder	$37\%$	$43\%$	$20\%$
Ours v.s. Qwen2.5-VL-72B	$40\%$	$37\%$	$23\%$
Ours v.s. Qwen3-VL-235B-A22B	$19\%$	$33\%$	$48\%$

Appendix

Table A5 : Human evaluation between MM-ReCoder and baselines on ChartMimic. The results are in line with the high-level score.

Model	Diagnosis error	Coding error	Regression
MM-ReCoder	8 (30.8%)	13 (50.0%)	5 (19.2%)
Qwen3-VL-8B	22 (37.9%)	28 (48.3%)	8 (13.8%)
Qwen3-VL-235B-A22B	15 (20.5%)	33 (45.2%)	25 (34.3%)

Appendix

Table A6 : Frequencies of self-correction failure modes.

Figure A2 : Qualitative result of MM-ReCoder for self-correction. The model removes the text labels from the pie chart in the second turn.

Figure A3 : Qualitative result of MM-ReCoder for self-correction. The model rescales the x-axis range and removes the grid lines in the second turn.

Figure A4 : Qualitative result of MM-ReCoder for self-correction. The model removes the hatch color in the second turn.

Figure A5 : Qualitative result of MM-ReCoder for self-correction. The model removes the colors filled in the radar charts in the second turn.

Figure A6 : Qualitative result of MM-ReCoder for self-correction. The model corrects the category orders in the second turn.

Figure A7 : Qualitative result of MM-ReCoder for self-correction. The model corrects the y-axis values of the data points in the second turn.

Figure A8 : Qualitative result of MM-ReCoder for self-correction. In the second turn, the model addresses the runtime error. In the third turn, the model removes the chart title overlapped with the legend.

Figure A9 : Qualitative results of MM-ReCoder and baselines.

Figure A10 : Failure cases of MM-ReCoder and baselines. The models tend to fail when the chart type is rare.

Chart-to-Code Generation

See all 10 papers

ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning
Mar 6, 2026Juyong Jiang, Jiasi Shen, Sunghun Kim +3LLM Code Generation LLM Self-Correction
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · Amazon AGI +1
ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation
Sep 21, 2026Lijian Wu, Henry Hengyuan Zhao, Zijian Zhang +3VLM Evaluation LLM Evaluation
CSU-JPG, Central South University · National University of Singapore
Schedule-and-Calibrate: Utility-Guided Multi-Task Reinforcement Learning for Code LLMs
May 7, 2026Yujia Chen, Yang Ye, Xiao Chu +2RL for Code Generation Reinforcement Learning
Harbin Institute of Technology, Shenzhen · Huawei Cloud Computing Technologies Co., Ltd.

Mar 6, 2026J/K move · Enter open · S save

ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

Juyong Jiang, Jiasi Shen, Sunghun Kim, Kang Min Yoo+2

The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · Amazon AGI+1

LLM Code Generation LLM Self-Correction LLM Self-Refinement RL for Code Generation

Open paper AskPDF arXiv

While Large Language Models (LLMs) have revolutionized code generation, standard ``System 1'' approaches that generate solutions in a single forward pass often hit a performance ceiling on complex algorithmic tasks. Existing iterative refinement strategies attempt to bridge this gap at inference time, yet they predominantly rely on external oracles, execution feedback, or computationally expensive prompt-response cycles. In this work, we propose ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model's weights. Unlike prior methods, ReflexiCoder shifts the paradigm from external-dependent refinement to an intrinsic, fully autonomous self-reflection and self-correction capabilities at inference time. We utilize an RL-only training paradigm with granular reward functions to optimize the entire reflection-correction trajectory, teaching the model how to debug without reliance on ground-truth feedback or execution engines at inference time. Extensive experiments across seven benchmarks demonstrate that our ReflexiCoder-8B establishes a new state-of-the-art (SOTA) among leading open-source models in the 1.5B to 14B range, achieving 94.51% (87.20%) on HumanEval (Plus), 81.80% (78.57%) on MBPP (Plus), 35.00% on BigCodeBench, 52.21% on LiveCodeBench, and 37.34% on CodeForces in a single-attempt setting, rivaling or surpassing proprietary models like GPT-5.1. Notably, our framework is significantly more token-efficient than base models, reducing inference-time compute overhead by approximately 40% through disciplined, efficient reasoning and reflection patterns. The source code and data are available at https://github.com/juyongjiang/ReflexiCoder.

Abstract

Chart-to-Code Generation

Explore similar work

ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation

Schedule-and-Calibrate: Utility-Guided Multi-Task Reinforcement Learning for Code LLMs