Visual-ERM: Reward Modeling for Visual Equivalence

Ziyu Liu; Shengyuan Ding; Xinyu Fang; Xuanlang Dai; Penghui Yang; Jianze Liang; Jiaqi Wang; Kai Chen

cs.CVMar 13, 2026

Visual-ERM: Reward Modeling for Visual Equivalence

Authors: Ziyu Liu, Shengyuan Ding, Xinyu Fang, Xuanlang Dai, Penghui Yang, Jianze Liang, Jiaqi Wang, Kai Chen,

Organizations: Shanghai Jiao Tong University · Shanghai AI Laboratory · Fudan University · CUHK

Abstract

Vision-to-code tasks require models to reconstruct structured visual inputs, such as charts, tables, and SVGs, into executable or structured representations with high visual fidelity. While recent Large Vision Language Models (LVLMs) achieve strong results via supervised fine-tuning, reinforcement learning remains challenging due to misaligned reward signals. Existing rewards either rely on textual rules or coarse visual embedding similarity, both of which fail to capture fine-grained visual discrepancies and are vulnerable to reward hacking. We propose Visual Equivalence Reward Model (Visual-ERM), a multimodal generative reward model that provides fine-grained, interpretable, and task-agnostic feedback to evaluate vision-to-code quality directly in the rendered visual space. Integrated into RL, Visual-ERM improves Qwen3-VL-8B-Instruct by +8.4 on chart-to-code and yields consistent gains on table and SVG parsing (+2.7, +4.1 on average), and further strengthens test-time scaling via reflection and revision. We also introduce VisualCritic-RewardBench (VC-RewardBench), a benchmark for judging fine-grained image-to-image discrepancies on structured visual data, where Visual-ERM at 8B decisively outperforms Qwen3-VL-235B-Instruct and approaches leading closed-source models. Our results suggest that fine-grained visual reward supervision is both necessary and sufficient for vision-to-code RL, regardless of task specificity.

Figures & tables

Figure 1: Overview of vision-to-code reward modeling and Visual-ERM . Top: vision-to-code parses a structured visual input (chart, table, or SVG) into code or markup, which is rendered back to an image so that quality can be judged in the visual space. Bottom: we compare reward signals along four properties: visual , fine-grained , interpretable , and task-agnostic . Text-based rewards (e.g., TEDS, edit distance) ignore visual cues; vision-encoder similarities (e.g., DINO [ 34 ] , CLIP [ 29 ] ) capture visual content but remain coarse and opaque. Visual-ERM combines all four properties, providing a reliable supervisor across vision-to-code tasks.

Figure 2: Analysis of existing rewards and downstream impact. Left: a text-based reward (TEDS) and a vision-based reward (DINO) assign near-perfect scores to predictions that Visual-ERM identifies as visually wrong. Right: (a) RL training curves where text- and vision-based rewards plateau via reward hacking while Visual-ERM keeps improving; (b, c) downstream gains on Chart-to-Code and Table-to-Markdown.

Figure 3: (a) Training data and model training. From paired ground-truth and rendered images, we obtain fine-grained discrepancy annotations (category, severity, location, description) and fine-tune a base LVLM into Visual-ERM . (b) Deployment. Visual-ERM produces unified judgments across chart, SVG, and table tasks, yielding scalar rewards for RL and natural-language feedback for test-time scaling.

Figure 4: VC-RewardBench. We construct VC-RewardBench by first leveraging several advanced proprietary models for preliminary annotation, followed by manual consolidation and filtering, resulting in 1,335 high-quality instances.

Model	ChartMimic [ 41 ] (Direct)				ChartMimic [ 41 ] (Customized)				Avg $\uparrow$
Model	Exec_rate $\uparrow$	Low $\uparrow$	High $\uparrow$	Overall $\uparrow$	Exec_rate $\uparrow$	Low $\uparrow$	High $\uparrow$	Overall $\uparrow$	Avg $\uparrow$
Baseline
InternVL3.5-8B [ 38 ]	62.2	41.9	48.9	45.4	79.7	53.0	63.3	58.2	51.8
Qwen3-VL-8B-Instruct [ 3 ]	82.2	62.7	72.7	67.7	86.8	66.3	76.8	71.6	69.6
VinciCoder-8B-SFT [ 47 ]	86.2	68.2	77.5	72.9	86.2	52.3	71.5	61.9	67.4
JanusCoderV-8B [ 36 ]	80.6	65.8	73.2	69.5	80.7	66.7	74.2	70.4	70.0

Table 1: Evaluation Results on Chart-to-Code Tasks. We evaluate on ChartMimic(Direct) and ChartMimic(Customized). ChartMimic(Direct) requires generating Python code that reproduces the input chart, while ChartMimic(Customized) requires generating code for a new chart that matches the given chart’s style and data constraints. Exec_rate denotes the execution success rate; Low and High denote the low-level and high-level scores, respectively.

Model	OmniDocBench [ 26 ]			OmniDocBench-v1.5 [ 26 ]			olmOCRBench [ 28 ]	Avg $\uparrow$
Model	TEDS $\uparrow$	TEDS-S $\uparrow$	Edit-Dist $\downarrow$	TEDS $\uparrow$	TEDS-S $\uparrow$	Edit-Dist $\downarrow$	TA $\uparrow$	Avg $\uparrow$
Baseline
Qwen2.5-VL-7B [ 4 ]	74.4	80.1	58.9	69.3	74.1	58.7	71.1	64.5
InternVL3.5-8B [ 38 ]	71.0	77.6	44.9	62.9	70.4	44.1	74.1	66.7
Qwen3-VL-8B-Instruct [ 3 ]	78.9	83.9	23.2	72.7	77.2	26.9	75.3	76.8
Qwen3-VL-8B-Instruct [ 3 ]

Table 2: Evaluation Results on Table-to-Markdown. TEDS-S denotes TEDS-Structure-Only, and TA represents the table subtask of olmOCRBench. For Edit-Dist, where lower values indicate better performance, we use (

100-\text{Edit-Dist}

) when computing the Average.

Table 7

Proprietary
Model	Chart			Table			SVG			AVG
Model	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$
GPT-4o [ 1 ]	22.8	28.3	48.5	32.9	35.7	49.5	13.0	19.3	50.3	25.0	29.5	56.5
GPT-5.2 [ 35 ]	30.1	32.6	64.8	39.3	40.6	54.6	28.5	32.2	61.1	32.7	35.0	58.9
Gemini-2.5-Pro [ 8 ]	33.7	37.5	61.8	46.4	48.0	49.9	29.3	34.3	63.3	37.8	40.9	59.1
Gemini-3-Flash [ 10 ]	38.5	41.3	62.8	48.1	50.1	45.6	33.3	67.5	64.3	40.6	43.4	53.4

Table 5: Evaluation Results on VC-RewardBench. We evaluate a range of proprietary and open-source models on Visual-ERM-Bench.

F1_{h}

denotes

F1_{hard}

, the strict-match F1 score;

F1_{s}

denotes

F1_{soft}

, the soft-match F1 score; and

S_{c}

denotes the correlation score, measuring the overall agreement between the predicted scores and the ground-truth labels.

Figure 5: Test-Time Scaling Cases. Using Visual-ERM ’s fine-grained feedback, we enable an inference, reflection and refinement loop. Predictions are generated as text and rendered as images for visualization.

Model	ChartMimic [ 41 ] (Direct)				ChartMimic [ 41 ] (Customized)				Avg $\uparrow$
Model	exec_rate $\uparrow$	average $\uparrow$	gpt_score $\uparrow$	overall $\uparrow$	exec_rate $\uparrow$	average $\uparrow$	gpt_score $\uparrow$	overall $\uparrow$	Avg $\uparrow$
Qwen3-VL-8B-Instruct [ 3 ] (+RL on Visual-ERM)	92.5	74.4	84.6	79.5	91.5	71.8	81.1	76.5	78.0
$+$ Reflection ( $w$ Visual-ERM) 2 rounds	92.8	75.2	84.5	79.8	92.0	74.4	82.8	78.6	79.2
$+$ Reflection ( $w$ Visual-ERM) 3 rounds	92.3	74.8	85.8	80.3	94.7	77.1	86.8	82.0	81.1
$+$ Reflection ( $w$ Visual-ERM) 4 rounds	92.0	74.5	85.4	80.0	94.2	76.9	86.1	81.5	80.7

Table 6: Ablation on the number of reflection rounds in test-time scaling. We evaluate test-time scaling with different numbers of reflection and revision rounds d.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Training Data Generation. We construct our Visual-ERM ’s training data as shown in Figure.

Models	ChartQA_TEST	CharXiv_DQ	CharXiv_RQ	DocVQA_VAL	InfoVQA_VAL	AVG
Qwen3-VL-8B-Instruct	82.9	83.8	46.0	95.6	83.1	78.3
$+$ RL on Chart-to-Code	82.6	83.5	47.2	95.6	83.1	78.4
$+$ RL on Table-to-Markdown	81.5	83.8	46.7	95.5	83.1	78.1
$+$ RL on SVG-to-Code	83.2	83.5	46.4	95.8	83.6	78.5

Appendix

Table 7: Results on general VQA benchmarks. We evaluate the policy model trained with Visual-ERM -guided RL on multiple general-purpose benchmarks (with an emphasis on chart- and document-centric VQA) to examine whether its improved vision-to-code performance comes at the cost of general multimodal capability.

Model	Chart			Table			SVG			AVG
Model	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$
Visual-ERM
$+$ Chart-Data-Only	40.1	42.6	58.1	17.2	20.4	10.5	11.3	15.3	50.1	26.3	29.3	23.7
$+$ Table-Data-Only	17.1	20.1	49.3	53.6	54.9	68.8	14.1	19.0	55.2	31.7	34.2	56.6
$+$ SVG-Data-Only	16.4	21.0	56.0	11.4	17.6	37.0	26.4	31.0	57.3	17.5	22.6	44.2
$+$ Mix-Data	39.9	42.8	61.2	56.4	57.6	74.8	28.3	32.6	59.6	42.1	44.7	58.4

Appendix

Table 8: Effect of Multi-Task Data Mixing. We compare reward models trained on mixed multi-task data vs. single-task data.

Baseline
Model	OmniDocBench			OmniDocBench-v1.5			olmOCRBench	Avg $\uparrow$
Model	TEDS $\uparrow$	TEDS-S $\uparrow$	Edit-Dist $\downarrow$	TEDS $\uparrow$	TEDS-S $\uparrow$	Edit-Dist $\downarrow$	TA $\uparrow$	Avg $\uparrow$
Qwen3-VL-8B-Instruct	78.9	83.9	23.2	72.7	77.2	26.9	75.3	76.8
RL with Different Reward Model
+ RM (table-data-only)	79.5	84.4	21.6	74.4	79.4	25.0	79.4	78.6
$\Delta$	+0.6	+0.5	+1.6	+1.7	+2.2	+1.9	4.1	+1.8

Appendix

Table 9: Ablation Study of Multi-Task Data Mixing . We perform RL on the table parsing task using different reward models: one reward model is trained only on table data, while the other reward model is jointly trained on table, chart, and SVG data. TEDS-S denotes TEDS-Structure-Only, and TA represents the table subtask of olmOCRBench. For Edit-Dist, where lower values indicate better performance, we use (

100-\text{Edit-Dist}

) when computing the Average.

Judger	Chart			Table			SVG			AVG
Judger	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$	$F1_{h}$	$F1_{s}$	$S_{c}$
GPT-5-mini	39.9	42.8	61.2	56.4	57.6	74.8	28.3	32.6	59.6	42.1	44.7	58.4
GPT-5.2	40.9	45.4	61.2	55.0	57.1	74.8	24.5	31.6	59.6	40.8	45.1	58.4
Gemini2.5-Pro	40.8	42.1	61.2	54.8	55.7	74.8	27.1	28.7	59.6	41.2	42.4	58.4
Gemini3-Flash	41.5	43.7	61.2	56.5	57.8	74.8	28.5	32.1	59.6	43.0	44.9	58.4

Appendix

Table 10: Ablation on LLM Judges. VC-RewardBench uses LLM-assisted judging to verify answer correctness. To assess the robustness of this evaluation protocol, we compare multiple LLM-as-Judge models and report the resulting consistency across judges.

Model	OmniDocBench			OmniDocBench-v1.5			olmOCRBench	Avg $\uparrow$
Model	TEDS $\uparrow$	TEDS-S $\uparrow$	Edit-Dist $\downarrow$	TEDS $\uparrow$	TEDS-S $\uparrow$	Edit-Dist $\downarrow$	TA $\uparrow$	Avg $\uparrow$
Qwen3-VL-8B-Instruct	78.9	83.9	23.2	72.7	77.2	26.9	75.3	76.8
GRPO (w/o RSR)	80.9	85.6	21.8	75.2	79.9	25.9	79.0	79.0
GRPO (w RSR)	81.4	86.3	20.7	75.4	80.4	24.2	78.1	79.5

Appendix

Table 11: Ablation Study of Reward Design . RSR denotes the Render-Success Reward. TEDS-S denotes TEDS-Structure-Only, and TA represents the table subtask of olmOCRBench. For Edit-Dist, where lower values indicate better performance, we use (

100-\text{Edit-Dist}

) when computing the Average.

Figure 7: Prompt for distillation. The prompt used to distill GPT-5-mini to construct reward-modeling training data for Visual-ERM . We show the Chart-specific prompt here; due to its length, this table includes only the first part.

Figure 8: Prompt for distillation. The prompt used to distill GPT-5-mini to construct reward-modeling training data for Visual-ERM . We show the Chart-specific prompt here; due to its length, this table includes only the second part.

Model	UniSVG(ISVGEN)
Model	SSIM $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$	Score $\uparrow$
Qwen3-VL-8B-Instruct	60.9	60.0	73.3	64.2
$+$ Reflection (Visual-ERM)	60.4	58.1	74.6	65.2
Qwen3-VL-8B-Instruct (+RL on Visual-ERM)	64.3	49.7	77.5	69.4
$+$ Reflection (Visual-ERM)	63.9	49.2	78.1	69.8

Appendix

Table 12: Test-Time Scaling on SVG-to-Code . We evaluate TTS on the SVG-to-Code task. Leveraging Visual-ERM as an evaluator, the model performs multiple rounds of self-reflection and revision, leading to improved parsing performance.

Figure 9: Visual-ERM inference prompt. The prompting template used by Visual-ERM at inference time. This prompt is specialized for the Chart-to-Code setting.

Figure 10: Visual-ERM inference prompt. The prompting template used by Visual-ERM at inference time. This prompt is specialized for the Table-to-Markdown setting.

Figure 11: Visual-ERM inference prompt. The prompting template used by Visual-ERM at inference time. This prompt is specialized for the SVG-to-Code setting.

Figure 12: VC-RewardBench evaluation prompt. For the subjective, free-form error descriptions in VC-RewardBench, we use an LLM-assisted judge to perform error matching and verification; the prompt used in this evaluation protocol is shown here.

Figure 13: Case study. We show representative examples from the Chart-to-Code reward-model training data and the benchmark.

Figure 14: Case study. We show representative examples from the Table-to-Markdown reward-model training data and the benchmark.

Figure 15: Case study. We show representative examples from the SVG-to-Code reward-model training data and the benchmark.

Figure 16: Further Case Analysis. We present additional cases to highlight the limitations of current reward models.

Reward Modeling

Reward modeling learns scores or rankings that capture human goals and task success, guiding AI selection, reinforcement learning, and alignment.

See all 325 papers

Jun 30, 2026

UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization

Visual-to-Code generation, which transforms scientific plots, vector graphics, and webpages into executable scripts, demands a level of pixel-precise alignment that standard Multimodal Large Language Models (MLLMs) fail to achieve through Supervised Fine-Tuning (SFT) alone. While Reinforcement Learning (RL) offers a theoretical pathway to bridge this gap, its application is hindered by two fundamental obstacles: (1) Reward Coarseness, where semantic metrics like CLIP scores fail to penalize fine-grained element deviations, and (2) Exploration Stagnation, where the sparse, heterogeneous code search space prevents the policy from bootstrapping valid trajectories. To overcome these limitations, we introduce UniCoder, a unified RL framework that integrates two novel mechanisms. First, we propose Symbolic Attribute Alignment, which employs a lightweight auxiliary LLM to parse generated code into discrete visual attributes (e.g., hex colors, coordinate limits), enabling dense, element-wise reward computation. Second, to escape local optima, we devise Reference-Guided Code Optimization, a strategy that dynamically injects ground-truth trajectories into low-performing rollout groups, transforming blind exploration into guided policy improvement. Extensive experiments on ChartMimic, UniSVG, Design2Code and ScreenBench benchmarks demonstrate that our 8B-parameter model not only surpasses all open-source baselines but also achieves state-of-the-art performance comparable to proprietary models, establishing a new paradigm for generalized visual-to-code synthesis.

Yaozhi Zheng, Yilei Jiang, Manyuan Zhang +5

The Chinese University of Hong Kong · Shanghai Artificial Intelligence Laboratory

RL for Code Generation Reinforcement Learning Reward Shaping Code Generation Multimodal Large Language Models

Ask PDF

Sep 28, 2026

Reinforcement Learning from Intermediate Renders for Image-to-Code Generation

Reinforcement learning is increasingly used to post-train vision-language models for image-to-code generation, such as generating SVG code from a reference image, by optimizing rewards computed from the final rendered output. However, relying on a single terminal reward provides sparse feedback that is poorly aligned with the contribution of individual tokens. A generated program may contain operations that accurately reproduce some parts of the target image alongside others that introduce errors, yet all tokens are trained from the same final outcome. We observe that many intermediate code prefixes are not only executable, but already produce meaningful partial renders that reflect progress toward the target. This property provides a natural source of denser supervision during generation. Based on this observation, we introduce IR4RL, an RL framework with a token-level render-progress reward that turns changes between intermediate renders into localized feedback for the generated sequence. We evaluate our approach on Image-to-SVG and Image-to-TikZ generation. Across both tasks, our method improves over supervised fine-tuning and standard GRPO, yielding new state-of-the-art open-source models. This shows that intermediate rendering provides a simple and effective source of process supervision for RL post-training of image-to-code models.

Omri Kaduri, Kate Feingold, Phillip Isola +1

Weizmann Institute of Science · MIT

RL for Code Generation Image-to-Code Generation Vector Graphics Generation Code Generation Process Supervision RL for VLMs

Ask PDF

Aug 11, 2026

VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?

Vision-language models (VLMs) have shown strong capabilities in generating visualization code from textual or visual specifications. However, real-world visualization authoring is inherently iterative: users frequently revise existing visualizations to repair flawed charts or adapt them to desired styles. Existing benchmarks primarily evaluate generation from scratch, leaving visualization code editing from multimodal feedback largely unexplored. We introduce VisEditBench, a benchmark of 1,395 human-annotated visualization code-editing tasks grounded in realistic visualization workflows and failure cases. VisEditBench covers two practical settings: feedback-guided repair, where models revise visualization code using buggy or marked charts together with textual feedback, and reference-guided restyling, where models modify code to match a target chart image. Evaluating 20 state-of-the-art VLMs reveals that visualization code editing remains challenging: Claude-4.6-Sonnet achieves the best overall pass rate of 74.46%, while most open-source models remain below 50%. Performance is particularly weak on visually grounded style adaptation, where Claude-4.6-Sonnet achieves only 55.71%. To establish a strong baseline, we further propose VisEditAgent, a render-grounded editing framework that iteratively generates, executes, validates, and refines candidate edits. Built on GPT-4o, VisEditAgent improves overall pass rate from 55.75% to 67.99%, demonstrating the importance of render-grounded feedback for faithful visualization editing. We will release VisEditBench at https://github.com/vis-nlp/VisEditBench.

Mizanur Rahman, Arshia Azimlu, Shadikur Rahman +4

‡York University · $Nanyang Technological University · ¶Salesforce AI Research

VLM Evaluation Data Visualization Execution-Guided Code Generation Code Generation Evaluation

Ask PDF

Abstract

Reward Modeling

Explore similar work

UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization

Reinforcement Learning from Intermediate Renders for Image-to-Code Generation

VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?