RULER: Instance-aware Rubric Rewards for SVG Generation

Hangyu Ran; Yuhao Zheng; Yingying Zhang; Kevin Qinghong Lin; Han Peng

cs.CVSep 21, 2026

RULER: Instance-aware Rubric Rewards for SVG Generation

Authors: Hangyu Ran, Yuhao Zheng, Yingying Zhang, Kevin Qinghong Lin, Han Peng

Organizations: Ant Group · The Hong Kong University of Science and Technology (Guangzhou) · Independent Researcher · University of Oxford

Abstract

Generating Scalable Vector Graphics (SVG) code from natural-language instructions is an open-ended task without absolute visual ground truth, leaving both evaluation and policy optimization without a faithful signal. Scalar metrics (CLIP, Aesthetic) calibrated on natural images transfer poorly to stylized vector content, and reusing them as RL rewards triggers reward hacking. We address both limitations with rubric-based scoring. We first establish empirically that prompting a vision-language judge with a multi-axis rubric correlates with human judgments far better than scalar metrics, both across samples and within instructions. Building on this finding, we introduce RULER (Instance-aware Rubric Rewards for Reinforcement Learning), which converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes; a judge VLM scores rendered rollouts item-by-item, and the weighted satisfactions form a fine-grained reward optimized via Group Relative Policy Optimization. Because the rubric is derived from text alone, RULER requires neither paired SVG ground truth nor human preference labels. On MMSVG-Illustration and MMSVG-Icon, RULER lifts the rubric score from 0.432/0.395 to 0.693/0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, with ablations identifying rubric design as the active lever for RL on open-ended SVG generation. The project page is available at https://hangyuran.github.io/RULER/.

Figures & tables

Figure 1: Comparison of evaluation metrics and the RULER rubric. Left: Standard scalar metrics (CLIP and Aesthetic) mis-rank or fail to penalize a broken SVG, whereas our Rubric score aligns with human preference. Right: The structured six-item instance-aware rubric spanning semantic, visual, and stylistic axes.

Reward Paradigm	Representative Metric	Visual	Semantic	GT-Free	Multi-Axis	Instance-
Reward Paradigm	Representative Metric	Granularity	Granularity	GT-Free	Multi-Axis	Aware
Pixel-based	SSIM / PSNR Wang et al. (2004)	Fine-grained	N/A	✗	✗	✗
Rule-based	Code Length Rodriguez et al. (2025b)	N/A	N/A	✓	✗	✗
Embedding-based	CLIPScore Hessel et al. (2021)	Coarse-grained	Coarse-grained	✓	✗	✗
Rubric-based	Universal Rubric He et al. (2026)	Fine-grained	Fine-grained	✗	✓	✗
Rubric-based	Instance-aware Rubric (RULER)	Fine-grained	Fine-grained	✓	✓	✓

Table 1: Comparison of reward paradigms for SVG generation. We evaluate existing approaches across five critical desiderata. GT-free denotes that ground-truth visual references are not required.

Figure 2: Overview of RULER. A frontier model derives an instance-aware rubric—six items across semantic, visual, and stylistic axes—from each text instruction (left). The policy samples SVG rollouts that are rendered and scored by a judge VLM against the rubric; the weighted per-item satisfactions form the GRPO reward signal that drives policy updates (right).

Figure 3: Rubric score shows superior human alignment compared with Aesthetic and CLIP.

Diffusion-Optimized Methods
Method	MMSVG-Illustration					MMSVG-Icon
Method	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens
VectorFusion	0.313	4.930	0.265	0.611	31.4k	0.309	4.624	0.254	0.510	31.4k
SVGDreamer	0.293	5.119	0.266	0.547	124.3k	0.276	4.850	0.253	0.413	124.3k
Foundation LLMs
Qwen3-8B	0.244	4.309	0.233	0.432	0.4k	0.266	4.424	0.234	0.395	0.3k

Table 2: Main results on MMSVG benchmarks. Our method achieves the best universal rubric scores on both benchmarks while maintaining competitive CLIP and HPS performance with significantly fewer tokens than optimization-based methods.

Baseline	Win	Tie	Loss	Win Rate
Qwen3-8B	122	14	14	89.7%
Qwen3-32B	84	23	43	66.1%
VectorFusion	73	13	64	53.3%
OmniSVG	93	11	46	66.9%
JanusCoder	139	6	5	96.5%

Table 3: Blinded human preference results. Win rates compare RULER against each baseline on 150 MMSVG-Bench prompts and exclude ties.

Method	MMSVG-Illustration					MMSVG-Icon
Method	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens
Zero-Shot	0.244	4.309	0.233	0.432	0.4k	0.266	4.424	0.234	0.395	0.3k
C+A+H RL	0.196	6.697	0.235	0.464	0.7k	0.232	6.210	0.239	0.262	6.3k
Universal Rubric RL	0.244	4.813	0.241	0.660	0.7k	0.266	4.793	0.244	0.591	0.3k
RULER	0.249	4.859	0.246	0.693	2.0k	0.269	4.873	0.247	0.683	0.7k

Table 4: Comparison of different RL reward designs on MMSVG benchmarks. C, A and H denote CLIP, Aesthetic and HPS, respectively.

Figure 4: Ablation of rubric design on MMSVG-Illustration and MMSVG-Icon.

Method	MMSVG-Illustration				MMSVG-Icon
Method	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$
Qwen3-4B	$0.240{\pm}0.002$	$4.227{\pm}0.034$	$0.231{\pm}0.001$	$0.372{\pm}0.017$	$0.262{\pm}0.002$	$4.402{\pm}0.021$	$0.232{\pm}0.001$	$0.338{\pm}0.017$
RULER-4B	$\mathbf{0.244{\pm}0.001}$	$\mathbf{4.616{\pm}0.020}$	$\mathbf{0.241{\pm}0.001}$	$\mathbf{0.561{\pm}0.008}$	$\mathbf{0.268{\pm}0.001}$	$\mathbf{4.778{\pm}0.019}$	$\mathbf{0.244{\pm}0.000}$	$\mathbf{0.559{\pm}0.006}$
Qwen3-8B	$0.249{\pm}0.003$	$4.299{\pm}0.015$	$0.234{\pm}0.001$	$0.440{\pm}0.011$	$0.266{\pm}0.003$	$4.434{\pm}0.015$	$0.234{\pm}0.001$	$0.394{\pm}0.016$
Qwen3-32B	$\mathbf{0.265{\pm}0.004}$	$4.395{\pm}0.035$	$0.241{\pm}0.000$	$0.585{\pm}0.005$	$\mathbf{0.279{\pm}0.002}$	$4.483{\pm}0.022$	$0.242{\pm}0.001$	$0.566{\pm}0.026$
JanusCoder	$0.221{\pm}0.018$	$4.357{\pm}0.053$	$0.224{\pm}0.006$	$0.313{\pm}0.109$	$0.249{\pm}0.006$	$4.453{\pm}0.031$	$0.230{\pm}0.003$	$0.324{\pm}0.040$

Table 5: Robustness across base-model scales on MMSVG benchmarks. We compare RULER-4B and RULER-8B with their corresponding base models and representative baselines. Results are reported as mean

\pm

standard deviation over five runs.

Method	Rubric Generator	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$
Qwen3-8B	–	$0.266{\pm}0.003$	$4.434{\pm}0.015$	$0.234{\pm}0.001$	$0.394{\pm}0.016$
RULER	Claude-Opus-4.6	$0.269{\pm}0.001$	$\mathbf{4.868{\pm}0.020}$	$\mathbf{0.247{\pm}0.000}$	$\mathbf{0.662{\pm}0.016}$
RULER	GPT-5.5	$\mathbf{0.278{\pm}0.001}$	$4.708{\pm}0.016$	$0.244{\pm}0.000$	$0.646{\pm}0.007$

Table 6: Robustness across rubric generators on MMSVG-Icon. RULER uses the same Qwen3-8B policy with different rubric generators. Results are reported as mean

\pm

standard deviation over five runs.

Figure 5: Qualitative comparison of scalable vector graphics generation between RULER and four baselines.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Comparison	Win	Tie	Loss	Win Rate
RULER vs. Qwen3-8B	122	14	14	89.7%
RULER vs. Qwen3-32B	84	23	43	66.1%
RULER vs. VectorFusion	73	13	64	53.3%
RULER vs. OmniSVG	93	11	46	66.9%
RULER vs. JanusCoder	139	6	5	96.5%

Appendix

Table 7: Blinded pairwise human preference results on 150 MMSVG-Bench prompts for each baseline comparison. Win rates are computed after excluding ties.

Method	MMSVG-Icon	MMSVG-Illustration
RULER	99.3%	100.0%
OmniSVG	99.3%	99.3%
DeepSeek-V3	100.0%	99.3%
JanusCoder	68.0%	68.7%
Qwen3-8B	94.7%	89.3%
Qwen3-32B	99.3%	100.0%

Appendix

Table 8: Render success rates under the single-run benchmark evaluation.

Method	Rubric Generator	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$
Qwen3-8B	–	$0.266{\pm}0.003$	$4.434{\pm}0.015$	$0.234{\pm}0.001$	$0.394{\pm}0.016$
Qwen3-32B	–	$\mathbf{0.279{\pm}0.002}$	$4.483{\pm}0.022$	$0.242{\pm}0.001$	$0.566{\pm}0.026$
JanusCoder	–	$0.249{\pm}0.006$	$4.453{\pm}0.031$	$0.230{\pm}0.003$	$0.324{\pm}0.040$
RULER	Claude-Opus-4.6	$0.269{\pm}0.001$	$\mathbf{4.868{\pm}0.020}$	$\mathbf{0.247{\pm}0.000}$	$\mathbf{0.662{\pm}0.016}$
RULER	GPT-5.5	$0.278{\pm}0.001$	$4.708{\pm}0.016$	$0.244{\pm}0.000$	$0.646{\pm}0.007$

Appendix

Table 9: Robustness to different rubric generators on MMSVG-Icon. Results are averaged over five independently seeded inference runs and reported as mean

\pm

standard deviation.

Statistic	MMSVG-Icon	MMSVG-Illustration	Overall
Prompts	19,531	11,206	30,737
Avg. Input Tokens	1,959	1,957	1,958
Avg. Rubric Tokens	1,010	1,057	1,027
Avg. Output Tokens	1,934	2,740	2,228
Total Cost	$1,135.77	$877.35	$2,013.12
Cost / Prompt	$0.0582	$0.0783	$0.0655

Appendix

Table 10: Rubric-generation cost for the RL training data. Avg. Output Tokens includes both the generated ideal SVG and rubric; Avg. Rubric Tokens counts the rubric portion only.

Text-to-SVG Generation

See all 13 papers

Sep 3, 2026

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

Scalable Vector Graphics (SVG) generation is attracting increasing attention as generative models improve in expressiveness and controllability. Progress, however, is held back by the lack of domain-specific evaluation protocols: current practice relies on metrics designed for natural images, most notably CLIPScore, which was never trained on vector graphics and aligns only partially with human judgment. We introduce \ours, a human-aligned evaluation framework for text-to-SVG generation. Through controlled caption and image perturbations, we first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations, and that off-the-shelf Vision-Language Model (VLM) judges, while more sensitive, respond unevenly across error types and SVG styles. We then introduce a human-annotated dataset for Semantic Alignment, measuring how faithfully a generated SVG reflects its caption. Building on it, we develop two complementary evaluators: CLIP scorers adapted to vector graphics and then aligned to human preferences, for fast large-scale evaluation, and a VLM judge trained with supervised fine-tuning and reward-shaped reinforcement learning, for more expressive and interpretable assessment. Using both, we benchmark major open-source, commercial, and optimization-based SVG generators on an independent caption set.

Marco Cipriano, Leonardo Zini, Alexandra Schild +5

Hasso-Plattner Institute, Germany · University of Modena and Reggio Emilia, Italy

VLM Evaluation Human Preference Modeling T2I Generation Text-to-SVG Generation Semantic Alignment Vector Graphics Generation

Ask PDF

Feb 22, 2026

VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing

We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, complex editing, and visual understanding. VectorGym addresses the lack of realistic, challenging benchmarks aligned with professional design workflows. Our benchmark comprises four tasks with expert human-authored annotations: the novel Sketch2SVG task (VG-Sketch); a new SVG editing dataset (VG-Edit) featuring complex, multi-step edits with higher-order primitives; Text2SVG generation (VG-Text); and SVG captioning (VG-Cap). Unlike prior benchmarks that rely on synthetic edits, VectorGym provides gold-standard human annotations that require semantic understanding and design intent. We also provide a multi-task reinforcement learning baseline that jointly optimizes across all four tasks using rendering-based rewards. This baseline, built on GRPO with curriculum learning, trains a Qwen3-VL 8B model that achieves state-of-the-art performance among open-source models, surpassing much larger models including Qwen3-VL 235B and matching GPT-4o. We also introduce a VLM-as-a-Judge metric for SVG generation, validated through human correlation studies. Our evaluation of frontier VLMs reveals significant performance gaps, positioning VectorGym as a rigorous framework for advancing visual code generation.

Joan Rodriguez, Haotian Zhang, Abhay Puri +14

Mila, Quebec AI Institute · ÉTS Montréal · QuiverAI +11

VLM Evaluation Code Generation Image Editing Code Generation Benchmarks Vector Graphics Generation Text-to-SVG Generation

Ask PDF

Aug 27, 2026

RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing

Reward models play an essential role in aligning visual generative models, yet most existing visual reward models use a single scalar score or rely on fixed criteria that cannot adapt to different instructions. This limits both interpretability and task sensitivity, especially for text-to-image generation and instruction-based image editing, where different inputs require different evaluation dimensions. We propose RubricRM, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images. We train dedicated RubricRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the rubric-based scoring paradigm, while GRPO further improves scoring through fine-grained dimension-level rewards. Experiments on multiple generation and editing benchmarks show that RubricRM outperforms existing specialized reward models and remains competitive with strong proprietary MLLM judges despite using smaller backbones. Our models, data, and code are available at https://github.com/zijiankan/RubricRM.

Zijian Kan, Wei Wang, Long Luo +6

Peking University · Alibaba Group · Alibaba DAMO Academy

Reward Modeling Multimodal Reward Modeling Rubric-Based Evaluation T2I Generation Evaluation Image Editing Evaluation

Ask PDF

Abstract

Text-to-SVG Generation

Explore similar work

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing

RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing