RULER: Instance-aware Rubric Rewards for SVG Generation

Hangyu Ran; Yuhao Zheng; Yingying Zhang; Kevin Qinghong Lin; Han Peng

cs.CVSep 21, 2026

RULER: Instance-aware Rubric Rewards for SVG Generation

Authors: Hangyu Ran, Yuhao Zheng, Yingying Zhang, Kevin Qinghong Lin, Han Peng

Organizations: Ant Group · The Hong Kong University of Science and Technology (Guangzhou) · Independent Researcher · University of Oxford

Abstract

Generating Scalable Vector Graphics (SVG) code from natural-language instructions is an open-ended task without absolute visual ground truth, leaving both evaluation and policy optimization without a faithful signal. Scalar metrics (CLIP, Aesthetic) calibrated on natural images transfer poorly to stylized vector content, and reusing them as RL rewards triggers reward hacking. We address both limitations with rubric-based scoring. We first establish empirically that prompting a vision-language judge with a multi-axis rubric correlates with human judgments far better than scalar metrics, both across samples and within instructions. Building on this finding, we introduce RULER (Instance-aware Rubric Rewards for Reinforcement Learning), which converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes; a judge VLM scores rendered rollouts item-by-item, and the weighted satisfactions form a fine-grained reward optimized via Group Relative Policy Optimization. Because the rubric is derived from text alone, RULER requires neither paired SVG ground truth nor human preference labels. On MMSVG-Illustration and MMSVG-Icon, RULER lifts the rubric score from 0.432/0.395 to 0.693/0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, with ablations identifying rubric design as the active lever for RL on open-ended SVG generation. The project page is available at https://hangyuran.github.io/RULER/.

Figures & tables

Figure 1: Comparison of evaluation metrics and the RULER rubric. Left: Standard scalar metrics (CLIP and Aesthetic) mis-rank or fail to penalize a broken SVG, whereas our Rubric score aligns with human preference. Right: The structured six-item instance-aware rubric spanning semantic, visual, and stylistic axes.

Reward Paradigm	Representative Metric	Visual	Semantic	GT-Free	Multi-Axis	Instance-
Reward Paradigm	Representative Metric	Granularity	Granularity	GT-Free	Multi-Axis	Aware
Pixel-based	SSIM / PSNR Wang et al. (2004)	Fine-grained	N/A	✗	✗	✗
Rule-based	Code Length Rodriguez et al. (2025b)	N/A	N/A	✓	✗	✗
Embedding-based	CLIPScore Hessel et al. (2021)	Coarse-grained	Coarse-grained	✓	✗	✗
Rubric-based	Universal Rubric He et al. (2026)	Fine-grained	Fine-grained	✗	✓	✗
Rubric-based	Instance-aware Rubric (RULER)	Fine-grained	Fine-grained	✓	✓	✓

Table 1: Comparison of reward paradigms for SVG generation. We evaluate existing approaches across five critical desiderata. GT-free denotes that ground-truth visual references are not required.

Figure 2: Overview of RULER. A frontier model derives an instance-aware rubric—six items across semantic, visual, and stylistic axes—from each text instruction (left). The policy samples SVG rollouts that are rendered and scored by a judge VLM against the rubric; the weighted per-item satisfactions form the GRPO reward signal that drives policy updates (right).

Figure 3: Rubric score shows superior human alignment compared with Aesthetic and CLIP.

Diffusion-Optimized Methods
Method	MMSVG-Illustration					MMSVG-Icon
Method	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens
VectorFusion	0.313	4.930	0.265	0.611	31.4k	0.309	4.624	0.254	0.510	31.4k
SVGDreamer	0.293	5.119	0.266	0.547	124.3k	0.276	4.850	0.253	0.413	124.3k
Foundation LLMs
Qwen3-8B	0.244	4.309	0.233	0.432	0.4k	0.266	4.424	0.234	0.395	0.3k

Table 2: Main results on MMSVG benchmarks. Our method achieves the best universal rubric scores on both benchmarks while maintaining competitive CLIP and HPS performance with significantly fewer tokens than optimization-based methods.

Baseline	Win	Tie	Loss	Win Rate
Qwen3-8B	122	14	14	89.7%
Qwen3-32B	84	23	43	66.1%
VectorFusion	73	13	64	53.3%
OmniSVG	93	11	46	66.9%
JanusCoder	139	6	5	96.5%

Table 3: Blinded human preference results. Win rates compare RULER against each baseline on 150 MMSVG-Bench prompts and exclude ties.

Method	MMSVG-Illustration					MMSVG-Icon
Method	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	Tokens
Zero-Shot	0.244	4.309	0.233	0.432	0.4k	0.266	4.424	0.234	0.395	0.3k
C+A+H RL	0.196	6.697	0.235	0.464	0.7k	0.232	6.210	0.239	0.262	6.3k
Universal Rubric RL	0.244	4.813	0.241	0.660	0.7k	0.266	4.793	0.244	0.591	0.3k
RULER	0.249	4.859	0.246	0.693	2.0k	0.269	4.873	0.247	0.683	0.7k

Table 4: Comparison of different RL reward designs on MMSVG benchmarks. C, A and H denote CLIP, Aesthetic and HPS, respectively.

Figure 4: Ablation of rubric design on MMSVG-Illustration and MMSVG-Icon.

Method	MMSVG-Illustration				MMSVG-Icon
Method	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$
Qwen3-4B	$0.240{\pm}0.002$	$4.227{\pm}0.034$	$0.231{\pm}0.001$	$0.372{\pm}0.017$	$0.262{\pm}0.002$	$4.402{\pm}0.021$	$0.232{\pm}0.001$	$0.338{\pm}0.017$
RULER-4B	$\mathbf{0.244{\pm}0.001}$	$\mathbf{4.616{\pm}0.020}$	$\mathbf{0.241{\pm}0.001}$	$\mathbf{0.561{\pm}0.008}$	$\mathbf{0.268{\pm}0.001}$	$\mathbf{4.778{\pm}0.019}$	$\mathbf{0.244{\pm}0.000}$	$\mathbf{0.559{\pm}0.006}$
Qwen3-8B	$0.249{\pm}0.003$	$4.299{\pm}0.015$	$0.234{\pm}0.001$	$0.440{\pm}0.011$	$0.266{\pm}0.003$	$4.434{\pm}0.015$	$0.234{\pm}0.001$	$0.394{\pm}0.016$
Qwen3-32B	$\mathbf{0.265{\pm}0.004}$	$4.395{\pm}0.035$	$0.241{\pm}0.000$	$0.585{\pm}0.005$	$\mathbf{0.279{\pm}0.002}$	$4.483{\pm}0.022$	$0.242{\pm}0.001$	$0.566{\pm}0.026$
JanusCoder	$0.221{\pm}0.018$	$4.357{\pm}0.053$	$0.224{\pm}0.006$	$0.313{\pm}0.109$	$0.249{\pm}0.006$	$4.453{\pm}0.031$	$0.230{\pm}0.003$	$0.324{\pm}0.040$

Table 5: Robustness across base-model scales on MMSVG benchmarks. We compare RULER-4B and RULER-8B with their corresponding base models and representative baselines. Results are reported as mean

\pm

standard deviation over five runs.

Method	Rubric Generator	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$
Qwen3-8B	–	$0.266{\pm}0.003$	$4.434{\pm}0.015$	$0.234{\pm}0.001$	$0.394{\pm}0.016$
RULER	Claude-Opus-4.6	$0.269{\pm}0.001$	$\mathbf{4.868{\pm}0.020}$	$\mathbf{0.247{\pm}0.000}$	$\mathbf{0.662{\pm}0.016}$
RULER	GPT-5.5	$\mathbf{0.278{\pm}0.001}$	$4.708{\pm}0.016$	$0.244{\pm}0.000$	$0.646{\pm}0.007$

Table 6: Robustness across rubric generators on MMSVG-Icon. RULER uses the same Qwen3-8B policy with different rubric generators. Results are reported as mean

\pm

standard deviation over five runs.

Figure 5: Qualitative comparison of scalable vector graphics generation between RULER and four baselines.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Comparison	Win	Tie	Loss	Win Rate
RULER vs. Qwen3-8B	122	14	14	89.7%
RULER vs. Qwen3-32B	84	23	43	66.1%
RULER vs. VectorFusion	73	13	64	53.3%
RULER vs. OmniSVG	93	11	46	66.9%
RULER vs. JanusCoder	139	6	5	96.5%

Appendix

Table 7: Blinded pairwise human preference results on 150 MMSVG-Bench prompts for each baseline comparison. Win rates are computed after excluding ties.

Method	MMSVG-Icon	MMSVG-Illustration
RULER	99.3%	100.0%
OmniSVG	99.3%	99.3%
DeepSeek-V3	100.0%	99.3%
JanusCoder	68.0%	68.7%
Qwen3-8B	94.7%	89.3%
Qwen3-32B	99.3%	100.0%

Appendix

Table 8: Render success rates under the single-run benchmark evaluation.

Method	Rubric Generator	CLIP $\uparrow$	Aesth. $\uparrow$	HPS $\uparrow$	Rubric $\uparrow$
Qwen3-8B	–	$0.266{\pm}0.003$	$4.434{\pm}0.015$	$0.234{\pm}0.001$	$0.394{\pm}0.016$
Qwen3-32B	–	$\mathbf{0.279{\pm}0.002}$	$4.483{\pm}0.022$	$0.242{\pm}0.001$	$0.566{\pm}0.026$
JanusCoder	–	$0.249{\pm}0.006$	$4.453{\pm}0.031$	$0.230{\pm}0.003$	$0.324{\pm}0.040$
RULER	Claude-Opus-4.6	$0.269{\pm}0.001$	$\mathbf{4.868{\pm}0.020}$	$\mathbf{0.247{\pm}0.000}$	$\mathbf{0.662{\pm}0.016}$
RULER	GPT-5.5	$0.278{\pm}0.001$	$4.708{\pm}0.016$	$0.244{\pm}0.000$	$0.646{\pm}0.007$

Appendix

Table 9: Robustness to different rubric generators on MMSVG-Icon. Results are averaged over five independently seeded inference runs and reported as mean

\pm

standard deviation.

Statistic	MMSVG-Icon	MMSVG-Illustration	Overall
Prompts	19,531	11,206	30,737
Avg. Input Tokens	1,959	1,957	1,958
Avg. Rubric Tokens	1,010	1,057	1,027
Avg. Output Tokens	1,934	2,740	2,228
Total Cost	$1,135.77	$877.35	$2,013.12
Cost / Prompt	$0.0582	$0.0783	$0.0655

Appendix

Table 10: Rubric-generation cost for the RL training data. Avg. Output Tokens includes both the generated ideal SVG and rubric; Avg. Rubric Tokens counts the rubric portion only.

Text-to-SVG Generation

See all 13 papers

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation
Sep 3, 2026Marco Cipriano, Leonardo Zini, Alexandra Schild +5VLM Evaluation Human Preference Modeling
Hasso-Plattner Institute, Germany · University of Modena and Reggio Emilia, Italy
VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing
Feb 22, 2026Joan Rodriguez, Haotian Zhang, Abhay Puri +14VLM Evaluation Code Generation
Mila, Quebec AI Institute · ÉTS Montréal · QuiverAI +11
RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing
Aug 27, 2026Zijian Kan, Wei Wang, Long Luo +6Reward Modeling Multimodal Reward Modeling
Peking University · Alibaba Group · Alibaba DAMO Academy

Sep 3, 2026J/K move · Enter open · S save

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

Marco Cipriano, Leonardo Zini, Alexandra Schild, Valentin Teutschbein+4

Hasso-Plattner Institute, Germany · University of Modena and Reggio Emilia, Italy

VLM Evaluation Human Preference Modeling T2I Generation Text-to-SVG Generation Semantic Alignment Vector Graphics Generation

Open paper AskPDF arXiv

Scalable Vector Graphics (SVG) generation is attracting increasing attention as generative models improve in expressiveness and controllability. Progress, however, is held back by the lack of domain-specific evaluation protocols: current practice relies on metrics designed for natural images, most notably CLIPScore, which was never trained on vector graphics and aligns only partially with human judgment. We introduce \ours, a human-aligned evaluation framework for text-to-SVG generation. Through controlled caption and image perturbations, we first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations, and that off-the-shelf Vision-Language Model (VLM) judges, while more sensitive, respond unevenly across error types and SVG styles. We then introduce a human-annotated dataset for Semantic Alignment, measuring how faithfully a generated SVG reflects its caption. Building on it, we develop two complementary evaluators: CLIP scorers adapted to vector graphics and then aligned to human preferences, for fast large-scale evaluation, and a VLM judge trained with supervised fine-tuning and reward-shaped reinforcement learning, for more expressive and interpretable assessment. Using both, we benchmark major open-source, commercial, and optimization-based SVG generators on an independent caption set.

Abstract

Text-to-SVG Generation

Explore similar work

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing

RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing