Alexa+ devices with screens display animated icons during conversations to visually reinforce what is being said. The original set of these icons was hand-drawn by designers one at a time, a process that produces polished results but is time-consuming and cannot scale to the range of topics encountered in open-ended conversation. We describe an automated pipeline, deployed in production, that generates these vectorised animated icons from a text description alone and surfaces relevant animations to customers based on their ongoing conversation. The pipeline decomposes icon creation into a sequence of stages (image generation, semantic decomposition, vectorisation, styling, and animation). Its quality is determined at the very first of these, image generation, on which every downstream stage depends. We therefore focus on improving this first and most critical stage. Because the icons are rendered on small display canvases under strict design guidelines, generations are often unsuitable for production. Supervised fine-tuning is not viable, as no dataset of line-art icons exists under these constraints. Instead, we design objective verifiers that automatically flag generations violating the design guidelines, and use them to fine-tune a flow matching image generation model via reinforcement learning from verifiable rewards (RLVR). The fine-tuned model raises the pass rate from 15.7% to 58.7% (3.7x) under a VLM judge that is independent of the reward model. Both the trained model and the verifier are deployed in the production pipeline, where the verifier also serves as the automated quality gate before final design review.
Figures & tables
Figure 1: Alexicons on an Alexa+ display device. Left: an Alexicon animating on the home screen. Right: two Alexicons shown at closer scale, rendered in the signature blue gradient with fluid line-art strokes.
Figure 2: The Alexicon production pipeline. Stage 1 generates black line-art from a text prompt and judges stroke separation quality. Stage 2 uses a VLM to identify semantic components and an image generation model to decompose the drawing into colour-coded parts. Stage 3 vectorises each component into centreline SVG paths and verifies vectorisation fidelity. Stage 4 applies the Alexicon gradient palette, animates the SVG, and exports a Lottie file for on-device playback.
Figure 3: Stroke crowding on a generated speaker icon.
Figure 4: The three crowding types detected by the measurement tool, with flagged regions in orange. (a) Narrow clearance on a timer dial: concentric rings sit too close, leaving white gaps thinner than one stroke width. (b) Fused strokes on a tennis racket: the string grid has merged into a solid black mass. (c) Enclosed holes on a cauldron: three of the four bubbles fill into solid dots at the required stroke width.
Figure 5: From measurement to labelled dataset. Left: the measurement tool flags five crowding regions on a generated guitar icon, labelling all as potential defects. Right: after human annotation, regions where tight spacing is structurally necessary (e.g. the neck meeting the head, etc.) are relabelled by-design , while regions where contours have genuinely merged remain defect . The annotated violations form the training data for the objective verifier.
Figure 6: Training reward over 5,000 optimiser steps for LoRA rank 64 and rank 128 . (a) Combined reward: both configurations improve steadily, with rank 64 reaching a higher plateau; the dashed line marks the theoretical maximum ( R=4.0 ). (b) Line separation score: the stroke-crowding component of the reward rises from ∼0.48 to ∼0.78 , showing that the model learns to space strokes farther apart over training. (c) Concept fidelity score: the VLM-based fidelity component rises from ∼0.5 to ∼2.0 , confirming that depiction accuracy improves alongside stroke separation rather than trading off against it.
pred. defect
pred. by-design
true defect
246
135
true by-design
107
281
Table 1: Confusion matrix on the 769 violations of the 136 held-out test icons, positive class defect . The 107 false defect predictions are the cell the downstream reward weights most heavily.
n
Base
Trained
Δ
Icon concepts
1,000
15.7%
58.7%
+43.0 pp
Table 2: Image generation pass rate on the 1,000 -concept evaluation list under the independent VLM judge, one generation per concept. Base is FLUX.2 [Klein] base 4B model; trained is the trained LoRA weights merged into the base weights. Both arms use same settings and are judged on centerline re-renders to the production stroke width.
Trained PASS
Trained FAIL
Base PASS
157
40
Base FAIL
530
273
Table 3: Paired verdicts on the 1,000 -concept evaluation list. The off-diagonal cells carry the test: 530 concepts the base failed and the trained model passed, against 40 in the reverse direction.
Figure 7: Base model (top) vs trained model (bottom) on five evaluation concepts, rendered at the production stroke width. In all five, the base outputs have strokes merge into solid masses, while the trained model learned to simplify internal structure and space contours apart.
Figure 8: Some concepts where both the base model (top) and trained model (bottom) fail. Failures fall into two categories: local junction merges where contours sit too close (cutting board, mixing bowl), and concepts whose visual complexity exceeds what the canvas can separate at the production stroke width (soccer kick, finish line, oven mitt).
Setting
Value
Base policy
FLUX.2 [klein] 4B model ( 4 B DiT +4 B Qwen3 encoder)
Trained parameters
LoRA only; rank 64 , alpha 128 , scaling 2.0
Optimiser
AdamW, β=(0.9,0.999) , ϵ=10−8 , weight decay 10−4
Learning rate
peak 5×10−5 , 50 -step linear warmup, cosine to 10% of peak
Gradient clipping
global norm 0.5
Precision
bfloat16 , TF32 matmuls, gradient checkpointing on
Table A.1: Shipped training configuration. Training occupied eight H200 GPUs on one node.
Group
Feature
What it captures
Per-violation
Contour persistence
how far the crowding runs along the contour, in stroke widths
Crowding type
which of the three types (narrow, fused, enclosed)
Region area
absolute size of the flagged region
Region aspect ratio
elongated channel vs. compact junction
Region height
height of the flagged region in stroke widths
Icon-level
Fusion count
how many fused-stroke regions the drawing has overall
Table B.1: The nine features used by the objective verifier, grouped by scope. Five describe the flagged region itself, two summarise the icon’s overall crowding, and two relate a region to its siblings. No feature reads an absolute canvas position, leaving the classifier invariant to translation and reflection.
Figure 9: Concepts where the base model (top of each pair) fails and the trained model (bottom) passes. The trained model consistently simplifies internal structure and spaces contours apart, from simple shapes (gift bow, blooming flower) to complex compositions (cauldron brew, top hat rabbit, vanishing act).
Figure 10: Concepts where both the base model (top of each pair) and the trained model (bottom) pass. These are typically simpler shapes or concepts with naturally wide spacing, confirming that fine-tuning does not degrade quality on concepts the base model already drew well.
Figure 11: Concepts where both the base model (top of each pair) and the trained model (bottom) fail. Failures are either local junction merges on moderately complex objects (measuring cup, trophy cup, thank you card) or complexity-driven: the concept demands more visual structure than the model can represent legibly (basketball arc, skateboard kick, compass).
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.
Generating Scalable Vector Graphics (SVG) code from natural-language instructions is an open-ended task without absolute visual ground truth, leaving both evaluation and policy optimization without a faithful signal. Scalar metrics (CLIP, Aesthetic) calibrated on natural images transfer poorly to stylized vector content, and reusing them as RL rewards triggers reward hacking. We address both limitations with rubric-based scoring. We first establish empirically that prompting a vision-language judge with a multi-axis rubric correlates with human judgments far better than scalar metrics, both across samples and within instructions. Building on this finding, we introduce RULER (Instance-aware Rubric Rewards for Reinforcement Learning), which converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes; a judge VLM scores rendered rollouts item-by-item, and the weighted satisfactions form a fine-grained reward optimized via Group Relative Policy Optimization. Because the rubric is derived from text alone, RULER requires neither paired SVG ground truth nor human preference labels. On MMSVG-Illustration and MMSVG-Icon, RULER lifts the rubric score from 0.432/0.395 to 0.693/0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, with ablations identifying rubric design as the active lever for RL on open-ended SVG generation. The project page is available at https://hangyuran.github.io/RULER/.
Hangyu Ran, Yuhao Zheng, Yingying Zhang +2
Ant Group · The Hong Kong University of Science and Technology (Guangzhou) · Independent Researcher +1
Most reward models for visual generation reduce rich human judgments to a single unexplained score, discarding the reasoning that underlies preference. We show that teaching reward models to produce explicit, multi-dimensional critiques before scoring transforms them from passive evaluators into active optimization tools, improving generators in two complementary ways: at training time, structured rationales provide interpretable, fine-grained rewards for reinforcement learning; at test time, a Generate-Critique-Refine loop turns critiques into targeted prompt revisions that improve outputs without any parameter updates. To train such a reward model without costly rationale annotations, we introduce Preference-Anchored Rationalization (PARROT), a principled framework that recovers high-quality rationales from readily available preference data through anchored generation, consistency filtering, and distillation. The resulting model, RationalRewards (8B), achieves state-of-the-art preference prediction among open-source reward models, competitive with Gemini-2.5-Pro, while using 10-20x less training data than comparable baselines. As an RL reward, it consistently improves text-to-image and image-editing generators beyond scalar alternatives. Most strikingly, its test-time critique-and-refine loop matches or exceeds RL-based fine-tuning on several benchmarks, suggesting that structured reasoning can unlock latent capabilities in existing generators that suboptimal prompts fail to elicit.
Reinforcement learning is increasingly used to post-train vision-language models for image-to-code generation, such as generating SVG code from a reference image, by optimizing rewards computed from the final rendered output. However, relying on a single terminal reward provides sparse feedback that is poorly aligned with the contribution of individual tokens. A generated program may contain operations that accurately reproduce some parts of the target image alongside others that introduce errors, yet all tokens are trained from the same final outcome. We observe that many intermediate code prefixes are not only executable, but already produce meaningful partial renders that reflect progress toward the target. This property provides a natural source of denser supervision during generation. Based on this observation, we introduce IR4RL, an RL framework with a token-level render-progress reward that turns changes between intermediate renders into localized feedback for the generated sequence. We evaluate our approach on Image-to-SVG and Image-to-TikZ generation. Across both tasks, our method improves over supervised fine-tuning and standard GRPO, yielding new state-of-the-art open-source models. This shows that intermediate rendering provides a simple and effective source of process supervision for RL post-training of image-to-code models.