cs.MANov 16, 2026

Learning Design Taste through Verifiable Synthetic Environments and Agentic Reinforcement Learning

Authors: Sujay choubey, Siddhant Saxena, Sarthak Kapil, Suhas Motwani

Abstract

Code-generating language models produce user interfaces that build and run, but the pages are often weak in hierarchy, spacing, typography and composition. We study whether a small code model can learn design taste through reinforcement learning in verifiable synthetic environments. The method has three components. First, a layered verification framework splits design quality into hard gates, render-grounded metrics, rubric judges, within-group relative preference and a blinded expert panel. A signal enters the reward only if it passes an admission protocol that tests agreement with experts, causal sensitivity, resistance to hack traps and stability. Second, following SCALER [1], synthetic design environments are parameterised generators with measured difficulty ladders: base pages are written once by an agentic coder, and instances are produced by syntax-tree perturbation operators and constraint samplers, scheduled by an adaptive curriculum with a quarantine rule against reward over-optimisation. Third, a 4B code model is trained with agentic reinforcement learning over multi-turn episodes in which it edits and builds a repository and reads a textual report of its own render, while hidden graders compute the reward. The admission protocol admits 7 of 16 render signals and 6 of 10 rubric dimensions. On TD-Bench, agentic training raises the Taste Index from 0.630 to 0.795 in-family and from 0.598 to 0.700 on held-out families, and wins 0.600 of blinded expert comparisons against the base model and 0.569 against a single-turn control. Most of the advantage depends on the inference-time loop, and cross-brief distinctiveness falls slightly below that of the base model, the only success criterion not met.

Figures & tables

Explore similar work

CardsList
  1. Learning to Detect UI Principle Violations via Reinforcement Learning

    Jul 22, 2026Nishi Mehta, Swathi Alse, Himani Kumawat +2Automated EvaluationLanguage Model Auditing

  2. Verifier-Grounded Agentic RL for Provably Correct Code

    Oct 13, 2026George Ma, Yuhao Zhang, Bing He +5RL for Code GenerationVerified Code Generation

  3. Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models

    Oct 26, 2026Xilin Wei, Shengyuan Ding, Xinyu Fang +13Coding Agent BenchmarksMultimodal Agent Evaluation