cs.AINov 3, 2026

Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning

Authors: Xujia Wang, Yixuan Liu, Shaohan Huang, Furu Wei, Lei Hou, Bin Xu, Juanzi Li

Organizations: Tsinghua University · Microsoft Research

Abstract

Generating HTML from natural-language specifications is an important application of code large language models (LLMs). Despite producing renderable pages, current models often fail to ensure functionality and visual quality during actual interaction. Existing methods typically target task-specific webpage supervision, leaving demanding interactive artifact optimization underexplored. Using mini-games as a challenging testbed, we expose limitations in interaction experience, reward signal, and credit assignment fidelity that are obscured by shallow webpage interactions. We introduce WebCatalyst, a browser-interaction reinforcement learning framework with three complementary components: a game-level GUI assessor for sustained interaction and evaluation, Visual Anchor for mitigating VLM reward hacking, and Action-Trace Credit for grounding token-level credit assignment in runtime execution traces. Trained on fewer than 1K requests, predominantly for games, WebCatalyst raises the Cookie-Bench Mini Games score from 68.8 to 75.3 and also the WebGen-Bench accuracy from 48.8 to 56.9 on Qwen3.6-35B-A3B, outperforming Claude Opus 4.7 in average rank across evaluated metrics. Further analysis shows that Visual Anchor prevents aesthetic degradation, while runtime-grounded credit assignment accelerates functional improvement. Together, our results show that improving feedback fidelity is the key to substantially advancing interactive HTML generation.

Figures & tables

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. WebWorld: The Browser as a World Model for Self-Improving Web Code

    Aug 31, 2026Jiajun Wu, Jian Yang, Yaxin Du +7Self-Training for Language ModelsCode Generation

  2. HTMLCure: Turning Browser Experience into State Guided Repair for Interactive HTML

    May 26, 2026Jiajun Wu, Jian Yang, Tuney Zheng +4Automated Software TestingAutomated Program Repair

  3. WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness

    Oct 2, 2026Yun-Yun Tsai, Yuning Mao, Shiqi Wang +2Code GenerationCode Generation Evaluation