MulFCoder: Framework-conditioned Multi-agent for MLLM-based Multi-framework Front-end Code Generation

Jie Wu; Haoran Ma; Shisong Tang; Yulin Xu; Xiaoyu Kang; Jiechao Gao

cs.SEOct 25, 2026

MulFCoder: Framework-conditioned Multi-agent for MLLM-based Multi-framework Front-end Code Generation

Authors: Jie Wu, Haoran Ma, Shisong Tang, Yulin Xu, Xiaoyu Kang, Jiechao Gao

Organizations: Communication University of China · Harvard University · Tsinghua university · University of Southern California ·

Abstract

Generating runnable front-end code from UI screenshots is a long-standing goal in automated software engineering. Existing MLLM-based methods predominantly focused on HTML/CSS, leaving multi-framework generation for React/Vue/Angular underexplored. Naively modifying prompts leads to substantial performance gaps across multi-framework and highly framework-specific error modes. To address this, we propose MulFCoder, a framework-conditioned multi-agent method that explicitly encodes framework constraints to bring multi-framework differences into a decidable rule space. MulFCoder orchestrates four agents: Grounder constructs an ElementTable, ContentTable, and macro-layout regions from detected UI elements; Planner builds a DOM-like hierarchical layout tree, produces a task schedule, and derives a framework-specific file Contract; Writer generates structured file writes or patches within a restricted edit window; Judger enforces lightweight, framework-conditioned constraints to accept or reject updates and trigger bounded repairs, preventing drift and deadlocks without expensive builds. Experiments demonstrate that MulFCoder substantially improves compilation success rate and reduces framework-specific errors, with particularly pronounced gains on Angular.

Figures & tables

Panel of Figure 1 — Figure 1: (a) On Design2Code dataset, five LLMs that generate code by directly rewriting prompts exhibit a substantial gap in compilation success rate (CSR) across frameworks, indicating that multi-framework discrepancies cannot be addressed by prompt changes alone and are instead driven by framework specific syntax and engineering constraints; (b) under the same setup, the Top-3 compilation error distributions of the five LLMs across three frameworks show clear framework specific error patterns; (c) an overview of the MulFCoder.

Base Model	Design2Code		DesignBench		Design2Code		DesignBench
Base Model	CLIP	CSR	CLIP	CSR	CLIP	CSR	CLIP	CSR
	VANILLA				REACT
Gemini-2.5-Pro	$0.92^{\phantom{+0.00}}$	$1.00^{\phantom{+0.00}}$	$0.83^{\phantom{+0.00}}$	$1.00^{\phantom{+0.00}}$	$0.91^{\phantom{+0.00}}$	$0.97^{\phantom{+0.00}}$	$0.84^{\phantom{+0.00}}$	$0.91^{\phantom{+0.00}}$
+ MulFCoder	$\mathbf{0.94}^{{\color[rgb]{0.7,0,0}+0.02}}$	$1.00^{\phantom{+0.00}}$	$\mathbf{0.87}^{{\color[rgb]{0.7,0,0}+0.04}}$	$1.00^{\phantom{+0.00}}$	$\mathbf{0.93}^{{\color[rgb]{0.7,0,0}+0.02}}$	$\mathbf{0.98}^{{\color[rgb]{0.7,0,0}+0.01}}$	$\mathbf{0.85}^{{\color[rgb]{0.7,0,0}+0.01}}$	$\mathbf{0.96}^{{\color[rgb]{0.7,0,0}+0.05}}$
GPT-4o	$0.91^{\phantom{+0.00}}$	$1.00^{\phantom{+0.00}}$	$0.81^{\phantom{+0.00}}$	$1.00^{\phantom{+0.00}}$	$0.89^{\phantom{+0.00}}$	$0.96^{\phantom{+0.00}}$	$0.79^{\phantom{+0.00}}$	$0.94^{\phantom{+0.00}}$
+ MulFCoder	$\mathbf{0.93}^{{\color[rgb]{0.7,0,0}+0.02}}$	$1.00^{\phantom{+0.00}}$	$\mathbf{0.84}^{{\color[rgb]{0.7,0,0}+0.03}}$	$1.00^{\phantom{+0.00}}$	$\mathbf{0.90}^{{\color[rgb]{0.7,0,0}+0.01}}$	$\mathbf{0.98}^{{\color[rgb]{0.7,0,0}+0.02}}$	$\mathbf{0.81}^{{\color[rgb]{0.7,0,0}+0.02}}$	$\mathbf{0.98}^{{\color[rgb]{0.7,0,0}+0.04}}$

Table 1: Performance comparison of MulFCoder across different frameworks. In the Vanilla setting, we observe consistent visual improvements. However, in strict engineering frameworks (React, Vue, Angular), we notice an ”Engineering Trade-off”: while MulFCoder significantly repairs build errors (boosting CSR), this occasionally necessitates simplifying complex UI components, leading to saturated or slightly regressed visual similarity scores (CLIP).

Panel of Figure 3 — Figure 3: Compilation error distribution.

Model	Design2Code			DesignBench
	React	Vue	Angular	React	Vue	Angular
Gemini.	35.7%	57.6%	40.3%	55.6%	47.1%	27.8%
GPT.	52.6%	55.8%	47.3%	66.7%	46.7%	23.6%
Doubao.	51.7%	50.0%	47.7%	46.2%	37.0%	37.5%
Qwen.	54.7%	47.6%	44.6%	47.6%	32.4%	35.5%
LLaVA.	47.2%	42.2%	37.3%	36.0%	31.0%	31.4%

Table 2: Compile error fix ratio of MulFCoder

Panel of Figure 4 — Figure 4: An UI screenshot generation example.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Easy			Medium			Hard
Model	React	Vue	Angular	React	Vue	Angular	React	Vue	Angular
Gemini-2.5-Pro+	1.00	0.98	0.95	1.00	0.86	0.85	0.89	0.89	0.62
GPT-4o+	1.00	1.00	0.87	1.00	0.86	0.75	0.93	0.89	0.44
Doubao-Seed-1.6+	1.00	0.96	0.93	0.89	0.86	0.80	0.88	0.66	0.57
Qwen2.5-VL-7B+	0.95	0.92	0.85	0.85	0.83	0.75	0.84	0.50	0.40
LLaVA-v1.5-7B+	0.96	0.87	0.81	0.85	0.76	0.70	0.66	0.30	0.10

Appendix

Table 3: CSR of three frameworks under different difficulty levels in DesignBench.

Code Generation

Code generation studies methods for turning natural-language, visual, or formal specifications into executable programs and code edits.

See all 322 papers

MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification
Aug 4, 2026Qiming Li, Shujie Hu, Haohan Liu +3Code Generation AI Agent Benchmarks
1Harbin Institute of Technology · 2Meituan, Longcat Team · 3Peng Cheng Laboratory
From Visual Widgets to UI Code: Efficient Tool-Grounded Generation
Aug 12, 2026Houston H. Zhang, Tao Zhang, Li Gu +5Image-to-Code Generation Code Generation
McMaster University · University of Toronto · Concordia University
EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation
Sep 15, 2025Jingyu Xiao, Zhongyi Zhang, Yuxuan Wan +3Efficient Multimodal Inference Code Generation
The Chinese University of Hong Kong, Hong Kong, China · Huazhong University of Science and Technology, Wuhan, China · Singapore Management University, Singapore, Singapore +1

Aug 4, 2026J/K move · Enter open · S save

MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

Qiming Li, Shujie Hu, Haohan Liu, Xiaocheng Feng+2

1Harbin Institute of Technology · 2Meituan, Longcat Team · 3Peng Cheng Laboratory

Code Generation AI Agent Benchmarks Code Generation Benchmarks

Open paper AskPDF arXiv

Recent advances in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in web UI generation. However, existing benchmarks predominantly focus on single-turn full-page generation from scratch, overlooking the iterative workflow of real-world frontend engineering, where developers repeatedly reconstruct missing regions and modify localized elements within existing codebases. To bridge this gap, we introduce MT-Web2Code, the first multimodal coding benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification, which contains 102 tasks spanning 16 vertical domains. To construct deterministic repair trajectories without costly turn-level human annotation, we develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages. We further propose a dual-axis evaluation protocol that measures target-region fidelity and the preservation of unaffected content, where regional reconstruction is assessed by a 5-dimensional VLM-based rubric and localized modification by deterministic pixel-grounded alignment. Experiments on 13 frontier coding agents reveal that current agents struggle to faithfully reconstruct target regions while preserving unaffected content, lack fine-grained visual-code alignment for localized edits, and suffer from error snowballing over multiple turns. Beyond benchmarking, our deterministic evaluation metrics provide fine-grained feedback signals that may facilitate future research on training iterative UI coding agents. Our evaluation code and data will soon be released.

Abstract

Code Generation

Explore similar work

MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

From Visual Widgets to UI Code: Efficient Tool-Grounded Generation

EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation