Multimodal large language models (MLLMs) achieve strong performance in translating WebUI screenshots into HTML/CSS. However, when they faced multiple frameworks (React/Vue/Angular), they often suffer from negative transfer that leads to compilation failures. These failures commonly stem from mixing framework specific syntax and violating framework constraints. This paper studies supervised fine-tuning for multi-framework WebUI code generation and proposes a phase-wise MLLM tuning method driven by compatibility and heterogeneity signals across frameworks. We first apply a normalization mapping that preserves each framework's native syntax while suppressing noise from variable naming and literal content. We then estimate cross framework heterogeneity and compatibility offline, and automatically derive training groups and their ordering by maximizing a phased objective. Finally, we perform phase-wise adapter tuning to reduce cross framework interference. Experiments on multiple benchmarks show that our proposed method improves both generation quality and compilation success rate compared with baselines.
Figures & tables
Framework
Train
Val
Test
Avg. Tokens
Vanilla
2,814
312
300
1,023
React
2,773
308
295
1,187
Vue
2,681
298
284
1,142
Angular
2,594
288
277
1,309
Total
10,862
1,206
1,156
1,165
Table 1: Dataset statistics per framework.
Method
Vanilla
React
Vue
Angular
Avg.
CLIP
CSR
CLIP
CSR
CLIP
CSR
CLIP
CSR
CLIP
CSR
Claude-3.7
0.7832
1.0000
0.7891
0.9492
0.7934
0.6655
0.6223
0.6029
0.7470
0.8044
GPT-4o
0.7635
1.0000
0.6763
0.9119
0.7734
0.6092
0.5867
0.5379
0.7000
0.7648
Gemini-2.0
0.7412
1.0000
0.7611
0.8407
0.6997
0.1408
0.6006
0.4838
0.7007
0.6163
Llama-3.2-11B
0.6304
1.0000
0.6401
0.4915
0.3241
0.5810
0.5834
0.5415
0.5445
0.6535
Qwen2.5-VL-7B
0.7411
1.0000
0.6875
0.1085
0.6452
0.1197
0.5186
0.0866
0.6481
0.3287
Table 2: Results on DesignBench Generation task. Best results are bold ; second best are underlined .
Method
Block Match
Text
Position
Color
CLIP
GPT-4o
93.0
98.2
85.5
84.1
90.4
Claude-3.7
90.2
97.5
77.9
71.4
87.0
GPT-4V
85.8
97.4
80.5
73.3
86.9
Design2Code-18B
78.5
96.4
74.3
67.0
85.8
ScreenCoder
84.3
97.1
79.2
75.6
87.3
EfficientUICoder
81.7
96.8
77.4
72.3
86.1
Table 3: Results on Design2Code benchmark. Best results are bold ; second best are underlined .
Table 4Table 5Figure 6
Figure 3: Qualitative code generation examples stratified by ( Δ , S ). Each row shows our method (center, green border) vs. Shared+Specific no-gate baseline (right, red border).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Layer
CLIP (Avg.)
CSR (Avg.)
Synergy Std.
Layer 8
0.7763±0.0079
0.8714±0.0061
0.0082
Layer 10
0.7798±0.0065
0.8741±0.0051
0.0071
Layer 12
0.7878±0.0049
0.8802±0.0043
0.0063
Layer 14
0.7924±0.0033
0.8917±0.0030
0.0041
Layer 15 (ours)
0.7950±0.0027
0.9007±0.0024
0.0031
Layer 18
0.7823±0.0038
0.8771±0.0035
0.0048
Appendix
Table 8: Effect of gradient computation layer (mean ± std over 3 seeds).
Distance Function
CLIP (Avg.)
CSR (Avg.)
Offline Comp.
MMD
0.7912±0.0033
0.8901±0.0029
1.4×
Sinkhorn-Wasserstein
0.7931±0.0030
0.8921±0.0028
3.7×
Ours (JS Divergence)
0.7950±0.0027
0.9007±0.0024
1.0×
Appendix
Table 9: Discrepancy distance function ablation (mean ± std over 3 seeds).
Normalization
CLIP (Avg.)
CSR (Avg.)
Weak (format only)
0.7812
0.8747
Strong (remove framework syntax)
0.7769
0.8694
Ours (moderate)
0.7950
0.9007
Appendix
Table 10: Code normalization strength ablation.
Method
Vanilla
React
Vue
Angular
Avg.
Claude-3.7
7.42
7.83
7.61
5.94
7.20
GPT-4o
7.23
7.41
7.39
5.61
6.91
Qwen2.5-VL-7B
6.81
5.12
5.34
1.03
4.58
ScreenCoder
7.34±0.08
7.52±0.11
7.48±0.09
6.12±0.14
7.12±0.10
EfficientUICoder
7.12±0.09
7.23±0.13
7.19±0.11
5.87±0.16
6.85±0.12
Ours
7.68±0.07
7.94±0.09
7.87±0.08
6.89±0.12
7.60±0.09
Appendix
Table 11: MLLM-as-Judge scores on the DesignBench Editing task (0–10 scale; mean ± std over 3 seeds for fine-tuned methods).
Method
Vanilla
React
Vue
Angular
Avg.
Claude-3.7
7.91
8.12
7.98
7.03
7.76
GPT-4o
7.78
7.93
7.82
6.87
7.60
Qwen2.5-VL-7B
7.34
6.23
6.47
2.14
5.55
ScreenCoder
7.62±0.08
7.81±0.10
7.74±0.09
6.58±0.13
7.44±0.10
EfficientUICoder
7.43±0.10
7.61±0.12
7.52±0.11
6.34±0.15
7.23±0.12
Ours
8.03±0.06
8.24±0.08
8.17±0.07
7.52±0.11
7.99±0.08
Appendix
Table 12: MLLM-as-Judge scores on the DesignBench Repair task (0–10 scale; mean ± std over 3 seeds for fine-tuned methods).