ScreenProof: Where Is This Screen Wrong? Finding UI Faults Without a Reference
Organizations: OPPO US Research Center · Cleveland State University · Shanghai Jiao Tong University
Abstract
A phone screen can be wrong in ways that need no reference: a label that runs off the screen, letters that touch, a line that breaks one character before its end, one button with sharp corners among round ones. We ask whether a model can box every such fault on a screenshot, given no target design and no reference image. Such a critic could serve as a cheap first pass over generated pages before a designer reviews them, but only if it stays silent on clean pages. ScreenProof poses this target-free critique over three families of faults: general faults, local style faults (one element breaks the style its page sets) and text layout faults (letter spacing, line height, line breaks). A data engine makes each fault in page code and records the changed element's box, and hard negatives change every sibling alike. No frontier judge is reliable: frontier models flag 18-62% of clean real screenshots and 18-82% of clean generated pages, find 47-72% of text layout faults, and rarely find a foreign icon (5-19%). Our small critic ProofJudge (Qwen3.5-9B with LoRA), trained on code-made pages with one to four faults each and keeping a finding only when it is confident, reaches F1 0.48, the highest of all judges (Gemini 4 Argon 0.37; open models at most 0.15; 0.46 even without the confidence cut), and is the only fault-finding judge within 15% false alarms on clean pages (2% real, 9% generated), the property such a first pass needs, at about a hundredth of Argon's cost. It finds the most general faults (59%) but fewer local style faults than Argon (25% against 41%) and fewer text layout faults than the API judges (43%; Gemini 4 Argon 47%). Against a named brand the task is harder still: models know a design language about as well as its public documentation states it.
Figures & tables
| Kind | UI | Target | Locates elements | Names convention | Judge trained / admitted on faults | |
|---|---|---|---|---|---|---|
| Design2Code [ 24 ] | generation benchmark | yes | one screenshot | block matching | – | – |
| UI2Code [ 33 ] | generation | yes | one screenshot | – | – | fine-tuned whole-page judge |
| ComUICoder [ 32 ] | generation | yes | pages | per element (detector) | – | – |
| MobileForge [ 29 ] | generation benchmark | yes | screens | – | – | positive anchor only |
| MultiRef [ 5 ] | generation | – | references with roles | – | – | – |
| DS-Compliance [ 4 ] | generation study | yes | design-system spec | – | checklist | – |
| Aspect | What goes wrong | Engine fault |
|---|---|---|
| General: visible without a design target | ||
| Layout | overflow, overlap, order, missing item, two selected | each, in code |
| Readability | contrast, tap size, typeface | each, in code |
| Display | broken image, null value, needless ellipsis | each, in code |
| Local style: one element unlike the rest of its page | ||
| Style | radius, material, accent, icon | one element |
| Kind | Rows | Images |
| Clean page | 7,193 | 2,605 |
| Designed difference (faulted twin) | 630 | 591 |
| General fault | 1,542 | 1,427 |
| Local style fault | 1,609 | 1,505 |
| Transplanted component | 2,110 | 1,484 |
| Text layout fault | 972 | 847 |
| Clean pages flagged | Recall | Speed | ||||||||
| Judge | real | generated | general | consistency | Precision | F1 | F1 .3 | No answer | s/page | GPUs |
| API judges | ||||||||||
| Gemini 4 Argon | 18% | 18% | 38% | 41% | 35% | 0.37 | 0.43 | 0 | 6.1 | – |
| gpt-6.1-sol | 38% | 36% | 45% | 32% | 25% | 0.30 | 0.36 | 3 | – | – |
| Claude Opus 5.5 (medium) | 62% | 82% | 49% | 49% | 22% | 0.31 | 0.35 | 0 | – | – |
| Gemini 3.8 Flash | 38% | 66% | 40% | 26% | 22% | 0.26 | 0.29 | 0 | 20.6 | – |
| Judge | Spacing | Lines | Breaks | All | Flagged | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| touch | wide | touch | tall | wrap | lone | word | ellipsis | clean | neg. | ||
| Gemini 4 Argon | 17% | 41% | 60% | 18% | 43% | 22% | 86% | 81% | 47% | 32% | 56% |
| gpt-6.1-sol | 27% | 59% | 70% | 35% | 26% | 25% | 100% | 81% | 54% | 49% | 68% |
| Claude Opus 5.5 (medium) | 73% | 80% | 70% | 35% | 43% | 55% | 98% | 82% | 72% | 80% | 96% |
| ProofJudge (ours) | 5% | 46% | 5% | 0% | 26% | 14% | 94% | 96% | 43% | 15% | 13% |
| LoRA rank | seed 42 | seed 7 | pair | test F1 | 95% interval vs rank 16 |
|---|---|---|---|---|---|
| 16 | 0.556 | 0.517 | 0.550 | 0.405 † | – |
| 64 | 0.561 | 0.550 | 0.571 | 0.445 | |
| 128 (ours) | 0.586 | 0.598 | 0.607 | 0.481 | |
| 256 | 0.605 | 0.576 | 0.599 | – | – |
| OwlEye | |||
| Judge | F1 | Normal flagged | Accuracy |
| API judges | |||
| Gemini 4 Argon | 0.86 | 28% | 84% |
| Gemini 3.8 Flash | 0.75 | 64% | 68% |
| gpt-6.1-sol | 0.76 | 63% | 68% |
| Claude Opus 5.5 (medium) | 0.79 | 53% | 73% |
Appendix figures & tables30 assets
Supplementary material from the paper’s appendix.
Appendix
| State | Apply (picture + card text) | General design | ||||
| Model | which brand / fact | whose rule | boxed source | broken dimension | say | see |
| Chance | 19% | 13% | 13% | 20% | 25% | 17% |
| gpt-6.1-sol | 1000 /1316 | 229/412 | 33/163 | 37 /105 | 28/28 | 46/69 |
| gpt-6-astra | 983/1316 | 242/412 | 39/163 | 33/105 | 31/31 | 48/70 |
| Gemini 4 Argon | 925/1315 | 285 /408 | 53 /163 | 22/105 | 31/31 | 59/70 |
| Gemini 3.1 Pro | 816/1316 | 209/408 | 25/163 | 25/105 | 31/31 | 41/70 |
| Generator | Ctx | Content | Layout | Style | Likeness | Spec. | Viol. | |
|---|---|---|---|---|---|---|---|---|
| gpt-6-astra | card | 47 | 100% | 98% | 98% | 9.9 | 7.5 | 1.8 |
| name | 47 | 100% | 99% | 96% | 9.8 | 7.3 | 2.0 | |
| gpt-6.1-sol | card | 47 | 100% | 99% | 97% | 9.9 | 7.4 | 1.8 |
| name | 47 | 100% | 99% | 97% | 9.9 | 7.3 | 1.9 | |
| Gemini 4 Argon | card | 47 | 98% | 89% | 95% | 9.4 | 7.4 | 1.9 |
| name | 47 | 98% | 90% | 95% | 9.4 | 7.3 | 2.0 |
| iOS 18 | iOS 26 | iOS 27 | ColorOS 15 | ColorOS 16 | ColorOS 17 | |
| Brand among eight | ||||||
| Gemini 3.8 Flash | 30/30 | 30/30 | 30/30 | 4/9 | 13/16 | 5/24 |
| Gemini 3.1 Pro | 30/30 | 30/30 | 30/30 | 3/9 | 5/16 | 10/24 |
| gpt-6-astra | 30/30 | 30/30 | 30/30 | 1/9 | 7/16 | 5/24 |
| Qwen3.5-9B base | 29/29 | 25/25 | 30/30 | 1/9 | 0/15 | 2/23 |
| ProofJudge v1 (earlier) | 30/30 | 29/30 | 28/30 | 1/9 | 1/16 | 1/24 |
| Judge context | Name-only pages | Card pages | Difference |
| Name only | 5.88 | 6.29 | |
| Name + card | 6.17 | 7.00 | |
| By brand, name-only pages card pages (name-only judge / card-aware judge) | |||
| iOS 26 | 6.25 5.75 / 6.88 7.50 | ||
| Material 3 Expressive | 7.25 8.00 / 7.38 7.88 | ||
| ColorOS 17 | 4.12 5.12 / 4.25 5.62 | ||
| Judge | Recall | FA | Prec. | F1 | Acc. | No ans. |
| API judges | ||||||
| Gemini 4 Argon | 96% | 28% | 77% | 0.86 | 84% | 0 |
| Gemini 3.8 Flash | 99% | 64% | 61% | 0.75 | 68% | 0 |
| gpt-6.1-sol | 100% | 63% | 61% | 0.76 | 68% | 0 |
| Claude Opus 5.5 (medium) | 99% | 53% | 65% | 0.79 | 73% | 0 |
| ProofJudge (ours) | ||||||
| Judge | Colour | Material | General | Radius | Foreign comp. | Icon, visible | Icon, looks normal |
|---|---|---|---|---|---|---|---|
| Gemini 4 Argon | 50% | 45% | 38% | 53% | 43% | 26% | 0% |
| gpt-6.1-sol | 40% | 54% | 45% | 47% | 30% | 21% | 0% |
| Gemini 3.8 Flash | 46% | 33% | 40% | 12% | 24% | 21% | 0% |
| GPT-6 Luna | 32% | 27% | 31% | 21% | 13% | 5% | 0% |
| Claude Opus 5.5 (medium) | 65% | 71% | 49% | 66% | 48% | 32% | 7% |
| ProofJudge (ours) | 41% | 44% | 59% | 41% | 20% | 16% | 0% |
| Judge | All icons | Visible | Brand only |
|---|---|---|---|
| Gemini 4 Argon | 8% | 26% | 0% |
| gpt-6.1-sol | 6% | 21% | 0% |
| Gemini 3.8 Flash | 6% | 21% | 0% |
| GPT-6 Luna | 2% | 5% | 0% |
| ProofJudge (ours) | 5% | 16% | 0% |
| Qwen3.5-9B base ∗ | 35% | 16% | 44% |
| Consistency F1 | Defect v2 F1 | |||
|---|---|---|---|---|
| Judge | IoU .5 | IoU .3 | IoU .5 | IoU .3 |
| Gemini 4 Argon | 0.37 | 0.43 | 0.17 | 0.21 |
| gpt-6.1-sol | 0.30 | 0.36 | 0.10 | 0.12 |
| Gemini 3.8 Flash | 0.26 | 0.29 | 0.09 | 0.12 |
| ProofJudge (ours) | 0.48 | 0.50 | – | – |
| ProofJudge v5 (final) | – | – | 0.19 | 0.20 |
| Judge | Effort | FA real | FA gen. | Swap | Local | F1 | s/page |
| Gemini 4 Argon | low | 10% | 20% | 5% | 27% | 0.24 | 4 |
| default | 15% | 30% | 50% | 33% | 0.33 | – | |
| default, rerun | 5% | 30% | 40% | 25% | 0.25 | 5 | |
| high | 15% | 30% | 5% | 29% | 0.22 | 19 | |
| gpt-6.1-sol | low | 40% | 27% | 10% | 33% | 0.24 | 14 |
| medium | 35% | 43% | 20% | 29% | 0.19 | – |
| Model | Method | False alarms | General recall | Brand recall | Defect F1 | General text |
|---|---|---|---|---|---|---|
| Qwen3.5-9B base | no training | 88% | 20% | 26% | 0.06 | 90% |
| ProofJudge v5 | LoRA r16 (SFT) | 37% | 57% | 6% | 0.18 | 87% |
| ProofJudge v5 | LoRA r64 | 62% | 63% | 12% | 0.21 | 90% |
| ProofJudge v5 | full FT (LM) | 65% | 59% | 12% | 0.21 | 74% |
| ProofJudge v5 | full FT (LM + vision) | 64% | 64% | 10% | 0.19 | 71% |
| ProofJudge v5 | LoRA r16 + GRPO 150 | 56% | 58% | 10% | 0.19 | 90% |
| Critic | Clean false alarms | General recall | Brand recall | Precision | F1 | Boxes / page |
|---|---|---|---|---|---|---|
| Gemini 4 Argon | 54% | 40% | 21% | 13% | 0.17 | 2.0 |
| Gemini 3.8 Flash | 89% | 32% | 15% | 6% | 0.09 | 3.3 |
| gpt-6-astra | 80% | 47% | 26% | 6% | 0.10 | 5.1 |
| gpt-6.1-sol | 75% | 50% | 27% | 6% | 0.10 | 6.0 |
| Gemini 3.1 Pro | 60% | 25% | 16% | 4% | 0.07 | 4.3 |
| Qwen3.5-9B base | 88% | 20% | 26% | 4% | 0.06 | 7.5 |
| FA real | FA gen. | Swap | Local | Radius | F1 | |
| Gemini 4 Argon | 15% | 20% | 36% | 32% | 83% | 0.31 |
| gpt-6.1-sol | 36% | 41% | 24% | 28% | 83% | 0.20 |
| v6-a, step 750 | 4% | 49% | 16% | 5% | 4% | 0.22 |
| v6-b, step 950 (chosen) | 4% | 20% | 8% | 7% | 4% | 0.21 |
| v6-b, step 1300 | 9% | 72% | 19% | 13% | 12% | 0.24 |
| v6-c, step 2229 | 13% | 90% | 19% | 21% | 50% | 0.25 |
| Hint | Fixed | Radius | Colour | Material | Harm, faulty | Harm, clean | Sent |
| None | 95% | 100% | 69% | 100% | 0% | 12% | 100% |
| True finding | 95% | 100% | 77% | 98% | 0% | 0% | 100% |
| Gemini 4 Argon findings | 55% | 79% | 38% | 48% | 11% | 15% | 100% |
| ProofJudge (ours) findings | 25% | 29% | 8% | 27% | 1% | 0% | 100% |
| ProofJudge r16 + cut (earlier) findings | 16% | 8% | 8% | 23% | 0% | 2% | 100% |
| ProofJudge v7b (earlier) findings | 42% | 67% | 8% | 40% | 6% | 8% | 100% |
Fault Detection
Fault detection identifies abnormal behavior in machines, sensors, infrastructure, software, and autonomous systems before it causes harm or disruption.
See all 115 papers