Autoregressive (AR) vision-language models (VLMs) have long dominated multimodal understanding, reasoning, and graphical user interface (GUI) grounding. Recently, discrete diffusion vision-language models (DVLMs) have shown strong performance in multimodal reasoning, offering bidirectional attention, parallel token generation, and iterative refinement. However, their potential for GUI grounding remains unexplored. In this work, we evaluate whether discrete DVLMs can serve as a viable alternative to AR models for GUI grounding. We adapt LLaDA-V for single-turn action and bounding-box prediction, framing the task as text generation from multimodal input. To better capture the hierarchical structure of bounding-box geometry, we propose a hybrid masking schedule that combines linear and deterministic masking, improving grounding accuracy by up to 6.1 points in Step Success Rate (SSR) over the GUI-adapted LLaDA-V trained with linear masking. Evaluations on four datasets spanning web, desktop, and mobile interfaces show that the adapted diffusion model with hybrid masking consistently outperforms the linear-masked variant and performs competitively with autoregressive counterparts despite limited pretraining. Systematic ablations reveal that increasing diffusion steps, generation length, and block length improves accuracy but also increases latency, with accuracy plateauing beyond a certain number of diffusion steps. Expanding the training data with diverse GUI domains further reduces latency by about 1.3 seconds and improves grounding accuracy by an average of 20 points across benchmarks. These results demonstrate that discrete DVLMs are a promising modeling framework for GUI grounding and represent an important step toward diffusion-based GUI agents.
Figures & tables
Figure 1: Overview of Hybrid Masking Adaptation of LLaDA-V for GUI Grounding. (a) The adapted framework takes a natural-language instruction and a GUI screenshot (either from web, desktop, or mobile interfaces) as input. LLaDA-V trained with the linear masking predicts the action type, optional type_in text, and anchor coordinates (x1,y1) . LLaDA-V model trained with the full deterministic masking predicts the remaining bounding-box coordinates (x2,y2) conditioned on the image, instruction, and anchor. (b) Linear Masking Phase: action and anchor tokens in the response are randomly masked according to the linear masking during the forward corruption process during training for coarse grounding during training. (c) Deterministic Masking Phase: all response tokens are fully masked in the forward corruption process during training, and LLaDA-V predicts the bounding-box extent during denoising.
Figure 2: The above figure shows an instance where LLaDA-V 8B trained with linear and full deterministic masking provides a more accurate target bounding box and action prediction compared to the model trained with linear masking. The green bounding box is the ground truth and the red one is the predicted. Both models receive image and natural language instruction and produce action type and target bounding as text, visualised on the GUI image.
UI Domain
Dataset
Percentage
Samples
Web
Mind2Web
16.67
20K
WebLinX
16.67
20K
OS-Atlas
16.67
20K
Total
50.00
60K
Mobile
OS-Atlas
16.67
20K
Rico Widget Caption
16.67
20K
Table 1: Training data composition for data scaling experiments. The dataset spans web, mobile, and desktop domains, totaling 120K samples.
Inference Parameters
Accuracy
Avg Lat
Diff. Steps
Gen Len
Block Len
Conv Steps
SSR (%)
F1 (%)
sec
32
32
32
13
78.15
99.00
2.56
64
64
64
25
80.67
99.00
4.84
128
128
128
25
80.63
99.87
5.01
Table 2: LLaDA-V 8B fine-tuned only on the Mind2Web training set (7k samples) without cropping and OCR-based target annotation, trained for 10 epochs. The fine-tuned model was evaluated on the Mind2Web test set.
Figure 3: GUI Data Scaling behavior of LLaDA-V 8B trained with Linear Masking: Comparison between LLaDA-V 8B trained on 7k web GUI samples from Mind2Web and 120k mobile, web and desktop GUI samples across four GUI grounding datasets. M2W: Mind2Web, SWT: ScreenSpot-Web-Text, SWI: ScreenSpot-Web-Icon, VWA: Visual Web Arena. Left plot shows Step Sucess Rate (SSR), the center plot shows the number of Converged Steps, and right shows average Inference Latency measured in seconds. Training with large-scale GUI multi-domain data improves SSR, reduces the number of Converged Steps required to produce a highly confident output while reducing Inference Latency, demonstrating better generalization and efficiency across GUI domains.
Inference Parameters
Accuracy
Avg Lat
Diff Steps
Gen Len
Block Len
Conv Steps
SSR (%)
F1 (%)
sec
64
64
64
25
80.67
99
4.84
64
64
64
25
83.31
99
4.46
Table 3: LLaDA-V 8B fine-tuned only on the Mind2Web training set (7K samples) without and with cropping and OCR-based target annotation, trained for 10 epochs. The highlighted row shows the model trained with random cropping and OCR-based target annotation. Both models were tested on the Mind2Web test set.
Dataset
Metric
Phi (3B)
Qwen 2.5 VL (3B)
Qwen 2.5 VL (7B)
LLaDA-V 8B (Lin)
LLaDA-V 8B (Ours)
M2W
Conv Steps
21.00
21.00
21.00
16.00
23.00
SSR (%)
56.80
79.30
81.90
82.40
83.90
F1 (%)
94.40
99.60
99.90
98.50
100.00
Lat. (s)
–
–
1.10
3.02
5.44
SWI
Conv Steps
23.00
23.00
23.00
18.00
23.00
SSR (%)
62.60
79.10
85.40
57.80
63.10
Table 4: AR vs. NAR GUI grounding comparison. The table reports GUI grounding performance on four benchmarks across the listed evaluation metrics. The rightmost column (in grey) highlights our proposed LLaDA-V variant trained with the hybrid linear and deterministic masking schedule.
Inference Parameters
Accuracy
Diff. Steps
Gen Len
Block Len
SSR (%)
F1 (%)
32
32
32
0.00
0.10
64
64
64
0.00
0.12
128
128
128
0.00
0.10
256
64
64
0.00
0.10
Table 5: Zero Shot performance of LLaDA-V 8B on the Mind2Web test set
Inference Parameters
Accuracy
Avg Lat
Diff. Steps
Gen Len
Block Len
Conv Steps
SSR (%)
F1 (%)
sec
32
32
32
13
78.15
99.00
2.56
64
64
64
25
80.67
99.00
4.84
128
128
128
25
80.63
99.87
5.01
256
64
64
25
80.69
99.87
4.84
Table 6: LLaDA-V 8B fine-tuned only on the Mind2Web training set (7k samples) without cropping and OCR based annotation. The table shows the effect of inference parameters on Avg Latency and Accuracy.
Dataset
Metric
LLaDA-V 8B (Lin)
LLaDA-V 8B (Ours)
M2W
Conv Steps
16.00
23.00
11.00
SSR (%)
82.40
83.90
81.00
F1 (%)
98.50
100.00
96.66
Lat. (s)
3.02
5.44
2.74
SWI
Conv Steps
18.00
23.00
11.00
SSR (%)
57.80
63.10
59.60
Table 7: Accuracy–Latency Trade-Off with Hybrid Masking. The table compares LLaDA-V trained with default linear masking (third column) and our hybrid masking (fourth and fifth columns) across four benchmarks. The hybrid model achieves higher SSR but with slightly higher latency, while reducing diffusion steps lowers latency with minimal loss in accuracy.
Inference Parameters
Accuracy
Latency (seconds)
Diff. Steps
Gen Len
Block Len
Conv Steps
SSR (%)
F1 (%)
Lowest
Highest
Avg
8
64
32
4
71.65
99.00
0.99
2.14
1.27
16
64
32
8
74.82
99.00
1.14
2.76
1.47
32
32
32
13
82.50
99.00
2.24
3.43
2.56
64
64
64
25
83.31
99.00
3.72
10.35
4.46
128
128
128
25
82.92
99.00
4.01
28.13
5.02
Table 8: LLaDA-V 8B inference performance across different inference settings. The model was fine-tuned on the Mind2Web training set (7k samples) with cropping and OCR-based target annotation, trained for 10 epochs.
Figure 4: Effect of data annotation quality on grounding accuracy. The figure compares predictions from LLaDA-V trained with default linear masking using the Mind2Web train split with and without cropped images and OCR-based annotations. The top example, trained without OCR text-based annotations and cropping, produces an inaccurate bounding box due to inconsistent icon-level targets and high-resolution inputs. The bottom example, trained with cropped images and OCR-guided text annotations, provides more stable supervision, allowing the model to correctly localize the target element. The green bounding box is the ground truth and the red one is the prediction.
Figure 5: Effect of hybrid masking on bounding-box accuracy. The figure compares predictions from LLaDA-V trained with default linear masking (top) and with the proposed hybrid masking that combines linear and deterministic full masking (bottom). The linear-masked model correctly predicts the action type but generates an inaccurate bounding box, missing the target region. In contrast, the hybrid-masked model, guided by conditional refinement between anchor and extent coordinates, produces a precise bounding box that accurately localizes the target element. The green bounding box is the ground truth and the red one is the prediction.