Computer-use agents learn to operate mobile interfaces from corpora of captured screens and accessibility trees. The same trees are what a screen reader consumes, so an agent trained on them inherits whatever labelling gaps the underlying applications carry. This paper measures one of those gaps, content-description coverage, in two Android corpora: Rico, collected in 2017, and AndroidControl, collected in 2024. Across 17,373,614 interface elements, 17.74% of the Rico elements that need a content description under our coding rule carry one, against 57.27% in AndroidControl. The difference of 39.5 percentage points holds under five alternative coding rules and when whole apps are resampled. It describes two datasets that also differ in app selection, capture method and platform version; it does not measure change over time. Input controls are the least labelled role in both corpora (2.20% and 27.21%, or 30.99% in AndroidControl if hint text counts as a name). Descriptions in the older corpus are also five times more likely to be placeholders (10.25% against 1.95%), so the difference is not an artefact of developers typing filler. Resolving 26,205 task actions to the elements they touched shows that, among tappable graphics with no text, demonstrations operate elements described 9.0 points more often than the others on the screens they visit, about 3.5 points of it from the choice of element within a screen. That is a selection effect inside the benchmark. A two-coder reliability check (kappa = 0.91 and 0.96) and the full pipeline accompany the paper.
Figures & tables
Rico 2017
AndroidControl 2024
Screens or step observations
66,261
49,822
Episodes
—
7,623
Elements parsed
6,371,389
11,002,225
In analysis population
2,862,639
6,831,063
Needs a label
502,585
1,462,544
Labelled
89,146
837,536
TABLE I: Analysis population and label coverage by corpus.
Fig. 1: The label-status decision. Questions 2 and 3 route exempt elements past the debt test to the final split. The shaded outcome, unlabelled , is the accessibility debt.
Fig. 2: Label coverage across the two corpora. (a) Overall coverage among elements requiring a description. (b) Coverage by element role; exempt roles are omitted. (c) Within AndroidControl, elements touched by a task demonstration against the background population on the same screens. Hatching duplicates the colour encoding so the figure reads in greyscale.
Role
Rico
AndroidControl
Diff
h
(%)
(%)
(pp)
button
45.28
72.26
+26.98
0.56
image
9.62
58.42
+48.81
1.11
input
2.20
27.21
+25.01
0.80
text
4.09
45.52
+41.43
1.07
other
26.63
38.91
+12.28
0.26
TABLE II: Label coverage by element role and the cross-corpus difference.
Population
Touched
Backgr.
Diff
Screen-cluster
Within
(%)
(%)
(pp)
95% CI
(pp)
All needing a label
54.14
56.88
−2.74
[−3.95,−1.47]
−0.80
Clickable only
55.90
52.59
+3.31
[2.15,4.51]
+1.99
Icon-like only
69.96
60.97
+8.99
[7.47,10.51]
+3.52
TABLE III: Label coverage of touched versus background elements on the same screens.
Density and n from
N
IRR [95% CI]
1
All screens
7,615
0.818[0.783,0.855]
2
All screens; split, clustered SE
7,615
0.819[0.740,0.906]
3
First in-app screen; split, cl. SE
7,407
0.986[0.895,1.086]
TABLE IV: Incidence rate ratio for label density in three models of demonstration length.
Split
Episodes
Coverage (%)
Mean steps
In distribution
364
55.17
5.37
Unseen category
349
56.20
5.77
Unseen app
318
54.78
5.74
Unseen task
401
56.26
5.72
TABLE V: Label density and mean demonstration length across AndroidControl’s generalisation splits.
Specification
Rico
AC
Diff
h
(%)
(%)
(pp)
A
Primary
17.74
57.27
+39.53
0.85
B
Exclude disabled elements
17.77
57.30
+39.54
0.85
C
Icon-like only
33.20
62.75
+29.56
0.60
D
Rico via superclass chain
19.36
57.27
+37.91
0.81
E
Uninformative labels absent
15.92
56.15
+40.23
0.87
TABLE VI: Cross-corpus coverage difference under five specifications.