End-users seldom read verbose privacy policies, leading app stores like Google Play to mandate simplified data safety declarations as a user-friendly alternative. However, these self-declared disclosures often contradict the full privacy policies, deceiving users about actual data practices and violating regulatory requirements for consistency. To address this, we introduce PrivPRISM, a robust framework that combines encoder and decoder language models to systematically extract and compare fine-grained data practices from privacy policies and to compare against data safety declarations, enabling scalable detection of non-compliance. Evaluating 7,770 popular mobile games uncovers discrepancies in nearly 53% of cases, rising to 61% among 1,711 widely used generic apps. Additionally, static code analysis reveals possible under-disclosures, with privacy policies disclosing just 66.8% of potential accesses to sensitive data like location and financial information, versus only 36.4% in data safety declarations of mobile games. Our findings expose systemic issues, including widespread reuse of generic privacy policies, vague / contradictory statements, and hidden risks in high-profile apps with 100M+ downloads, underscoring the urgent need for automated enforcement to protect platform integrity and for end-users to be vigilant about sensitive data they disclose via popular apps.
Figures & tables
Figure 1: Top: Data safety declarations of three games by Spider Solitaire Card Games and bottom: corresponding privacy policy highlighting inconsistencies. Data safety labels only report collecting and sharing device identifiers for analytics and ads, yet their privacy policy omit ad-related details and instead disclose access to more sensitive data such as GPS location, user accounts, and app activity. Identified via PrivPRISM framework deployment to mobile games.
Figure 2: Terminology used in PrivPRISM
Figure 3: End-to-end pipeline of our framework (top) and a walkthrough example (bottom)
Ours (EDPC)
F.T. Llama3.1
Z.S.GPT4o
Class
Pr
Re
Pr
Re
Pr
Re
c0
0.94
0.74
0.90
0.21
0.88
0.69
c1
0.96
0.66
0.88
0.25
0.85
0.68
c4
0.73
0.41
0.51
0.19
0.87
0.54
μcˉ
0.91
0.56
0.41
0.24
0.85
0.54
c0 -first party collection, c1 -third party sharing
Table 1: Benchmarked test-set results for OPP-115 dataset conducted for our Explained Data Practice Classification (EDPC) module compared against fine-tuned Llama3.1 and zero-shot prompted GPT4o; both performing as explainable classifiers.
Figure 4: Data item mapping verifier embedding alignment and transferability
Downloads:
∼5M+
∼1−5M
Collect
Share
Collect
Share
PP Compliance
81.23
68.08
83.28
69.17
DS Compliance
25.44
22.58
21.29
18.47
Table 2: Overall Data Practice Compliance (%) Landscape
Figure 5: Structural composition of a privacy policy - A column represents the data practice category distribution of the policy dataset at a given normalised length.
p[0]
p[1]
p[2]
p[3]
p[4]
p[5]
p[6]
d[0]
0.71
4.00
0.78
1.40
7.50
33.06
20.41
d[1]
2.63
17.09
6.76
1.96
1.55
50.00
11.81
d[2]
20.00
7.49
14.66
5.71
12.88
43.48
35.10
d[4]
0.00
7.69
0.00
0.00
0.00
40.00
7.14
d[8]
16.55
1.45
34.39
6.57
6.37
1.76
25.76
d[9]
26.46
2.05
7.11
7.83
9.33
1.95
17.19
Table 3: Data purpose compliance (%): Each row represents kth purpose compliance score for a given valid data item category j .
Figure 6: Four types of moving average filtered data item compliance scores based on the app popularity. Moving average window = 50, Shade = 0.5*STD
Figure 7: Number of data item declarations mentioned in PP, DS and within APK evidence for 4,538 game apps.
Figure 8: Data item collection or sharing frequency for game and non-game apps if mentioned in privacy policy (PP) or data safety (DS). X axis is normalised for easier comparison; i.e., the x-axis is normalised to show how many data items are observed per 1,000 privacy policies.
Figure 9: APK evidence analysis for non-game apps. Note: the x-axes are normalised for easier observations, therefore are in the same scale.
Figure 10: Privacy Policy and Data Safety collection and sharing compliance scores with respect to the corresponding non-game app’s category. Note that several categories contained less than 50 apps when we selected the most popular non-game app corpus. Data points: mean compliance per category, Shade: 0.5*STD
Figure 11: (a): A case study of ‘FARM STUDIO’, (b,c): Existing policy links but invalid contents. Validated 2025.12
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: Combinations of Google Data Safety declaration labels, compared with data practice, data item and data purpose categories we utilise in PrivPRISM framework.
PrivPRISM’s EDPC
F.T. Llama3.1
Z.S. GPT4o
Class
Pr
Re
F1
Pr
Re
F1
Pr
Re
F1
Support
c0 First Party Collection / Use
0.94
0.74
0.83
0.90
0.21
0.34
0.88
0.69
0.78
289
c1 Third Party Sharing / Collection
0.96
0.66
0.78
0.88
0.25
0.38
0.85
0.68
0.76
204
c2 User Choice / Control
0.95
0.46
0.62
0.65
0.19
0.30
0.87
0.54
0.67
115
c3 User Access, Edit and Deletion
0.94
0.37
0.53
0.60
0.15
0.24
0.72
0.63
0.68
41
c4 Introductory / Generic
0.73
0.41
0.52
0.51
0.25
0.33
0.94
0.25
0.39
118
Appendix
Table 4: Data Practice Classification Results. EDPC: Explained Data Practice Classifier, F.T.:Fine-tuned, Z.S.:Zero Shot
Trained on: GPT
Trained on: GPT
Trained on: Llama
Trained on: Llama
Tested on: GPT
Tested on: Llama
Tested on: Llama
Tested on: GPT
Class
P
R
F1
P
R
F1
P
R
F1
P
R
F1
Support
Name
0.97
0.92
0.94
0.98
0.95
0.96
0.97
0.93
0.95
0.96
0.92
0.94
199 149*
Email
0.99
0.98
0.99
0.99
0.99
0.99
0.97
0.98
0.97
0.98
0.99
0.98
347 295*
User Acct
0.95
0.80
0.87
0.96
0.67
0.79
0.84
0.69
0.75
0.93
0.83
0.88
475 436*
Financial
0.95
0.98
0.96
0.94
0.91
0.92
0.94
0.85
0.89
0.94
0.99
0.97
270 205*
Appendix
Table 5: Transferability of self-supervised mapping verifier training
Class
Pr
Re
F1
Support
0 - Analytics
0.96
0.85
0.91
627
1 - Dev. Commu.
0.80
0.88
0.84
320
2 - Fraud Prevention
0.95
0.95
0.95
1530
3 - Advertising
0.97
0.95
0.96
590
4 - Personalisation
0.86
0.95
0.90
796
5 - Account Manage
0.96
0.88
0.92
847
Appendix
Table 6: Alignment of 110M encoder model against the ground truth assumption from a 8B decoder model
Figure 14: An example output from our framework for a given PP, DS pair with 0% overall compliance score. Note: For output visualisation only - a more readable version is discussed in the main text, Fig. 1 .