The regulation of Artificial Intelligence is susceptible to various forms of capture by industry actors that hold undue influence over the policymaking process, due in part to their information advantage in developing these technologies. In the following paper, we investigate the roles of public commenters in providing evidence to potential regulation in the USA. We build a corpus of 40 potential regulations, and 2,664 public comments submitted by a variety of stakeholders. We computationally process these texts to extract evidence meant to inform regulators, in the form of formal citations and informal references. We find significant differences in the type and amount of evidence supplied: Holding comment length constant, business commenters introduce altogether less evidence, and government, university, and civil society commenters provide significantly more research and legal sources. The publication status of cited research is nearly evenly split, with preprints and peer-reviewed articles being introduced by commenters of all types. We further explore patterns of self-citation and find that while it is prevalent amongst all groups, a small cluster of organizations account for over half of all self-citations in our corpus. Our findings complicate a narrative that public comments alone are means to information capture. We interpret these results in terms of the role that commenters attempt to play in the policymaking process-being either an Issue Advocate or an Honest Broker. These findings on the supply of evidence in public comments shed light on an often overlooked form of policymaking that has implications for how and through what sources information is made accessible to regulators of AI.
Figures & tables
Label
Precision
Recall
F1
citation_sentence
0.819
0.896
0.856
citation_link
0.800
0.822
0.811
citation_footnote/reference
0.810
0.811
0.810
author_name
0.894
0.876
0.885
author_affiliation
0.758
0.649
0.699
Table 1. Classification performance by label.
Figure 1. Distribution of evidence cited, by commenter type
Figure 2. Rate ratios of expected evidence type cited relative to business/industry commenters
Predictor
b
SE
z
OR
95% CI
p
Intercept ( Business/Industry )
0.041
0.172
0.24
1.04
[0.74, 1.46]
.812
Government
1.024
0.479
2.14
× 2.78
[1.09, 7.12]
.033
Civil Society Org.
0.234
0.224
1.05
× 1.26
[0.82, 1.96]
.295
Trade/Coalition Org.
0.371
0.215
1.73
× 1.45
[0.95, 2.21]
.084
University
0.695
0.305
2.28
× 2.00
[1.10, 3.64]
.023
Table 3. Mixed-effects logistic regression of self-citation, by commenter type
Contrast
b
SE
z
Ratio
95% CI
p
Business/industry vs.
Government
−0.766
0.260
−2.94
0.47
[0.22, 0.98]
.039
Individual
0.479
0.126
3.78
1.61
[1.13, 2.31]
.002
Civil society organization
−0.842
0.122
−6.89
0.43
[0.30, 0.61]
< .001
Trade/coalition
−0.509
0.114
−4.45
0.60
[0.43, 0.83]
< .001
University
−0.877
0.165
−5.31
0.42
[0.26, 0.67]
< .001
Table 4. Pairwise comparisons of expected citation counts by commenter type
Figure 3. Citation of Peer-reviewed Research vs Preprints
Commenter type
Works
Self- citations
Self-citation rate
Preprint self-citations
% to preprints (95% CI)
Business/industry
748
117
15.6%
75
64.1% (55.1, 72.2)
Government
173
27
15.6%
5
18.5% (8.2, 36.7)
Civil Society
1,302
179
13.7%
87
48.6% (41.4, 55.9)
Trade/coalition organization
443
7
1.6%
2
28.6% (8.2, 64.1)
University
1,146
462
40.3%
178
38.5% (34.2, 43.0)
Table 5. Self-citation and preprint self-citation by commenter type
Figure 4. Identified Comment Duplicates
Label
Ground Truth
Model’s Annotation
citation_sentence
A dynamic, risk-based approach to AI regulation is necessary (National Institute of Standards and Technology [NIST], 2023).
I. Dynamic, Risk-Based Approach: A dynamic, risk-based approach to AI regulation is necessary (National Institute of Standards and Technology [NIST], 2023).
National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0) . https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0) . https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
Artificial intelligence governance studies how laws, institutions, and technical controls shape AI development, deployment, accountability, and social effects.