Computational resources are increasingly central to NLP research, but how closely reported GPU capability aligns with scholarly impact remains unclear. We analyze 13,921 ACL, EMNLP, and NAACL main-conference papers published between 2020 and 2025, using GPU resources as our operational measure of computational resources. From full texts, we extract GPU models and counts, standardize each paper's largest reported configuration into a comparable hardware-capability measure, and link these data to citation, award, topic, and institutional metadata. GPU reporting became more common but remained incomplete, while reported capability increased mainly through newer hardware generations and medium-scale multi-GPU configurations. Resource concentration substantially exceeded impact concentration: the annual top 20% of GPU-quantifiable papers accounted for 83.9%-89.9% of reported GPU capability, but only 27%-32% of citations and 20%-33% of paper awards. In adjusted models, a tenfold increase in aggregate reported GPU capability was associated with a 3.52-percentage-point increase in within-NLP topic-year citation percentile, but increased model R^2 by only 0.0042. GPU count showed more consistent positive associations with citation and award outcomes than newer hardware generation. Overall, reported GPU resources are associated with scholarly impact but provide little standalone explanation of research influence.
Figures & tables
Figure 1: Estimated training compute for selected milestone language models has increased substantially over time, alongside advances in accelerator hardware.
Figure 2: Overview of our computational resource measurement and analysis framework. We collect ACL, EMNLP, and NAACL papers from the ACL Anthology and enrich them with OpenAlex metadata, extract reported GPU models and quantities with human validation and LLM-based extraction, normalize raw hardware mentions to canonical GPU attributes, and use the resulting corpus to analyze reporting trends, reported compute scale, concentration across topics, organizations, and regions, and scholarly impact.
Group
#Papers
Definition
Example
Model-reported
6,900 (49.6%)
At least one reported GPU model can be normalized to hardware specifications
RTX 4090
Strict
5,360 (38.5%)
Both a standardizable GPU model and an explicit device quantity are observed
8×A100 GPU
Table 1: Dataset statistics used in this study.
Figure 3: Reporting completeness of GPU resources in NLP papers.
Figure 4: Evolution of reported GPU scale, generation, and capacity in NLP research. The left panel shows GPU count over time; the middle panel shows the share of GPU generations; the right panel shows normalized reported GPU capacity distributions and annual 95th percentile (P95) values.
Figure 5: Reported GPU capacity by institutional access regime. Panel a shows median reported peak GPU configuration capacity across institutional access regimes. Points denote medians, horizontal intervals show the interquartile range, and sample sizes are reported on the right. Panel b shows share of papers in the yearly top-20% reported GPU capacity tail. The dashed vertical line marks the 20% reference level.
Figure 6: Reported GPU hardware capacity across major NLP topics: median, interquartile range (IQR), and the 90th percentile (P90). The full results for all 29 topic categories are reported in .
Figure 7: Concentration of reported GPU capability and scholarly impact. Panel (a) shows annual shares of total reported GPU capability, citations, and awards accounted for by papers in the top 20% of reported GPU capability. Panel (b) shows overlap between high-capability and high-impact papers. High-capability papers are defined as the top 20% by reported GPU capability, and high-impact papers as the top 10% by citations within publication-year-by-venue groups.
GPU-capability cutoff
Citation-impact cutoff
Top 5%
Top 10%
Top 20%
Top 10%
2.10×
1.60×
1.49×
Top 20%
2.14×
1.59×
1.53×
Top 30%
1.93×
1.55×
1.49×
Table 2: Robustness to alternative thresholds. Each entry reports the ratio of high-impact rates between papers above and below the GPU-capability cutoff. Values above one indicate higher citation impact among high-capability papers.
Outcome
Aggregate GPU capability
GPU count
Ampere or newer
ΔR2 aggregate / joint
NLP topic-year percentile (primary)
+3.52∗∗ pp
+4.57∗∗∗ pp
+4.16∗∗ pp
0.0042/0.0093
OpenAlex percentile (secondary)
+1.26 pp
+1.96 pp
+2.02 pp
0.0009/0.0033
log(1+citations) , OLS
+18.8∗∗∗%
+24.7∗∗∗%
+15.0∗%
0.0052/0.0086
Citation count, PPML
+61.2∗∗∗%
+69.3∗∗∗%
+31.0∗%
–
Top-10% cited, LPM
+3.84∗∗ pp
+4.41∗∗ pp
+2.17 pp
0.0043/0.0051
Awarded, LPM
+0.86 pp
+1.33∗ pp
−0.25 pp
0.0011/0.0018
Table 3: Adjusted associations between reported GPU resources and scholarly impact.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Type of dataset
Role in analysis
Paper corpus
Defines the 13,921 ACL/EMNLP/NAACL main conference papers.
Parsed paper text
Source for GPU hardware mentions and evidence spans.
OpenAlex metadata
Provides citation counts, author/team size, and citation percentile measures.
Organization variables
Capture industry participation, industry-academia collaboration, cross-sector collaboration, international collaboration, and organization counts.
Topic labels
Provide a 29-topic taxonomy based on title and abstract classification.
Award labels
Identify best paper and related award indicators for 2020–2025 main conference papers.
Appendix
Table 4: Dataset components.
Case type
Example pattern
Annotation decision
Complete GPU reporting
trained on 8 NVIDIA A100 80 GB GPUs
Annotate GPU model, memory, and quantity; eligible for paper-level capacity estimation.
Model-only reporting
experiments ran on A100 GPUs
Annotate GPU model; quantity is null , and lower-bound analyses assign one GPU.
Ambiguous hardware reporting
experiments were run on GPU servers
Not standardizable to a specific GPU model; excluded from capacity estimation.
Citation or prior-work hardware
Brown et al. used 1024 TPU v3 chips
Prior-work hardware; not annotated as this paper’s hardware.
Model/API/software names
we use GPT-4o, Claude, Gemini, and vLLM
Model/API/software names are not hardware; not annotated.
Appendix
Table 5: Boundary cases in computational resource annotation.
Model
Matching criterion
Precision
Recall
F1
deepseek-v3.2
Name only
0.918±0.0028
0.948±0.0013
0.933±0.0019
Exact match
0.865±0.0021
0.894±0.0024
0.879±0.0020
gpt-4o-mini
Name only
0.907±0.0099
0.842±0.0066
0.874±0.0069
Exact match
0.864±0.0130
0.802±0.0115
0.832±0.0115
gpt-5.4-mini
Name only
0.927±0.0092
0.933±0.0096
0.930±0.0092
Exact match
0.842±0.0126
0.848±0.0133
0.845±0.0128
Appendix
Table 6: LLM extraction performance on the human-validated set. Scores are reported as mean ± standard deviation. Bold values indicate the best score within each matching criterion.
Figure 8: Prompt used to extract computational resources.
Figure 9: Prompt used to classify NLP topics.
Reporting outcome
N
Event rate
LPM R2
Δ LPM R2
Logit AUC
Δ AUC
McFadden pseudo- R2
Δ pseudo- R2
Reports at least one standardizable GPU model
13,745
0.5011
0.0804
0.0009
0.6572
0.0014
0.0602
0.0007
Reports standardizable GPU model and quantity
13,745
0.3875
0.0910
0.0013
0.6768
0.0015
0.0735
0.0011
Appendix
Table 7: Incremental explanatory power of organizational variables for GPU reporting sample membership.
Predictor
Coef. (pp)
95% CI
p -value
Outcome: Reports at least one standardizable GPU model
Industry participation
-0.935
[-4.523, 2.654]
0.610
Industry–academia collaboration
1.938
[-2.287, 6.162]
0.369
Cross-sector collaboration
2.329
[-0.298, 4.957]
0.082
International collaboration
-1.595
[-3.627, 0.436]
0.124
log(1 + number of organizations)
-2.320
[-5.672, 1.031]
0.175
Appendix
Table 8: Full linear probability model coefficients for GPU reporting sample membership.
Audit outcome
Papers
Share
Consumption-related signal visible
92
38.3%
No consumption-related signal visible
148
61.7%
Total
240
100.0%
Appendix
Table 9: Manual audit of consumption-related reporting among 240 papers reporting GPU usage.
Figure 10: GPU hardware reporting completeness across venues.
Figure 11: Reported GPU count distributions over time.
Figure 12: Transition of leading reported GPU models in NLP papers.
Figure 13: Growth in reported GPU memory capacity in NLP papers.
Figure 14: Distribution of reported peak GPU configuration capacity in NLP papers.
Figure 15: Country-level concentration in high reported GPU compute.
Figure 16: Reported GPU hardware capacity across NLP research topics. Panel (a) reports the median, interquartile range, and P90 of paper-level peak reported GPU configuration capacity. Panel (b) reports the share of papers at or above the year-specific P90 capacity threshold. Because ties at the P90 threshold are retained, the overall share can exceed 10%; the dashed line indicates the empirical overall share of 19.5%.
Predictor
M1
M2
M3
M4
M5
M6
Industry
0.267∗∗∗
0.474∗∗∗
(0.016)
(0.042)
Industry-academia
0.195∗∗∗
−0.299∗∗∗
(0.017)
(0.047)
Cross-sector
0.160∗∗∗
0.057∗
(0.015)
(0.023)
Appendix
Table 10: Associations between institutional characteristics and reported GPU capacity.
Citation cutoff
Coef. (pp)
95% CI
p
ΔR2
Top 5%
+4.04
[1.81, 6.26]
<0.001
0.0085
Top 10%
+3.84
[1.14, 6.53]
0.005
0.0043
Top 20%
+7.20
[3.80, 10.59]
<0.001
0.0086
Appendix
Table 11: Sensitivity of the continuous reported GPU-capability model to alternative high-impact thresholds. Coefficients are percentage-point differences associated with a tenfold increase in reported GPU capability. All models include the same fixed effects and controls as the baseline specification.
Outcome
Estimator
N
β
SE
95% CI
p
Implied effect
ΔR2
NLP topic–year percentile (primary)
OLS
2,194
0.0352
0.0115
[0.0127, 0.0577]
0.002
+3.52 pp
0.0042
OpenAlex field-normalized percentile
OLS
2,194
0.0126
0.0088
[ −0.0046 , 0.0298]
0.151
+1.26 pp
0.0009
log(1+citations)
OLS
2,194
0.1719
0.0491
[0.0757, 0.2681]
<0.001
+18.8%
0.0052
Citation count
PPML
2,194
0.4775
0.0852
[0.3105, 0.6445]
<0.001
+61.2%
–
Top-10% cited
LPM
2,194
0.0384
0.0138
[0.0114, 0.0654]
0.005
+3.84 pp
0.0043
Awarded
LPM
5,357
0.0086
0.0045
[ −0.0002 , 0.0174]
0.056
+0.86 pp
0.0011
Appendix
Table 12: Full adjusted models relating aggregate reported GPU capability to scholarly impact. The implied effects correspond to a tenfold increase in aggregate reported GPU capability. For OLS log-citation and PPML models, percentage effects are calculated as 100×(eβ−1) . For percentile and linear-probability outcomes, effects are expressed in percentage points (pp). Incremental R2 is calculated against a controls-only model estimated on the same sample; it is not reported for PPML.
Outcome
Term
β
SE
95% CI
p
Implied effect
Joint ΔR2
NLP topic–year percentile
GPU count
0.0457
0.0133
[0.0196, 0.0718]
<0.001
+4.57 pp
0.0093
Newer generation
0.0416
0.0135
[0.0151, 0.0681]
0.002
+4.16 pp
OpenAlex field-normalized percentile
GPU count
0.0196
0.0101
[ −0.0002 , 0.0394]
0.052
+1.96 pp
0.0033
Newer generation
0.0202
0.0105
[ −0.0004 , 0.0408]
0.055
+2.02 pp
log(1+citations)
GPU count
0.2206
0.0571
[0.1087, 0.3325]
<0.001
+24.7%
0.0086
Newer generation
0.1396
0.0552
[0.0314, 0.2478]
0.011
+15.0%
Appendix
Table 13: Full joint models separating reported GPU count from hardware generation. GPU count and the Ampere-or-newer/equivalent indicator are entered simultaneously. GPU-count effects correspond to a tenfold increase in the reported count; newer-generation effects compare Ampere-or-newer/equivalent hardware with earlier generations. Citation models use N=2,194 and the award model uses N=5,357 . The joint incremental R2 is the increase from adding both hardware terms to the controls-only model estimated on the same sample.
Term
β (SE)
Profile 95% CI
Penalized LRT p
Holm p
OR [95% CI]
GPU count (tenfold increase)
0.5380 (0.1883)
[0.1730, 0.8907]
0.0043
0.0085
1.713 [1.189, 2.437]
Ampere-or-newer/equivalent
−0.1115 (0.2734)
[ −0.6322 , 0.4362]
0.6833
0.6833
0.894 [0.531, 1.547]
Appendix
Table 14: Firth penalized logistic regression of paper-award status on reported GPU count and hardware generation. The model uses the same 5,357 papers and 111 award-positive cases as the joint linear probability model. GPU count and the Ampere-or-newer/equivalent indicator are entered simultaneously with the baseline fixed effects and controls. The joint null that both hardware coefficients equal zero is rejected, χ2(2)=8.3178 , p=0.0156 .
Outcome
Specification
β (SE)
95% CI
p
R2 controls/full
ΔR2
NLP topic–year percentile
Common-sample baseline
0.0313 (0.0118)
[0.0081, 0.0545]
0.008
0.0895/0.0928
0.0032
+ pre-publication controls
0.0274 (0.0121)
[0.0037, 0.0512]
0.024
0.1330/0.1354
0.0023
+ public artifact
0.0265 (0.0121)
[0.0028, 0.0502]
0.028
0.1362/0.1384
0.0022
OpenAlex field-normalized percentile
Common-sample baseline
0.0116 (0.0091)
[ −0.0062 , 0.0295]
0.202
0.0909/0.0916
0.0008
+ pre-publication controls
0.0105 (0.0094)
[ −0.0080 , 0.0290]
0.264
0.1172/0.1178
0.0006
+ public artifact
0.0102 (0.0095)
[ −0.0084 , 0.0287]
0.282
0.1179/0.1185
0.0006
Appendix
Table 15: Robustness of aggregate reported GPU-capability estimates to expanded observable controls. All models use the same complete-case sample of 2,077 papers. The coefficient represents the association with a tenfold increase in aggregate reported GPU capability. Incremental R2 compares each full model with a model containing the same controls but excluding reported GPU capability.
Characteristic
Main Conference
Findings
Main + Findings
Total papers
13,921
9,917
23,838
Papers reporting standardized GPU model
6,900
5,824
12,724
Reporting rate (%)
49.6
58.7
53.4
Papers reporting GPU model + count
5,360
4,186
9,546
Strict reporting rate (%)
38.5
42.2
40.0
Citation-analysis sample
2,194
1,620
3,814
Appendix
Table 16: Sample coverage and reported GPU resources in Main Conference and Findings papers.
Outcome
Main Conference β (SE) [ ΔR2 ]
Findings β (SE) [ ΔR2 ]
Pooled β (SE) [ ΔR2 ]
Findings − Main difference ( p )
NLP topic–year citation percentile
0.035 (0.011) [0.0042]
0.046 (0.014) [0.0071]
0.039 (0.009) [0.0051]
+0.011 (0.551)
OpenAlex field-normalized percentile
0.013 (0.009) [0.0009]
0.040 (0.011) [0.0073]
0.024 (0.007) [0.0029]
+0.027 (0.051)
log(1+citations)
0.172 (0.049) [0.0052]
0.207 (0.054) [0.0083]
0.185 (0.036) [0.0061]
+0.035 (0.630)
Citation count (PPML)
0.477 (0.085) [—]
0.390 (0.090) [—]
0.456 (0.066) [—]
−0.088 (0.481)
Top-10% cited (LPM)
0.038 (0.014) [0.0043]
0.059 (0.018) [0.0093]
0.047 (0.011) [0.0062]
+0.020 (0.372)
N
2,194
1,620
3,814
3,814
Appendix
Table 17: Replication of citation-impact associations and incremental explanatory power across publication tracks.
Track
High-impact rate among high-capability papers
High-impact rate among other papers
Risk ratio
High-capability share among high-impact papers
High-capability papers not high-impact
Main Conference
14.5%
9.1%
1.59
28.6%
85.5%
Findings
15.8%
9.0%
1.75
30.6%
84.2%
Main + Findings
14.7%
9.2%
1.60
28.6%
85.3%
Appendix
Table 18: Overlap between high reported GPU capability and high citation impact across publication tracks.
Assessing a cited paper's impact is typically done by analyzing its citation context in isolation within the citing paper. While this focuses on the most directly relevant text, it prevents relative comparisons across all the works a paper cites. We propose Crystal, which instead jointly ranks all cited papers within a citing paper using large language models (LLMs). To mitigate LLMs' positional bias, we rank each list three times in a randomized order and aggregate the impact labels through majority voting. This joint approach leverages the full citation context, rather than evaluating citations independently, to more reliably distinguish impactful references. Crystal outperforms a prior state-of-the-art impact classifier by +9.5% accuracy and +8.3% F1 on a dataset of human-annotated citations. Crystal further gains efficiency through fewer LLM calls and outperforms prior baselines using an open-weight model, enabling scalable, cost-effective citation impact analysis. In a case study of ACL Test-of-Time award-winning papers, we find that Crystal's impact characterizations align closely with long-term scientific recognition. We release Crystal-Bank, a 46.8k-paper dataset with rankings and impact labels, along with code.
Hannah Collison, Benjamin Van Durme, Daniel Khashabi
Natural Language Processing (NLP) has traditionally been published in its core disciplinary venues like ACL. However, advances in Large Language Models (LLMs) has led to a blurring of the disciplinary lines between NLP and general Machine Learning (ML), with authors regularly publishing in venues from both fields. Here, we ask whether the disciplinary center of gravity is shifting. Using NLP research published from 2010 to 2026 and studies of both established and new authors, we find that a migration is taking place. First, comparing the pre- and post-LLM eras, established authors lost 19.2pp of share at flagship *ACL main-conference tracks while gaining 14.8pp in the newer Findings tracks, and general ML venues rose 8.6pp, even when adjusting for parallel growth in the fields. Second, among newer authors who debut with at least three first-author NLP-topic papers, the share whose work appears mostly at *ACL venues fell from 84% (2019) to 74% (2024), while the share appearing mostly at general ML venues rose from 5% to 21%. Using causal inference techniques, we estimate that these general ML venues confer a significant citation premium, which influences venue selection. Together, these results point to a significant shift in where NLP research is published.
Source code and data published at computational linguistics (*CL) venues are increasingly being shared via GitHub. While this generally is a favourable development for the accessibility and potential reusability of research artifacts in natural language processing (NLP), the long-term availability of such repositories has not been evaluated. In this squib, we discuss the availability of repositories linked in papers published in the Computational Linguistics (CL) journal as well as at ACL and its co-located events over the past ten years. Contrary to our expectations, we find that GitHub repositories linked in more recent ACL publications are unavailable at similar rates as in older publications, in parts due to an increase in empty and placeholder repositories. Similar trends hold for other *CL venues, but not for platforms other than GitHub.