Can LLMs Predict Academic Collaboration? Topology Heuristics vs. LLM-Based Link Prediction on Real Co-authorship Networks

Fan Huang; Munjung Kim

cs.SIApr 1, 2026

Can LLMs Predict Academic Collaboration? Topology Heuristics vs. LLM-Based Link Prediction on Real Co-authorship Networks

Authors: Fan Huang, Munjung Kim

Organizations: Luddy School of Informatics, Computing, and Engineering Indiana University Bloomington · School of Data Science University of Virginia

Abstract

Can large language models (LLMs) predict which researchers will collaborate? We study this question through link prediction on real-world co-authorship networks from OpenAlex (9.96M authors, 108.7M edges), evaluating whether LLMs can predict future scientific collaborations using only author profiles, without access to graph structure. Using Qwen2.5-72B-Instruct across three historical eras of AI research, we find that LLMs and topology heuristics capture distinct signals and are strongest in complementary settings. On new-edge prediction under natural class imbalance, the LLM achieves AUROC 0.714--0.789, outperforming Common Neighbors, Jaccard, and Preferential Attachment, with recall up to 92.9%; under balanced evaluation, the LLM outperforms all topology heuristics in every era (AUROC 0.601--0.658 vs.\ best-heuristic 0.525--0.538); on continued edges, the LLM (0.687) is competitive with Adamic-Adar (0.684). Critically, 78.6--82.7% of new collaborations occur between authors with no common neighbor -- a blind spot where all topology heuristics score zero but the LLM still achieves AUROC 0.652 by reasoning from author metadata alone. A temporal metadata ablation reveals that research concepts are the dominant signal (removing concepts drops AUROC by 0.047--0.084). Providing pre-computed graph features to the LLM degrades performance due to anchoring effects, confirming that LLMs and topology methods should operate as separate, complementary channels. A socio-cultural ablation finds that name-inferred ethnicity and institutional country do not predict collaboration beyond topology, reflecting the demographic homogeneity of AI research. A node2vec baseline achieves AUROC comparable to Adamic-Adar, establishing that LLMs access a fundamentally different information channel -- author metadata -- rather than encoding the same structural signal differently.

Figures & tables

Figure 1: Ground truth construction pipeline. Three stages progressively narrow the OpenAlex corpus (249.1M works) to AI co-authorship edges, split them across three historical eras with train-eval temporal windows, detect communities via Louvain, and select the top-1 (largest) community per era for evaluation.

Figure 2: Empirical basis for the three-era temporal split. (a) Network scale: edges (bars) and authors (line). (b) Edge growth rate: the 2.61 $\times$ spike marks Boundary 1; deceleration to 0.99 $\times$ marks Boundary 2. (c) Average degree peaks then declines. (d) Edges-per-author peak then drop.

Era	Name	Train	Eval	Authors	New Edges	2-hop Cands.
1	Classical AI	04–07	08–09	38,842	45,317	1.17M
2	DL Explosion	10–15	16–17	393,843	914,270	61.6M
3	Democratization	18–21	22–23	555,434	2,704,671	329.7M

Table 1: Temporal eras and evaluation community scale. Edges classified as continued (both periods), new (eval only), or dropped (train only).

Figure 3: Two prediction methods. (a) Topology heuristics: five structure-based scoring functions applied to candidate pairs. (b) LLM confirmation: author profiles provided to Qwen2.5-72B for binary link prediction.

Figure 4: Experimental sample networks for each era (5,000 stratified candidate pairs). Nodes colored by sub-community; red edges indicate true future collaborations (positives). Only 47 (Era 1), 14 (Era 2), and 7 (Era 3) of 5,000 pairs are positives, illustrating the extreme class imbalance.

Heuristic	Era 1	Era 2	Era 3
Adamic-Adar	0.750	0.826	0.823
Resource Allocation	0.735	0.813	0.803
Common Neighbors	0.656	0.699	0.726
Jaccard Coefficient	0.601	0.662	0.545
Preferential Attach.	0.491	0.437	0.574

Table 2: Link prediction AUROC for topology heuristics across three eras.

Era	$N$	Pos:Neg	LLM	AA	RA	CN
Era 1	5,000	47:4,953	0.714	0.801	0.805	0.682
Era 2	5,000	14:4,986	0.777	0.879	0.880	0.716
Era 3	5,000	7:4,993	0.789	0.879	0.878	0.758

Table 3: New-edge prediction AUROC under natural class imbalance (5,000 AA-decile stratified pairs per era). AA and RA dominate across all eras; the LLM outperforms CN. Era 3 contains only 7 positives, so its AUROC estimates carry high variance.

Era	$N$	Pos:Neg	LLM	AA	RA	CN
Era 1	500	250:250	0.658	0.533	0.531	0.538
Era 2	500	250:250	0.615	0.520	0.524	0.525
Era 3	500	250:250	0.601	0.509	0.512	0.533

Table 4: New-edge prediction AUROC under balanced sampling (250 positive, 250 negative, AA-quintile stratified). The LLM outperforms all topology heuristics in every era.

Era	w/ Conc.	No-Conc.	Era-Restr.	$\Delta$ (conc.)	$\Delta$ (era)
Era 1	0.573	0.526	0.544	$+$ 0.047	$+$ 0.018
Era 2	0.694	0.610	0.562	$+$ 0.084	$-$ 0.048
Era 3	0.707	0.627	0.570	$+$ 0.080	$-$ 0.057

Table 5: Temporal metadata ablation AUROC (50,000 pairs per era).

\Delta

(conc.) = effect of removing concepts;

\Delta

(era) = effect of era-restricting counts and institution.

Figure 5: 68 positive pairs plotted by AA score vs. LLM probability, colored by agreement category. The LLM catches 40% of positives that AA misses (red squares, lower-left quadrant), while only 1 pair is caught by AA alone. Dashed/dotted lines show Era 1 AA threshold (0.48) and LLM threshold (0.5).

Category	Method	Era 1	Era 2
Topology	AA / CN / JC / RA	0.500	0.500
Topology	Pref. Attachment	0.095	0.848
Embedding	N2V-Hadamard	0.651	0.916
Embedding	N2V-Cosine	0.154	0.848
Metadata	$\text{cited\_by}(u)\times\text{cited\_by}(v)$	0.876	0.942
	$\|\text{concepts}(u)\cap\text{concepts}(v)\|$	0.779	0.801

Table 6: Cold-start prediction AUROC (top-1,000 authors, 500 sampled pairs, 1 positive per era). CN-based heuristics score exactly 0.500. Era 3 is omitted because zero cold-start positives exist among the top-1,000 hubs (cold-start collaborations in Era 3 occur only among lower-degree authors).

Figure 6: AUROC by edge type and prediction method (natural imbalance). (a) New edges: AA and RA lead, but LLM outperforms CN and JC; under balanced sampling the LLM outperforms all heuristics (Table 4 ). (b) Continued edges: LLM (0.687) is competitive with AA (0.684). (c) Cold-start: metadata features dominate; LLM (0.652) provides signal where topology scores zero. Dashed line = random.

Prompt Variant	AUROC	$\Delta$ vs. Base
Base (metadata only)	0.699	—
+ Country	0.686	$-0.013$
+ Ethnicity	0.686	$-0.013$
+ Both	0.683	$-0.016$

Table 7: LLM prompt ablation with socio-cultural factors (Era 1,

N{=}5{,}000

, 47 positives).

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Window	Authors	Edges	Avg Deg	Edge Gr.	E/A
2004–05	631,767	2,257,606	7.15	—	3.6
2006–07	753,373	2,486,310	6.60	1.10 $\times$	3.3
2008–09	875,530	3,322,828	7.59	1.34 $\times$	3.8
2010–11	1,056,752	8,686,468	16.44	2.61 $\times$	8.2
2012–13	1,220,422	12,628,718	20.70	1.45 $\times$	10.3
2014–15	1,358,721	17,771,014	26.16	1.41 $\times$	13.1

Appendix

Table 8: OpenAlex AI co-authorship network statistics by two-year window. Dashed lines mark era boundaries.

Figure 7: Link prediction evaluation pipeline. The OpenAlex AI co-authorship network is split into three historical eras with train/eval periods. Author profiles are fetched via the OpenAlex API. Candidate pairs are scored by topology heuristics or classified by an LLM.

Figure 8: Distribution of Adamic-Adar scores for 5,000 stratified candidate pairs per era. Positives cluster at higher AA scores but remain needle-in-haystack rare: 47 (Era 1), 14 (Era 2), and 7 (Era 3) out of 5,000 pairs.

Model	Prompt Variant	AUROC
Qwen-72B	v2 (metadata only)	0.547
Qwen-72B	v2 + network statistics	0.486
GPT-4o	v2 + network statistics	0.467

Appendix

Table 9: Prompt-design pilot AUROC (550 balanced pairs, Era 1).

Field	Era-Restricted?	Method	Effort
display_name	Yes (stable)	No action needed	None
works_count	Yes (direct)	counts_by_year	1 call/author
cited_by_count	Yes (direct)	counts_by_year	1 call/author
institution	Yes (direct)	affiliations	1 call/author
concepts/topics	No (indirect)	Per-work aggregation	Millions of calls

Appendix

Table 10: OpenAlex API temporal metadata feasibility.

Method	Top-5 Concepts (2004–07)	Overlap
Career-level	CS, AI, ML, Math, Neural Networks	(reference)
Mean score	Pixel, Convexity, Variation, Isomap, SGD	0/5
Max score	RL, Isomap, Pixel, Dim. Reduction, Curse of Dim.	0/5
Frequency	CS, AI, Math, ML, Kernel	4/5
Citation-wtd	CS, AI, Training, ML, Layer	3/5
Top-5 voting	CS, AI, Math, Curse of Dim., ML	4/5

Appendix

Table 11: Era-restricted concept profiles for Yoshua Bengio (2004–2007, 48 works) under five aggregation methods.

Figure 9: Topology heuristic AUROC across three eras. AA and RA consistently outperform; PA hovers near random.

Era	Best N2V Op.	N2V AUROC	Best Topo	Topo AUROC	$\Delta$
Classical AI	L2 distance	0.781	AA	0.773	$+$ 0.008
DL Explosion	Cosine	0.834	AA	0.830	$+$ 0.004
Democratization	Cosine	0.792	RA	0.854	$-$ 0.062

Appendix

Table 12: node2vec vs. topology heuristic AUROC across three eras (50,000 stratified pairs per era).

Method	AUROC
Resource Allocation	0.706
LLM (Qwen-72B)	0.687
Adamic-Adar	0.684
Edge Weight	0.659
Common Neighbors	0.619
Jaccard Coefficient	0.618

Appendix

Table 13: Continued-edge prediction AUROC (Era 1,

N=150

, 67 continued vs. 83 dropped).

Figure 10: LLM calibration by Adamic-Adar decile across three eras. The narrow gap between TPR (red) and FPR (blue) indicates poor discrimination.

Figure 11: Cold-start rate by author degree bin. The fraction of new edges with zero common neighbors decreases monotonically with degree, from 87–91% for degree-1 authors to 40–51% for degree-1,000+.

Figure 12: Top-K sweep: cold-start new edges (bars) and cold-start rate (line) by author rank. In Era 3, cold-start is absent among top-1,000 hubs but accounts for 79% of new edges in the full network.

Figure 13: Shortest-path distance distribution for cold-start vs. 2-hop new edges. Cold-start pairs have median distance 4–5, confirming they are structurally distant.

Figure 14: Cross-community rate for cold-start vs. 2-hop new edges. Cold-start edges are overwhelmingly cross-community (81–97%) compared to 2-hop edges (39–67%).

Category	Feature	AUROC
Topology	Adamic-Adar	0.801
Topology	Resource Allocation	0.805
Metadata	Concept Overlap	0.555
Metadata	Same Institution	0.527
Socio-cultural	Same Ethnicity	0.475
	Same Country	0.448

Appendix

Table 14: Feature AUROC on Era 1 candidate pairs (

N=5{,}000

, 47 positives). Socio-cultural features fall below random (0.5).

Feature	Collab. Same Rate	Non-collab. Same Rate	Homophily Ratio
Ethnicity	83.0%	88.0%	0.94
Country	40.4%	50.8%	0.80
Continent	46.8%	57.8%	0.81

Appendix

Table 15: Homophily analysis on Era 1 candidate pairs. Ratios below 1.0 indicate anti-homophily.

Figure 15: Feature AUROC by category and era for all 11 features. All socio-cultural features fall below the random baseline.

Figure 16: Spearman correlation matrix for all 11 features across three eras. Cross-category correlations are negligible ( $|r|<0.1$ ).

Figure 17: (a) Homophily ratios by era: values below 1.0 indicate anti-homophily. (b) LLM prompt ablation: adding socio-cultural context consistently degrades AUROC.

Figure 18: Community structure of the Era 1 (Classical AI) training graph (38.8K authors, 109K edges; 1,500 subsampled).

Figure 19: Community structure of the Era 2 (DL Explosion) training graph (393.8K authors, 1.95M edges; 1,500 subsampled).

Figure 20: Community structure of the Era 3 (Democratization) training graph (555.4K authors, 4.1M edges; 1,500 subsampled).

Link Prediction

Link prediction estimates missing or future relationships in graphs, from social ties and knowledge-graph facts to biological interactions.

See all 44 papers

Jul 23, 2026

Scientific exploration, collaboration and labor division in the large language model era

Large language models (LLMs) have rapidly and significantly entered scientific workflows, but it remains unclear how their diffusion is associated with changes in scientists' strategies in research directions and team building. We link PubMed Central full text with OpenAlex publication and collaboration histories for 775,323 scientists and analyze CRediT contribution statements from 137,120 multi-author papers. After 2022, scientists increasingly published across more intellectually distant fields and entered fields in which they had not previously worked. These increases in interdisciplinarity and exploration were especially pronounced among established scientists and scientists from non-English-speaking low- and middle-income countries. Authors with stronger AI-writing signals were already more interdisciplinary and exploratory before the widespread adoption of LLMs, and the gap widened further after 2022 compared with authors with weaker AI-writing signals. Scientists' collaboration networks also became more interdisciplinary after 2022. Yet, among authors with stronger AI-writing signals, research interdisciplinarity was less closely tied to the disciplinary diversity of their collaborators. The division of labor within research teams also became more differentiated. Contributors on papers published after 2022 reported narrower role sets on average, coauthors shared fewer roles in common, and their role profiles became less rigid and more fluid. Software and validation roles increased, while conceptual and management roles decreased. These patterns suggest that team members are taking on more distinct responsibilities and may rely less on one another to perform research tasks. Overall, this study indicates that the LLM era coincides with a broader reorganization of scientific exploration, collaboration, and the division of labor.

Xiang Zheng, Xi Hong, Jialin Liu +1

Information School, University of Wisconsin–Madison, Madison, WI, USA.

Scientific Discovery AI-Assisted Scientific Research

Ask PDF

May 21, 2026

LLM-Metrics: Measuring Research Impact Through Large Language Model Memory

Citation counts remain the dominant metric for assessing research impact, yet they suffer from well-documented limitations: temporal lag, disciplinary bias, and Matthew effects. Here we propose LLM-Metrics, a research-impact assessment metric derived from the parametric memory of large language models (LLMs). The central hypothesis is that high-impact papers receive greater exposure in the academic community, that this exposure enters LLM training data in textual form, and that models consequently form stronger parametric memory of these papers. We designed four types of multiple-choice probes, covering title recognition, author recognition, method recognition, and venue recognition, and evaluated 549 computer science papers published in 2023-2024 across 17 LLMs spanning 0.5B to 72B parameters from six vendors. Of the 17 models, 15 produced positive predictions, 9 of which were significant at p less than 0.05, with an overall Spearman correlation of rho = 0.1495 and p = 0.0004 against citation counts. Three additional findings support the proposed mechanism. First, the predictive signal was stronger for 2024 papers, rho = 0.1880, whose citation counts were near zero at model-training time, reducing the plausibility of a simple reverse-causality explanation. Second, author-recognition probes showed the strongest discriminative power, consistent with an exposure-driven memory mechanism. Third, model scale and predictive power were non-monotonic: a 3B-parameter model, Llama-3.2-3B-Instruct, with rho = 0.1829, outperformed most larger models, supporting a selective-memory hypothesis in which the limited capacity of smaller models can serve as an effective information filter. LLM-Metrics offers a real-time, cross-disciplinary, citation-independent paradigm for research assessment.

Si Shen, Wenhua Zhao, Danhao Zhu

School of Economics and Management Department of Computer Science and Engineering Nanjing University of Science and Technology Nanjing, 210094, China · College of Information Management Nanjing Agricultural University Nanjing, 210095, China · Department of Criminal Science and Technology Jiangsu Police Institute Nanjing, 210031, China

Topic pending

Ask PDF

May 27, 2026

Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation

Large language models (LLMs) are increasingly used as scholar recommenders, shaping who is seen as an expert in academia. Existing audits remain English-centric, single discipline, and persona-agnostic, leaving the source of output variability poorly understood. To this end, we propose a benchmark that disentangles the effects of model choice and prompt design on recommendations. We audit 43 LLMs by varying persona prompts (language, location, role-and-task) and context (field, seniority, k). Recommended scholars are compared against Semantic Scholar over six scientific disciplines to measure technical quality (factuality, coverage) and social representativeness (diversity, parity). Basic technical quality is driven by model choice, factuality and parity by context, and diversity by location. South Africa prompts yield less factual lists, while Japan prompts yield highly factual but homogeneous lists skewed toward highly productive scholars. Prompt design is thus a non-trivial axis of LLM-based scholar discovery and should be systematically audited alongside model choice.

Annabella Sánchez-Guzmán, Lukas Eberhard, Denis Helic +1

Escuela Superior Polit´ecnica del Litoral · Graz University of Technology · Complexity Science Hub

LLM Auditing LLM Prompting Recommender Systems Large Language Model-Based Recommendation Persona Prompting Demographic Bias in Language Models

Ask PDF

Abstract

Link Prediction

Explore similar work

Scientific exploration, collaboration and labor division in the large language model era

LLM-Metrics: Measuring Research Impact Through Large Language Model Memory

Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation