Big Claims, Small Margins: Do Reported GraphLLM Gains Survive Simple Baselines?

Michal Podstawski

cs.LGOct 20, 2026

Big Claims, Small Margins: Do Reported GraphLLM Gains Survive Simple Baselines?

Authors: Michal Podstawski

Organizations: NASK National Research Institute, Warsaw, Poland.

Abstract

LLM-based methods for node classification on text-attributed graphs (TAGs) routinely report state-of-the-art accuracy. We ask how these results compare with inexpensive baselines run on the same splits. On the splits of three recent studies, we tune four baseline representatives with validation-only selection and compare them with the highest mean that each paper reports for an LLM-based method in the matched setting. Correct & Smooth on TF-IDF features trails this reference by a median of 2.3 percentage points, and an MLP on frozen e5-large-v2 embeddings by 2.8 points. Correct & Smooth is within 3 points of the reference or better on 15 of 24 tasks, but the reference is significantly higher on 12 of 22 testable tasks, so several reported gains remain substantial. Reading released code next to the papers, we also document evaluation risks: selection of hyperparameters or checkpoints on the test set, labels of test-pool nodes used in training, baseline numbers reused without disclosure, and a text-label misalignment in a released Cora loader. Reports of the same method on the same split also differ by up to 12.5 points across papers whose configurations are not documented to match.

Figures & tables

Name	Representative
TF-IDF	TF-IDF $\to$ logistic regression (no graph)
TF-IDF+Prop	propagated TF-IDF $\to$ logistic regression
C&S	Correct & Smooth on the TF-IDF predictions
e5-MLP	frozen e5-large-v2 embedding $\to$ MLP (no graph)

Table 1: Baseline families and the representative reported in Table 2 .

LLM4HeG (NAACL’25)
Dataset	Reference	TF-IDF	TF-IDF+Prop	C&S	e5-MLP
Cornell	Vicuna-7B 77.6 $\pm$ 3.1	73.6 $\pm$ 1.4 ( $-$ 4.0)	62.6 $\pm$ 5.5 ( $-$ 15.0) ▼	75.7 $\pm$ 2.5 ( $-$ 1.9)	80.7 $\pm$ 2.4 (+3.1)
Texas	Vicuna-7B 89.1 $\pm$ 3.5	83.2 $\pm$ 3.1 ( $-$ 5.9) ▼	81.1 $\pm$ 6.1 ( $-$ 8.0)	80.9 $\pm$ 3.6 ( $-$ 8.2) ▼	88.6 $\pm$ 2.6 ( $-$ 0.5)
Wisconsin	7B-to-560M 87.2 $\pm$ 2.6	81.8 $\pm$ 1.5 ( $-$ 5.4) ▼	79.3 $\pm$ 2.5 ( $-$ 7.9) ▼	80.4 $\pm$ 2.8 ( $-$ 6.8) ▼	83.2 $\pm$ 3.3 ( $-$ 4.0)
Actor	Vicuna-7B 76.8 $\pm$ 0.5	71.9 $\pm$ 0.4 ( $-$ 4.9) ▼	73.1 $\pm$ 0.7 ( $-$ 3.7) ▼	73.5 $\pm$ 0.8 ( $-$ 3.3) ▼	76.0 $\pm$ 0.3 ( $-$ 0.8) ▼
Amazon	Bloom-1B 52.3 $\pm$ 0.6	55.2 $\pm$ 0.8 (+2.8) ▲	47.4 $\pm$ 1.1 ( $-$ 4.9) ▼	55.9 $\pm$ 1.9 (+3.5) ▲	57.7 $\pm$ 0.9 (+5.4) ▲

Table 2: Highest mean reported for an LLM-based method in the matched setting (reference) vs. our representatives, fixed by name, on the same splits (test accuracy, %, mean

\pm

sample std). In parentheses: ours

-

reference. ▲ / ▼ : significant after Holm correction (Welch, Section 3.4 ). † : no std or run count reported, so significance is not assessable. Reference sources: LLM4HeG Table 2; KCoT Table 2 (Single Focus, node classification); LLMNodeBed Table 2. Our baselines use 10 independent tuning runs on one fixed split for LLM4HeG and the LLaGA split, and 4 runs on LLMNodeBed (re-drawn splits in the supervised setting and for Books, Photo and Computer).

Name	Tasks	Median gap	Gap $\leq 3$	▼ / ▲	Testable
TF-IDF	24	5.5	3	17 / 1	22
TF-IDF+Prop	24	3.0	12	16 / 0	22
C&S	24	2.3	15	12 / 1	22
e5-MLP	24	2.8	12	10 / 1	22

Table 3: Summary of Table 2 per representative. Gap

=

reference

-

ours, in points; “Gap

\leq 3

” counts tasks within 3 points of the reference or better. ▼ / ▲ : significantly below/above the reference, as in Table 2 ; Testable: tasks where a test is possible.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	GCN (ours)	GCN (lit.)	SGC (ours)	SGC (lit.)
Cora	81.9 $\pm$ 0.8	81.5	81.0	81.0
CiteSeer	70.8 $\pm$ 1.2	70.3	71.9	71.9
PubMed	79.2 $\pm$ 0.4	79.0	78.8	78.9

Appendix

Table 4: GCN and SGC on the Planetoid public splits (accuracy, %; GCN: mean

\pm

std over 10 seeds). Literature values: Kipf and Welling (2017) ; Wu et al. (2019) .

LLMs for Graph Learning

Large language models for graph learning combine language understanding with graph structure for prediction, reasoning, and knowledge discovery.

See all 74 papers

Where LLM Annotators Fail: Label-Free Learning on Graphs with LLMs
May 27, 2026Safal Thapaliya, Jiatan Huang, Chuxu ZhangLLM-Assisted Annotation Text-Attributed Graph Learning
University of Connecticut, USA
GaLoRA: Parameter-Efficient Graph-Aware LLMs for Node Classification
Mar 11, 2026Mayur Choudhary, Saptarshi Sengupta, Katerina PotikaLLMs for Graph Learning Text-Attributed Graph Learning
Department of Computer Science, San Jose State University
GraphInfer-Bench: Benchmarking LLM's Inference Capability on Graphs
Jun 10, 2026Zhuoyi Peng, Jingzhou Jiang, Hanlin Gu +2LLM Evaluation Graph Neural Networks
The Hong Kong University of Science and Technology · Webank

May 27, 2026J/K move · Enter open · S save

Where LLM Annotators Fail: Label-Free Learning on Graphs with LLMs

Safal Thapaliya, Jiatan Huang, Chuxu Zhang

University of Connecticut, USA

LLM-Assisted Annotation Text-Attributed Graph Learning Noisy-Label Learning Classification Learning with Noisy Labels Node Classification+1

Open paper AskPDF arXiv

Node classification on graphs often requires labeled nodes, yet obtaining labels at graph scale is expensive. When node attributes contain semantic content, such as paper abstracts, web pages, or product descriptions, large language models (LLMs) can provide low-cost supervision by annotating a small subset of nodes. However, these LLM-generated labels are noisy, and existing label-free graph learning methods usually treat this noise as either global or class-conditional. We find that LLM annotation errors are not only class-dependent but also region-dependent: within the same class, reliability can vary sharply across feature-space clusters. In light of this, we propose Cluster-Aware Noise Estimation (CANE), a label-free learning framework that estimates cluster-conditional LLM reliability without ground truth labels, and uses this estimate to decide which pseudo-labels to trust, and which labels to correct. Across various graph benchmarks and GNN backbones, CANE improves over the strongest label-free baselines, with the largest gains on datasets exhibiting stronger cluster-conditional noise.

Abstract

LLMs for Graph Learning

Explore similar work

Where LLM Annotators Fail: Label-Free Learning on Graphs with LLMs

GaLoRA: Parameter-Efficient Graph-Aware LLMs for Node Classification

GraphInfer-Bench: Benchmarking LLM's Inference Capability on Graphs