Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting

Hanxu Yang; Yuhuan Zhao; Xiaodong He; Zhao Kang

cs.LGSep 26, 2026

Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting

Authors: Hanxu Yang, Yuhuan Zhao, Xiaodong He, Zhao Kang

Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge. Graph prompting offers a parameter-efficient alternative to fine-tuning, but existing methods largely treat pre-trained models as opaque feature extractors, ignoring their internal spectral structure. In this work, we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias: optimization during pre-training disproportionately aligns representations with directions associated with large singular values, leaving low-energy directions under-explored. We show that these underutilized directions can encode complementary information that is beneficial for downstream adaptation, especially under distribution shift. To leverage this insight, we propose Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders. SRP applies a learnable soft-thresholding mask in the spectral domain to down-weight dominant directions while amplifying weaker ones. In addition, SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation under the frozen encoder. Extensive experiments across multiple benchmarks demonstrate that SRP achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation.

Figures & tables

Figure 1: Left: DPMS and

\Delta_{q}

in the singular values of the quintices of

W_{down}

. Right: Information density of encoder and null space.

Figure 2: Overall framework of Spectral Reverse Prompt (SRP). SRP decomposes each frozen GNN layer weight to expose spectral bias, then uses a spectral channel to suppress dominant directions and amplify weak ones, and a null-space channel to exploit information that is not captured by the frozen encoder. The two channels are fused into layer-wise prompts injected into the frozen GNN for downstream tuning.

Pre-training	Tuning Methods	Cora	CiteSeer	PubMed	ogbn-arxiv	Flickr
GraphCL	Classifier Only	$53.05_{\pm 4.76}$	$38.62_{\pm 3.43}$	$64.28_{\pm 4.51}$	$21.15_{\pm 1.64}$	$24.32_{\pm 2.93}$
	GPPT	$50.96_{\pm 6.67}$	$39.50_{\pm 1.67}$	$60.47_{\pm 4.75}$	$17.99_{\pm 1.14}$	$24.35_{\pm 1.84}$
	GraphPrompt	$55.71_{\pm 4.62}$	$40.81_{\pm 2.11}$	$63.47_{\pm 2.23}$	$21.03_{\pm 1.92}$	$26.08_{\pm 3.44}$
	ALL-in-one	$38.00_{\pm 4.17}$	$40.27_{\pm 2.09}$	$58.61_{\pm 3.49}$	$16.42_{\pm 2.98}$	$25.08_{\pm 3.44}$
	GPF	$58.52_{\pm 4.07}$	$43.55_{\pm 2.80}$	$67.67_{\pm 3.14}$	$21.73_{\pm 1.75}$	$23.98_{\pm 1.71}$
	GPF-plus	$52.24_{\pm 4.59}$	$38.47_{\pm 3.27}$	$64.30_{\pm 4.58}$	$21.03_{\pm 1.96}$	$25.32_{\pm 2.02}$

Table 1: Performance on node-level benchmarks (5-seed avg.). Bold = best, underline = second best.

Pre-training	Tuning Methods	ENZYMES	DD	NCI1	NCI109	Mutagenicity
GraphCL	Classifier Only	$30.50_{\pm 1.16}$	$62.89_{\pm 2.19}$	$62.49_{\pm 1.95}$	$61.68_{\pm 0.93}$	$66.62_{\pm 1.87}$
	GraphPrompt	$27.83_{\pm 1.61}$	$64.33_{\pm 1.79}$	$63.19_{\pm 1.71}$	$62.18_{\pm 0.48}$	$67.62_{\pm 0.65}$
	ALL-in-one	$25.92_{\pm 0.55}$	$66.54_{\pm 1.82}$	$57.52_{\pm 2.61}$	$62.74_{\pm 0.78}$	$63.43_{\pm 2.53}$
	GPF	$30.08_{\pm 1.25}$	$64.54_{\pm 2.22}$	$62.66_{\pm 1.83}$	$62.29_{\pm 0.90}$	$66.54_{\pm 1.85}$
	GPF-plus	$31.00_{\pm 1.50}$	$67.26_{\pm 2.29}$	$64.56_{\pm 1.10}$	$62.84_{\pm 0.22}$	$66.82_{\pm 1.63}$
	EdgePrompt	$29.50_{\pm 1.57}$	$64.16_{\pm 2.13}$	$63.05_{\pm 2.11}$	$62.59_{\pm 0.93}$	$66.87_{\pm 1.88}$

Table 2: Performance on graph-level benchmarks (5-seed avg.). Bold = best, underline = second best.

Variant	GraphCL			SimGRACE
Variant	Cora	CiteSeer	Avg. $\Delta$	Cora	CiteSeer	Avg. $\Delta$
SRP (full)	$\bm{68.1_{\pm 2.3}}$	$\bm{49.0_{\pm 2.3}}$	$-0.0$	$\bm{67.2_{\pm 7.3}}$	$\bm{51.4_{\pm 5.2}}$	$-0.0$
SRP-NM (w/o null channel)	$63.4_{\pm 2.8}$	$41.1_{\pm 3.8}$	$-6.3$	$60.3_{\pm 5.9}$	$43.5_{\pm 2.9}$	$-7.4$
SRP-NS (w/o spectral channel)	$64.4_{\pm 2.1}$	$45.6_{\pm 3.2}$	$-3.6$	$63.5_{\pm 6.6}$	$47.7_{\pm 4.3}$	$-3.7$
SRP-NR (w/o reverse)	$64.7_{\pm 2.7}$	$48.0_{\pm 4.0}$	$-2.2$	$63.4_{\pm 4.4}$	$49.9_{\pm 4.9}$	$-2.7$
SRP-Bi (bidirectional)	$64.7_{\pm 3.3}$	$47.9_{\pm 3.3}$	$-2.3$	$63.0_{\pm 4.5}$	$50.0_{\pm 4.8}$	$-2.8$

Table 3: Ablation Study of Different Variants on GraphCL and SimGRACE.

Figure 3: Label Sensitivity and Parameter Sensitivity of NS-PCA and bottleneck rank.

Figure 4: Convergence speed of different methods.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Nodes	#Edges	#Feat.	#Classes	Task
Cora	2,708	10,556	1,433	7	Node
CiteSeer	3,327	9,104	3,703	6	Node
PubMed	19,717	88,648	500	3	Node
ogbn-arxiv	169,343	1,166,243	128	40	Node
Flickr	89,250	899,756	500	7	Node
Cornell	183	298	1703	5	Node

Appendix

Table 4: Node-level benchmark datasets. #Feat. is the raw input feature dimension; #Classes is the number of downstream classes.

Dataset	#Graphs	Avg. Nodes	Avg. Edges	#Feat.	#Classes	Task
ENZYMES	600	32.63	124.27	3	6	Graph
DD	1,178	284.32	1,431.32	89	2	Graph
NCI1	4,110	29.87	64.60	37	2	Graph
NCI109	4,127	29.68	64.26	38	2	Graph
Mutagenicity	4,337	30.32	61.54	14	2	Graph

Appendix

Table 5: Graph-level benchmark datasets (TUDataset). Avg. Nodes and Avg. Edges are computed over the graph collection.

Backbone	Dataset	Classifier-only	GraphLoRA	SRP
GraphGPS	Cora	$48.17\pm 6.12$	$64.33\pm 6.88$	$\mathbf{66.62\pm 5.27}$
GraphGPS	CiteSeer	$39.13\pm 4.57$	$46.80\pm 3.38$	$\mathbf{58.74\pm 5.01}$
GraphGPS	NCI1	$63.32\pm 1.51$	$62.49\pm 1.93$	$\mathbf{66.81\pm 2.13}$
GraphGPS	ENZYMES	$35.08\pm 2.82$	$45.00\pm 3.25$	$\mathbf{47.75\pm 2.71}$
Graphormer	Cora	$51.13\pm 4.04$	$62.88\pm 5.49$	$\mathbf{65.69\pm 5.97}$
Graphormer	CiteSeer	$39.01\pm 1.74$	$47.13\pm 1.72$	$\mathbf{54.89\pm 1.60}$

Appendix

Table 6: Performance of SRP with GraphGPS and Graphormer backbones. Results are reported as mean

\pm

standard deviation over five runs.

Mask setting	Cora	CiteSeer	NCI1	ENZYMES
$\gamma=0.001$	64.55	47.64	66.63	36.42
$\gamma=0.1$	64.88	48.03	66.52	36.92
$\gamma=1$	67.80	48.81	66.96	36.50
$\gamma=10$	68.10	48.99	66.80	36.67
$\gamma=100$	68.06	48.87	66.84	36.25
Hard threshold (STE)	67.95	48.90	66.89	35.67

Appendix

Table 7: Sensitivity to the temperature coefficient

\gamma

Dataset	Strong- $k$	Random- $k$	Weak- $k$	Hard (STE)	Soft $\gamma=10$
Cora	64.25	64.44	67.80	67.95	68.10
CiteSeer	47.10	47.46	48.78	48.90	48.99
NCI1	66.42	66.45	66.87	66.89	66.80

Appendix

Table 8: Matched-cardinality comparison of different spectral selections.

Dataset	Classifier-only	GraphLoRA	SRP	SRP vs. GraphLoRA
Cornell	$32.22\pm 8.35$	$45.56\pm 14.34$	$\mathbf{51.67\pm 5.72}$	$+6.11$
Squirrel	$23.08\pm 1.98$	$23.06\pm 1.40$	$\mathbf{25.12\pm 1.47}$	$+2.06$
Chameleon	$31.74\pm 1.88$	$31.43\pm 1.44$	$\mathbf{33.58\pm 2.76}$	$+2.15$

Appendix

Table 9: Performance on heterophilic graph benchmarks.

Dataset	GPF	SRP
Cora	0.121	0.077
CiteSeer	0.131	0.084
PubMed	0.678	0.508
ENZYMES	0.221	0.292
DD	0.191	0.136
NCI1	0.342	0.297

Appendix

Table 10: Per-epoch runtime in seconds. Lower is better.

Graph Neural Networks

Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.

See all 726 papers

Freeze, Prompt, and Adapt: A Framework for Source-free Unsupervised GNN Prompting
May 22, 2025Peyman Baghershahi, Sourav MedyaSource-Free Domain Adaptation Unsupervised Domain Adaptation
Department of Computer Science University of Illinois Chicago
A Cross-graph Tuning-free GNN Prompting Framework
Apr 1, 2026Yaqi Chen, Shixun Huang, Ryan Twemlow +6Graph Neural Networks Prompt Engineering
University of Wollongong, Northfields Avenue, Gwynneville, 2500, NSW, Australia · Wuhan University, No. 299, Luojia Hill, Wuhan, 430072, Hubei, China
Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective
Jun 2, 2026Yancheng Chen, Dun Ma, Shuai Zhang +6Prompt Tuning Graph Foundation Models
Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Institute of Information Engineering, Chinese Academy of Sciences +4

May 22, 2025J/K move · Enter open · S save

Freeze, Prompt, and Adapt: A Framework for Source-free Unsupervised GNN Prompting

Peyman Baghershahi, Sourav Medya

Department of Computer Science University of Illinois Chicago

Source-Free Domain Adaptation Unsupervised Domain Adaptation Prompt Engineering Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Prompt tuning has become a key mechanism for adapting pre-trained Graph Neural Networks (GNNs) to new downstream tasks. However, existing approaches are predominantly supervised, relying on labeled data to optimize the prompting parameters and typically fine-tuning a task-specific prediction head -- practices that undermine the promise of parameter-efficient adaptation. We propose Unsupervised Graph Prompting Problem (UGPP), a challenging new setting where the pre-trained GNN is kept entirely frozen, labels on the target domain are unavailable, the source data is inaccessible, and the target distribution exhibits covariate shift. To address this, we propose UGPrompt, the first fully unsupervised GNN prompting framework. UGPrompt leverages consistency regularization and pseudo-labeling to train a prompting function, complemented with diversity and domain regularization to mitigate class imbalance and distribution mismatch. Our extensive experiments demonstrate that UGPrompt consistently outperforms state-of-the-art supervised prompting methods with access to labeled data, demonstrating the viability of unsupervised prompting as a practical adaptation paradigm for GNNs.

Abstract

Graph Neural Networks

Explore similar work

Freeze, Prompt, and Adapt: A Framework for Source-free Unsupervised GNN Prompting

A Cross-graph Tuning-free GNN Prompting Framework

Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective