Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting

Hanxu Yang; Yuhuan Zhao; Xiaodong He; Zhao Kang

cs.LGSep 26, 2026

Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting

Authors: Hanxu Yang, Yuhuan Zhao, Xiaodong He, Zhao Kang

Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge. Graph prompting offers a parameter-efficient alternative to fine-tuning, but existing methods largely treat pre-trained models as opaque feature extractors, ignoring their internal spectral structure. In this work, we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias: optimization during pre-training disproportionately aligns representations with directions associated with large singular values, leaving low-energy directions under-explored. We show that these underutilized directions can encode complementary information that is beneficial for downstream adaptation, especially under distribution shift. To leverage this insight, we propose Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders. SRP applies a learnable soft-thresholding mask in the spectral domain to down-weight dominant directions while amplifying weaker ones. In addition, SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation under the frozen encoder. Extensive experiments across multiple benchmarks demonstrate that SRP achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation.

Figures & tables

Figure 1: Left: DPMS and

\Delta_{q}

in the singular values of the quintices of

W_{down}

. Right: Information density of encoder and null space.

Figure 2: Overall framework of Spectral Reverse Prompt (SRP). SRP decomposes each frozen GNN layer weight to expose spectral bias, then uses a spectral channel to suppress dominant directions and amplify weak ones, and a null-space channel to exploit information that is not captured by the frozen encoder. The two channels are fused into layer-wise prompts injected into the frozen GNN for downstream tuning.

Pre-training	Tuning Methods	Cora	CiteSeer	PubMed	ogbn-arxiv	Flickr
GraphCL	Classifier Only	$53.05_{\pm 4.76}$	$38.62_{\pm 3.43}$	$64.28_{\pm 4.51}$	$21.15_{\pm 1.64}$	$24.32_{\pm 2.93}$
	GPPT	$50.96_{\pm 6.67}$	$39.50_{\pm 1.67}$	$60.47_{\pm 4.75}$	$17.99_{\pm 1.14}$	$24.35_{\pm 1.84}$
	GraphPrompt	$55.71_{\pm 4.62}$	$40.81_{\pm 2.11}$	$63.47_{\pm 2.23}$	$21.03_{\pm 1.92}$	$26.08_{\pm 3.44}$
	ALL-in-one	$38.00_{\pm 4.17}$	$40.27_{\pm 2.09}$	$58.61_{\pm 3.49}$	$16.42_{\pm 2.98}$	$25.08_{\pm 3.44}$
	GPF	$58.52_{\pm 4.07}$	$43.55_{\pm 2.80}$	$67.67_{\pm 3.14}$	$21.73_{\pm 1.75}$	$23.98_{\pm 1.71}$
	GPF-plus	$52.24_{\pm 4.59}$	$38.47_{\pm 3.27}$	$64.30_{\pm 4.58}$	$21.03_{\pm 1.96}$	$25.32_{\pm 2.02}$

Table 1: Performance on node-level benchmarks (5-seed avg.). Bold = best, underline = second best.

Pre-training	Tuning Methods	ENZYMES	DD	NCI1	NCI109	Mutagenicity
GraphCL	Classifier Only	$30.50_{\pm 1.16}$	$62.89_{\pm 2.19}$	$62.49_{\pm 1.95}$	$61.68_{\pm 0.93}$	$66.62_{\pm 1.87}$
	GraphPrompt	$27.83_{\pm 1.61}$	$64.33_{\pm 1.79}$	$63.19_{\pm 1.71}$	$62.18_{\pm 0.48}$	$67.62_{\pm 0.65}$
	ALL-in-one	$25.92_{\pm 0.55}$	$66.54_{\pm 1.82}$	$57.52_{\pm 2.61}$	$62.74_{\pm 0.78}$	$63.43_{\pm 2.53}$
	GPF	$30.08_{\pm 1.25}$	$64.54_{\pm 2.22}$	$62.66_{\pm 1.83}$	$62.29_{\pm 0.90}$	$66.54_{\pm 1.85}$
	GPF-plus	$31.00_{\pm 1.50}$	$67.26_{\pm 2.29}$	$64.56_{\pm 1.10}$	$62.84_{\pm 0.22}$	$66.82_{\pm 1.63}$
	EdgePrompt	$29.50_{\pm 1.57}$	$64.16_{\pm 2.13}$	$63.05_{\pm 2.11}$	$62.59_{\pm 0.93}$	$66.87_{\pm 1.88}$

Table 2: Performance on graph-level benchmarks (5-seed avg.). Bold = best, underline = second best.

Variant	GraphCL			SimGRACE
Variant	Cora	CiteSeer	Avg. $\Delta$	Cora	CiteSeer	Avg. $\Delta$
SRP (full)	$\bm{68.1_{\pm 2.3}}$	$\bm{49.0_{\pm 2.3}}$	$-0.0$	$\bm{67.2_{\pm 7.3}}$	$\bm{51.4_{\pm 5.2}}$	$-0.0$
SRP-NM (w/o null channel)	$63.4_{\pm 2.8}$	$41.1_{\pm 3.8}$	$-6.3$	$60.3_{\pm 5.9}$	$43.5_{\pm 2.9}$	$-7.4$
SRP-NS (w/o spectral channel)	$64.4_{\pm 2.1}$	$45.6_{\pm 3.2}$	$-3.6$	$63.5_{\pm 6.6}$	$47.7_{\pm 4.3}$	$-3.7$
SRP-NR (w/o reverse)	$64.7_{\pm 2.7}$	$48.0_{\pm 4.0}$	$-2.2$	$63.4_{\pm 4.4}$	$49.9_{\pm 4.9}$	$-2.7$
SRP-Bi (bidirectional)	$64.7_{\pm 3.3}$	$47.9_{\pm 3.3}$	$-2.3$	$63.0_{\pm 4.5}$	$50.0_{\pm 4.8}$	$-2.8$

Table 3: Ablation Study of Different Variants on GraphCL and SimGRACE.

Figure 3: Label Sensitivity and Parameter Sensitivity of NS-PCA and bottleneck rank.

Figure 4: Convergence speed of different methods.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Nodes	#Edges	#Feat.	#Classes	Task
Cora	2,708	10,556	1,433	7	Node
CiteSeer	3,327	9,104	3,703	6	Node
PubMed	19,717	88,648	500	3	Node
ogbn-arxiv	169,343	1,166,243	128	40	Node
Flickr	89,250	899,756	500	7	Node
Cornell	183	298	1703	5	Node

Appendix

Table 4: Node-level benchmark datasets. #Feat. is the raw input feature dimension; #Classes is the number of downstream classes.

Dataset	#Graphs	Avg. Nodes	Avg. Edges	#Feat.	#Classes	Task
ENZYMES	600	32.63	124.27	3	6	Graph
DD	1,178	284.32	1,431.32	89	2	Graph
NCI1	4,110	29.87	64.60	37	2	Graph
NCI109	4,127	29.68	64.26	38	2	Graph
Mutagenicity	4,337	30.32	61.54	14	2	Graph

Appendix

Table 5: Graph-level benchmark datasets (TUDataset). Avg. Nodes and Avg. Edges are computed over the graph collection.

Backbone	Dataset	Classifier-only	GraphLoRA	SRP
GraphGPS	Cora	$48.17\pm 6.12$	$64.33\pm 6.88$	$\mathbf{66.62\pm 5.27}$
GraphGPS	CiteSeer	$39.13\pm 4.57$	$46.80\pm 3.38$	$\mathbf{58.74\pm 5.01}$
GraphGPS	NCI1	$63.32\pm 1.51$	$62.49\pm 1.93$	$\mathbf{66.81\pm 2.13}$
GraphGPS	ENZYMES	$35.08\pm 2.82$	$45.00\pm 3.25$	$\mathbf{47.75\pm 2.71}$
Graphormer	Cora	$51.13\pm 4.04$	$62.88\pm 5.49$	$\mathbf{65.69\pm 5.97}$
Graphormer	CiteSeer	$39.01\pm 1.74$	$47.13\pm 1.72$	$\mathbf{54.89\pm 1.60}$

Appendix

Table 6: Performance of SRP with GraphGPS and Graphormer backbones. Results are reported as mean

\pm

standard deviation over five runs.

Mask setting	Cora	CiteSeer	NCI1	ENZYMES
$\gamma=0.001$	64.55	47.64	66.63	36.42
$\gamma=0.1$	64.88	48.03	66.52	36.92
$\gamma=1$	67.80	48.81	66.96	36.50
$\gamma=10$	68.10	48.99	66.80	36.67
$\gamma=100$	68.06	48.87	66.84	36.25
Hard threshold (STE)	67.95	48.90	66.89	35.67

Appendix

Table 7: Sensitivity to the temperature coefficient

\gamma

Dataset	Strong- $k$	Random- $k$	Weak- $k$	Hard (STE)	Soft $\gamma=10$
Cora	64.25	64.44	67.80	67.95	68.10
CiteSeer	47.10	47.46	48.78	48.90	48.99
NCI1	66.42	66.45	66.87	66.89	66.80

Appendix

Table 8: Matched-cardinality comparison of different spectral selections.

Dataset	Classifier-only	GraphLoRA	SRP	SRP vs. GraphLoRA
Cornell	$32.22\pm 8.35$	$45.56\pm 14.34$	$\mathbf{51.67\pm 5.72}$	$+6.11$
Squirrel	$23.08\pm 1.98$	$23.06\pm 1.40$	$\mathbf{25.12\pm 1.47}$	$+2.06$
Chameleon	$31.74\pm 1.88$	$31.43\pm 1.44$	$\mathbf{33.58\pm 2.76}$	$+2.15$

Appendix

Table 9: Performance on heterophilic graph benchmarks.

Dataset	GPF	SRP
Cora	0.121	0.077
CiteSeer	0.131	0.084
PubMed	0.678	0.508
ENZYMES	0.221	0.292
DD	0.191	0.136
NCI1	0.342	0.297

Appendix

Table 10: Per-epoch runtime in seconds. Lower is better.

Graph Neural Networks

Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.

See all 726 papers

May 22, 2025

Freeze, Prompt, and Adapt: A Framework for Source-free Unsupervised GNN Prompting

Prompt tuning has become a key mechanism for adapting pre-trained Graph Neural Networks (GNNs) to new downstream tasks. However, existing approaches are predominantly supervised, relying on labeled data to optimize the prompting parameters and typically fine-tuning a task-specific prediction head -- practices that undermine the promise of parameter-efficient adaptation. We propose Unsupervised Graph Prompting Problem (UGPP), a challenging new setting where the pre-trained GNN is kept entirely frozen, labels on the target domain are unavailable, the source data is inaccessible, and the target distribution exhibits covariate shift. To address this, we propose UGPrompt, the first fully unsupervised GNN prompting framework. UGPrompt leverages consistency regularization and pseudo-labeling to train a prompting function, complemented with diversity and domain regularization to mitigate class imbalance and distribution mismatch. Our extensive experiments demonstrate that UGPrompt consistently outperforms state-of-the-art supervised prompting methods with access to labeled data, demonstrating the viability of unsupervised prompting as a practical adaptation paradigm for GNNs.

Peyman Baghershahi, Sourav Medya

Department of Computer Science University of Illinois Chicago

Source-Free Domain Adaptation Unsupervised Domain Adaptation Prompt Engineering Parameter-Efficient Fine-Tuning

Ask PDF

Apr 1, 2026

A Cross-graph Tuning-free GNN Prompting Framework

GNN prompting aims to adapt models across tasks and graphs without requiring extensive retraining. However, most existing graph prompt methods still require task-specific parameter updates and face the issue of generalizing across graphs, limiting their performance and undermining the core promise of prompting. In this work, we introduce a Cross-graph Tuning-free Prompting Framework (CTP), which supports both homogeneous and heterogeneous graphs, can be directly deployed to unseen graphs without further parameter tuning, and thus enables a plug-and-play GNN inference engine. Extensive experiments on few-shot prediction tasks show that, compared to SOTAs, CTP achieves an average accuracy gain of 30.8% and a maximum gain of 54%, confirming its effectiveness and offering a new perspective on graph prompt learning.

Yaqi Chen, Shixun Huang, Ryan Twemlow +6

University of Wollongong, Northfields Avenue, Gwynneville, 2500, NSW, Australia · Wuhan University, No. 299, Luojia Hill, Wuhan, 430072, Hubei, China

Graph Neural Networks Prompt Engineering

Ask PDF

Jun 2, 2026

Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective

Graph Foundation Models (GFMs), built upon the Pre-training and Adaptation paradigm, have emerged as a research hotspot in graph learning. For GNN-based GFMs, graph prompt tuning has become the prevailing adaptation method for downstream tasks. Although recent methods explain why graph prompt tuning works, how to rigorously measure its adaptation capacity remains an open problem. Addressing this problem is critical for understanding the capability limits of graph prompt tuning and for developing more powerful adaptation methods. In this paper, we propose Prismatic Space Theory (PS-Theory), a novel mathematical framework to quantify the capacity of adaptation methods, while focusing on establishing the upper bound for the adaptation capacity of graph prompt tuning. Building upon the proposed PS-Theory, we further introduce Message Tuning for GFMs (MTG), a lightweight approach that injects a small set of learnable message prototypes into each layer of the GNN backbone to adaptively guide message fusion without updating pre-trained weights. Through our PS-Theory, we prove that the adaptation capacity of MTG can exceed the theoretical upper bound of graph prompt tuning. Extensive experiments demonstrate that MTG consistently outperforms graph prompt baselines across diverse benchmark datasets, providing strong empirical support for our theoretical findings.

Yancheng Chen, Dun Ma, Shuai Zhang +6

Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Institute of Information Engineering, Chinese Academy of Sciences +4

Prompt Tuning Graph Foundation Models Parameter-Efficient Fine-Tuning

Ask PDF

Abstract

Graph Neural Networks

Explore similar work

Freeze, Prompt, and Adapt: A Framework for Source-free Unsupervised GNN Prompting

A Cross-graph Tuning-free GNN Prompting Framework

Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective