Riemannian Graph Tokenizer for Structural Knowledge Transfer

Qimin Zhou; Haibo Liu; Yujie Wang; Li Sun; Chuan Shi

cs.LGOct 11, 2026

Riemannian Graph Tokenizer for Structural Knowledge Transfer

Authors: Qimin Zhou, Haibo Liu, Yujie Wang, Li Sun, Chuan Shi

Organizations: Beijing University of Posts and Telecommunications, Beijing, China

Abstract

Foundation models are at the forefront of artificial intelligence. A tokenizer, converting the raw input into discrete representations that the model can understand, plays an important role to the success of foundation models. Unlike the text tokenizer that is well studied in large language models, graph tokenizer is still at its early stage, facing the challenges of tackling the non-Euclidean structures and capturing the structural semantics. How to design a graph tokenizer for structural knowledge transfer? To this end, we propose a Riemannian Graph Tokenizer (RGT) that bridges the structural knowledge and quantized representations to support cross-domain structural knowledge transfer. The connection is established by Riemannian geometry. Specifically, we first define the geometric vocabulary (trees, cycles and sequences), which captures fundamental structural patterns and reflects the intrinsic geometry of graph. Second, we construct a Riemannian quantizer with Riemannian Straight-Through Estimator to tokenise graph structures across multiple domains into discrete tokens. To ensure consistency and transferability across diverse geometric spaces, RGT further incorporates a geometry-aligned decoder that projects manifold-specific tokens into a unified tangent space. The theoretical analysis and geometric interpretations are provided to support the effectiveness of our proposed method. Extensive experiments across diverse datasets demonstrate that RGT significantly enhances structural knowledge transferability across graph domains.

Figures & tables

Figure 1. Overall architecture of the proposed Riemannian Graph Tokenizer.

	Node Classification Results
Method	Pubmed		Github		USA		Europe
	ACC	F1	ACC	F1	ACC	F1	ACC	F1
DGI	75.66 $\pm$ 0.86	73.66 $\pm$ 1.46	74.10 $\pm$ 0.15	42.56 $\pm$ 0.05	38.90 $\pm$ 1.57	33.06 $\pm$ 3.98	24.48 $\pm$ 0.97	9.83 $\pm$ 0.31
GraphMAE	79.47 $\pm$ 0.45	78.75 $\pm$ 0.40	67.57 $\pm$ 2.76	63.14 $\pm$ 1.81	32.78 $\pm$ 6.63	21.42 $\pm$ 5.39	31.28 $\pm$ 2.10	20.96 $\pm$ 3.47
OFA	52.23 $\pm$ 6.45	50.35 $\pm$ 3.34	-	-	-	-	-	-
ZeroG	69.12 $\pm$ 0.89	68.35 $\pm$ 1.23	-	-	-	-	-	-

Table 1. Cross-domain transfer learning performance of node classification on four datasets.

	Link Prediction Results
Method	Pubmed		Github		USA		Europe
	AUC	AP	AUC	AP	AUC	AP	AUC	AP
DGI	56.19 $\pm$ 0.19	61.56 $\pm$ 0.35	55.75 $\pm$ 8.68	53.66 $\pm$ 7.79	56.17 $\pm$ 2.31	48.39 $\pm$ 1.35	58.82 $\pm$ 5.38	51.38 $\pm$ 3.30
GraphMAE	84.40 $\pm$ 0.38	82.36 $\pm$ 0.36	65.94 $\pm$ 0.18	67.51 $\pm$ 0.14	65.81 $\pm$ 6.51	67.36 $\pm$ 1.27	45.58 $\pm$ 1.73	46.28 $\pm$ 0.81
OFA	85.32 $\pm$ 1.67	83.28 $\pm$ 1.39	-	-	-	-	-	-
OpenGraph	65.04 $\pm$ 0.98	64.31 $\pm$ 3.98	78.34 $\pm$ 0.92	79.76 $\pm$ 1.20	77.89 $\pm$ 2.34	79.32 $\pm$ 1.32	72.34 $\pm$ 1.51	70.32 $\pm$ 1.22

Table 2. Cross-domain transfer learning performance of link prediction on four datasets.

	Node Classification Results
Method	Pubmed		Github		USA		Europe
	ACC	F1	ACC	F1	ACC	F1	ACC	F1
DGI	54.09 $\pm$ 1.92	45.66 $\pm$ 7.34	44.26 $\pm$ 0.14	44.72 $\pm$ 3.03	26.69 $\pm$ 2.94	13.58 $\pm$ 4.84	25.62 $\pm$ 0.00	10.20 $\pm$ 0.00
GraphMAE	47.03 $\pm$ 7.27	42.31 $\pm$ 9.40	55.25 $\pm$ 2.03	46.78 $\pm$ 1.74	23.69 $\pm$ 1.70	14.73 $\pm$ 2.83	27.07 $\pm$ 3.20	18.54 $\pm$ 2.30
OFA	41.25 $\pm$ 0.94	38.45 $\pm$ 2.96	-	-	-	-	-	-
ZeroG	74.52 $\pm$ 0.52	74.15 $\pm$ 0.15	-	-	-	-	-	-

Table 3. One-shot learning performance of node classification on four datasets.

	Node Classification Results
Method	Pubmed		Github		USA		Europe
	ACC	F1	ACC	F1	ACC	F1	ACC	F1
DGI	64.07 $\pm$ 8.43	61.88 $\pm$ 4.84	75.56 $\pm$ 1.37	51.21 $\pm$ 7.40	35.84 $\pm$ 1.28	28.02 $\pm$ 3.20	25.62 $\pm$ 0.00	10.20 $\pm$ 0.00
GraphMAE	72.09 $\pm$ 1.86	71.62 $\pm$ 2.28	57.12 $\pm$ 2.16	52.39 $\pm$ 1.97	31.42 $\pm$ 5.58	21.11 $\pm$ 4.59	26.74 $\pm$ 1.59	14.20 $\pm$ 3.00
OFA	41.26 $\pm$ 4.36	39.67 $\pm$ 5.35	-	-	-	-	-	-
ZeroG	72.26 $\pm$ 3.21	69.48 $\pm$ 2.39	-	-	-	-	-	-

Table 4. Five-shot learning performance of node classification on four datasets.

Trees	Cycles	Sequences	Cora	Citeseer	Pubmed	USA
$\mathcal{H}^{32}_{-1}$	$\mathcal{S}^{32}_{1}$	$\mathcal{R}^{32}_{0}$	80.72 $\pm$ 0.97	66.40 $\pm$ 1.53	76.26 $\pm$ 0.73	56.72 $\pm$ 2.51
$\mathcal{H}^{32}_{-1}$	$\mathcal{R}^{32}_{0}$	$\mathcal{S}^{32}_{1}$	78.53 $\pm$ 0.78	64.98 $\pm$ 2.22	74.52 $\pm$ 0.81	52.69 $\pm$ 1.47
$\mathcal{S}^{32}_{1}$	$\mathcal{H}^{32}_{-1}$	$\mathcal{R}^{32}_{0}$	79.42 $\pm$ 0.83	65.77 $\pm$ 1.58	74.34 $\pm$ 0.82	52.77 $\pm$ 1.05
$\mathcal{S}^{32}_{1}$	$\mathcal{R}^{32}_{0}$	$\mathcal{H}^{32}_{-1}$	77.98 $\pm$ 1.21	64.58 $\pm$ 2.56	73.88 $\pm$ 0.89	51.13 $\pm$ 1.35
$\mathcal{R}^{32}_{0}$	$\mathcal{S}^{32}_{1}$	$\mathcal{H}^{32}_{-1}$	78.32 $\pm$ 0.53	65.64 $\pm$ 1.20	75.26 $\pm$ 0.63	53.71 $\pm$ 1.24
$\mathcal{R}^{32}_{0}$	$\mathcal{H}^{32}_{-1}$	$\mathcal{S}^{32}_{1}$	76.85 $\pm$ 1.21	63.69 $\pm$ 1.32	74.26 $\pm$ 0.93	52.21 $\pm$ 1.37

Table 5. Link prediction results of geometric ablation are reported in terms of AUC.

Panel of Figure 2 — Figure 2. Hyperparameter sensitivity analysis of codebook size.

Panel of Figure 3 — Figure 3. Heatmap about $\lambda$ and $\epsilon$ and radar plot of p-value.

Seed	RiemannGFM (Acc)	RGT (Acc)	RiemannGFM (F1)	RGT (F1)
Seed 0	73.94 $\pm$ 5.66	76.22 $\pm$ 0.80	74.25 $\pm$ 4.25	76.14 $\pm$ 0.77
Seed 1	71.52 $\pm$ 5.44	75.94 $\pm$ 0.88	72.11 $\pm$ 4.01	75.89 $\pm$ 0.73
Seed 2	72.03 $\pm$ 5.70	76.08 $\pm$ 0.79	73.09 $\pm$ 4.37	76.07 $\pm$ 0.80
Seed 3	74.11 $\pm$ 5.61	76.35 $\pm$ 0.85	73.64 $\pm$ 4.40	76.25 $\pm$ 0.82
Seed 4	69.77 $\pm$ 5.58	75.71 $\pm$ 0.81	72.87 $\pm$ 4.16	75.83 $\pm$ 0.75
Mean	72.67 $\pm$ 5.60	76.06 $\pm$ 0.83	73.19 $\pm$ 4.24	76.04 $\pm$ 0.77

Table 6. Cross-domain transfer learning performance of node classification on Pubmed (Bold: best).

Panel of Figure 4 — Figure 4. Ablation study on key components.

Pretraining	Method	Citeseer	Cora	USA	Brazil
Flickr	OpenGraph	73.17 $\pm$ 0.23	77.46 $\pm$ 0.23	53.16 $\pm$ 0.98	61.16 $\pm$ 0.19
	RiemannGFM	98.27 $\pm$ 1.38	96.84 $\pm$ 0.24	93.40 $\pm$ 0.84	84.03 $\pm$ 0.26
	RGT (Ours)	99.11 $\pm$ 0.01	97.81 $\pm$ 0.32	93.41 $\pm$ 0.02	84.92 $\pm$ 0.50
AComp	OpenGraph	72.95 $\pm$ 0.28	78.47 $\pm$ 0.25	79.79 $\pm$ 0.81	68.31 $\pm$ 0.14
	RiemannGFM	98.50 $\pm$ 0.01	96.59 $\pm$ 0.00	92.85 $\pm$ 0.13	84.32 $\pm$ 0.13
	RGT (Ours)	98.87 $\pm$ 0.03	97.91 $\pm$ 0.17	92.90 $\pm$ 0.02	84.89 $\pm$ 0.11

Table 7. Cross-domain link prediction performance on different pretraining datasets.

Figure 5. Cross-domain Node Classification Performance on Web-Scale Graphs.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6. Cross-Domain Transfer of Geometric Vocabulary.

Hyperparameter	Pretraining	Fine-tuning
Learning Rate	0.001	0.01
Epochs	3	120
Batch Size	32	64
Dropout	0.1	0.1
Drop Edge Rate	–	0.2
Drop Feature Rate	–	0.3

Appendix

Table 8. Hyperparameters for RGT in pretraining and fine-tuning

	Node Classification Results
Method	Cora		Citeseer		Brazil
	ACC	F1	ACC	F1	ACC	F1
DGI	80.62 $\pm$ 1.63	80.67 $\pm$ 1.97	69.67 $\pm$ 0.42	62.92 $\pm$ 1.81	31.78 $\pm$ 1.62	29.15 $\pm$ 4.16
GraphMAE	79.77 $\pm$ 0.34	79.82 $\pm$ 0.23	50.20 $\pm$ 7.07	44.97 $\pm$ 8.20	32.14 $\pm$ 5.83	21.14 $\pm$ 7.72
OFA	56.92 $\pm$ 3.09	54.21 $\pm$ 2.02	49.68 $\pm$ 1.36	50.22 $\pm$ 2.13	-	-
ZeroG	59.20 $\pm$ 1.07	55.32 $\pm$ 1.34	52.34 $\pm$ 0.89	55.67 $\pm$ 1.22	-	-

Appendix

Table 9. Cross-domain transfer learning performance of node classification on three datasets.

	Link Prediction Results
Method	Cora		Citeseer		Brazil
	AUC	AP	AUC	AP	AUC	AP
DGI	66.06 $\pm$ 1.66	65.64 $\pm$ 1.91	82.81 $\pm$ 1.12	84.72 $\pm$ 1.15	64.09 $\pm$ 5.91	59.09 $\pm$ 7.17
GraphMAE	54.86 $\pm$ 0.20	57.87 $\pm$ 1.58	79.18 $\pm$ 1.02	75.51 $\pm$ 0.94	74.00 $\pm$ 2.81	71.86 $\pm$ 4.04
OFA	70.84 $\pm$ 2.01	71.34 $\pm$ 1.89	75.61 $\pm$ 2.34	76.13 $\pm$ 1.26	-	-
OpenGraph	74.03 $\pm$ 0.97	73.21 $\pm$ 0.84	74.45 $\pm$ 0.46	76.32 $\pm$ 0.76	65.31 $\pm$ 1.54	64.39 $\pm$ 1.65

Appendix

Table 10. Cross-domain transfer learning performance of link prediction on three datasets.

	Node Classification Results
Method	Cora		Citeseer		Brazil
	ACC	F1	ACC	F1	ACC	F1
DGI	39.21 $\pm$ 1.70	34.33 $\pm$ 2.09	34.13 $\pm$ 3.08	31.41 $\pm$ 1.11	29.92 $\pm$ 3.24	21.75 $\pm$ 2.54
GraphMAE	51.86 $\pm$ 8.81	50.90 $\pm$ 8.49	41.24 $\pm$ 8.71	36.07 $\pm$ 9.75	26.56 $\pm$ 3.41	17.53 $\pm$ 3.34
OFA	57.35 $\pm$ 3.09	55.92 $\pm$ 2.85	36.89 $\pm$ 3.98	38.48 $\pm$ 5.76	-	-
ZeroG	63.59 $\pm$ 1.75	63.24 $\pm$ 1.22	60.34 $\pm$ 1.21	58.24 $\pm$ 1.67	-	-

Appendix

Table 11. One-shot learning performance of node classification on three datasets.

	Node Classification Results
Method	Cora		Citeseer		Brazil
	ACC	F1	ACC	F1	ACC	F1
DGI	64.54 $\pm$ 4.19	65.27 $\pm$ 4.90	48.03 $\pm$ 3.56	44.73 $\pm$ 3.02	30.62 $\pm$ 3.24	29.29 $\pm$ 2.76
GraphMAE	72.85 $\pm$ 1.61	71.80 $\pm$ 1.74	47.74 $\pm$ 4.99	40.88 $\pm$ 6.81	28.57 $\pm$ 6.47	20.81 $\pm$ 5.85
OFA	65.23 $\pm$ 3.19	62.72 $\pm$ 2.35	43.25 $\pm$ 3.78	40.21 $\pm$ 4.78	-	-
ZeroG	63.59 $\pm$ 1.75	63.20 $\pm$ 1.52	55.92 $\pm$ 5.21	52.74 $\pm$ 4.16	-	-

Appendix

Table 12. Five-shot learning performance of node classification on three datasets.

		Testing Datasets
Pre-training	Method	Citeseer	Cora	USA	Brazil
Flickr	OpenGraph	58.86 $\pm$ 0.89	59.43 $\pm$ 1.60	25.97 $\pm$ 4.98	17.04 $\pm$ 4.12
	Riemannian	65.74 $\pm$ 2.07	77.62 $\pm$ 0.78	51.34 $\pm$ 1.61	23.85 $\pm$ 9.29
	RGT (Ours)	66.82 $\pm$ 0.75	80.34 $\pm$ 0.70	54.20 $\pm$ 1.36	28.46 $\pm$ 4.62
AComp	OpenGraph	46.53 $\pm$ 1.6	50.68 $\pm$ 1.29	25.88 $\pm$ 3.84	15.56 $\pm$ 3.99
	Riemannian	66.4 $\pm$ 1.76	80.02 $\pm$ 0.87	48.66 $\pm$ 1.49	30.38 $\pm$ 3.28

Appendix

Table 13. Cross-domain node classification performance on different pre-training datasets.

Graph Representation Learning

Graph representation learning develops methods for encoding nodes, edges, and whole graphs for prediction, comparison, and transfer.

See all 296 papers

Graph Tokenization for Bridging Graphs and Transformers
Mar 11, 2026Zeyuan Guo, Enmao Diao, Cheng Yang +1Graph Representation Learning Transformer
Beijing University of Posts and Telecom., Beijing 100876, China · DreamSoul
Graphlets as Building Blocks for Structural Vocabulary in Knowledge Graph Foundation Models
May 7, 2026Kossi Amouzouvi, Robert Wardenga, Jens Lehmann +1Graph Representation Learning Graph Foundation Models
ScaDS.AI Dresden/Leipzig, Technische Universität Dresden, Dresden, Germany · alphaspeech - c/o alpha NT GmbH, Dresden, Germany · Amazon, Technische Universität Dresden, Dresden, Germany +1
Riemannian Geometry Speaks Louder Than Words: From Graph Foundation Model to Next-Generation Graph Intelligence
Mar 23, 2026Philip S. Yu, Li SunGraph Foundation Models Graph Representation Learning
University of Illinois Chicago, Chicago, USA · Beijing University of Posts and Telecommunications, Beijing, China

Mar 11, 2026J/K move · Enter open · S save

Graph Tokenization for Bridging Graphs and Transformers

Zeyuan Guo, Enmao Diao, Cheng Yang, Chuan Shi

Beijing University of Posts and Telecom., Beijing 100876, China · DreamSoul

Graph Representation Learning Transformer

Open paper AskPDF arXiv

The success of large pretrained Transformers is closely tied to tokenizers, which convert raw input into discrete symbols. Extending these models to graph-structured data remains a significant challenge. In this work, we introduce a graph tokenization framework that generates sequential representations of graphs by combining reversible graph serialization, which preserves graph information, with Byte Pair Encoding (BPE), a widely adopted tokenizer in large language models (LLMs). To better capture structural information, the graph serialization process is guided by global statistics of graph substructures, ensuring that frequently occurring substructures appear more often in the sequence and can be merged by BPE into meaningful tokens. Empirical results demonstrate that the proposed tokenizer enables Transformers such as BERT to be directly applied to graph benchmarks without architectural modifications. The proposed approach achieves state-of-the-art results on 14 benchmark datasets and frequently outperforms both graph neural networks and specialized graph transformers. This work bridges the gap between graph-structured data and the ecosystem of sequence models. Our code is available at bluehere.

Abstract

Graph Representation Learning

Explore similar work

Graph Tokenization for Bridging Graphs and Transformers

Graphlets as Building Blocks for Structural Vocabulary in Knowledge Graph Foundation Models

Riemannian Geometry Speaks Louder Than Words: From Graph Foundation Model to Next-Generation Graph Intelligence