Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA

Ankit Grover; Lodovico Giaretta; Rémi Bourgerie; Sarunas Girdzijauskas

cs.LGApr 1, 2026

Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQA

Authors: Ankit Grover, Lodovico Giaretta, Rémi Bourgerie, Sarunas Girdzijauskas

Organizations: KTH Royal Institute of Technology, Stockholm, Sweden · RISE Research Institutes of Sweden, Stockholm, Sweden

Abstract

The integration of Graph Neural Networks (GNNs) with Large Language Models (LLMs) has emerged as a promising paradigm for Graph Question Answering (GraphQA). However, effective methods for encoding complex structural information into the LLM's latent space remain an open challenge. Current state-of-the-art architectures, such as G-Retriever, typically rely on standard GNNs and aggressive mean pooling to compress entire graph substructures into a single token, creating a severe information bottleneck. This work mitigates this bottleneck by investigating two orthogonal strategies: (1) increasing the bandwidth of the graph-to-LLM interface via multi-token pooling, and (2) enhancing the semantic quality of the graph encoder via global attention mechanisms. We evaluate a suite of hierarchical pruning and clustering-based pooling operators including Top-k, SAGPool, DiffPool, MinCutPool, and Virtual Node Pooling (VNPool) to project graph data into multiple learnable tokens. Empirically, we demonstrate that while pooling introduces significant instability during soft prompt tuning, the application of Low-Rank Adaptation (LoRA) effectively stabilizes specific hierarchical projections (notably VNPool and pruning methods), though dense clustering operators remain challenging. This stabilization allows compressed representations to rival full-graph baselines (achieving ~73% Hit@1 on WebQSP). Conceptually, we demonstrate that a Graph Transformer with VNPool implementation functions structurally as a single-layer Perceiver IO encoder. Finally, we adapt the FandE (Features and Edges) Score to the generative GraphQA domain. Our analysis reveals that the GraphQA benchmark suffers from representational saturation, where target answers are often highly correlated with isolated node features. The implementation is available at https://github.com/Agrover112/G-Retriever/tree/all_good/

Figures & tables

Figure 1: The proposed Multi-Token Graph Pooling Framework. Dotted lines indicate alternate pooling methods for generating tokens.Current GraphRAG systems compress subgraphs into a single token by mean pooling or use node-level positional embeddings as tokens. Using hierarchical Clustering and Pruning operators to project the graph into a rich sequence of $k$ soft tokens (yellow squares with colored borders) preserve structural fidelity better than simple linearization while remaining more efficient than full textualization.

Panel of Figure 2 — Figure 2: System Architecture Comparison. We contrast two training paradigms: (a) keeping the LLM frozen using Soft Prompt Tuning versus (b) adapting the LLM using LoRA. Our results show that dense pooling methods (VNPool) fail to converge under (a) due to optimization difficulties when using a frozen backbone, but achieve high performance and stability under (b).

Method	ExplaGraphs	WebQSP
All Tokens	88.3%	56.7%
All Tokens + Textualization	87.3%	71.4%

Table 1: Impact of Textualization on GraphQA Performance. Removing textual scaffolding improves performance on the reasoning-heavy ExplaGraphs but causes a catastrophic collapse on the entity-dense WebQSP.

Method	ExplaGraphs	WebQSP
Baselines
All Tokens	$80.6\pm 9.6$	$71.3\pm 1.1$
Mean Pooling	$\mathbf{86.0\pm 2.0}$	$70.7\pm 0.7$
Rand- $k$	$63.1\pm 3.1$	$71.6\pm 0.5$
Pooling Operators
Top- $k$	$80.1\pm 13.1$	$\mathbf{71.8\pm 1.5}$

Table 2: Prompt Tuning (PT) Performance. Mean

\pm

STD. High-variance and lower performance in complex operators (VNPool, DiffPool) demonstrate the optimization challenges of PT compared to simpler baselines.

	ExplaGraphs				WebQSP
Method	r=2, a=4	r=4, a=8	r=8, a=16	r=16, a=32	r=2, a=4	r=4, a=8	r=8, a=16	r=16, a=32
Baselines
All Tokens	$70.2\pm 21.2$	$73.8\pm 18.5$	$84.7\pm 7.5$	$72.6\pm 16.9$	$72.8\pm 1.3$	$\mathbf{73.6\pm 0.6}$	$\mathbf{73.8\pm 0.8}$	$73.0\pm 0.7$
Mean Pooling	$\mathbf{87.2\pm 2.6}$	$\mathbf{86.2\pm 2.7}$	$\mathbf{87.5\pm 1.8}$	$\mathbf{87.5\pm 1.8}$	$71.9\pm 0.7$	$71.1\pm 0.8$	$71.5\pm 2.8$	$71.9\pm 2.4$
Rand- $k$	$53.4\pm 1.2$	$52.1\pm 3.7$	$66.1\pm 16.0$	$52.9\pm 1.9$	$\mathbf{73.2\pm 0.6}$	$72.9\pm 0.8$	$72.8\pm 1.0$	$\mathbf{74.8\pm 1.0}$
Pooling Strategies

Table 3: LoRA fine-tuning across varying ranks (

r

) and alphas (

a

). The use of LoRA adapters provides the necessary interface flexibility to map hierarchical graph signals which otherwise fail to converge into the LLM’s latent space.

Feature-Only ( $S_{F}$ )
Encoder	ExplaGraphs	WebQSP
Transformer	$\mathbf{86.2\pm 2.2}$	$\mathbf{71.2\pm 0.6}$
MLP	$83.5\pm 4.2$	$70.3\pm 0.3$
Edge-Only ( $S_{E}$ )
Graph Transformer	$\mathbf{86.0\pm 2.0}$	$70.7\pm 0.7$
GCN	$85.2\pm 1.9$	$\mathbf{71.7\pm 0.6}$

Table 4: Performance of single-token baselines under soft Prompt Tuning. Note the high parity between the Feature-Only and Structure-Aware encoders.

Model Pair ( $S_{F}$ and $S_{E}$ )	ExplaGraphs	WebQSP
MLP and GCN	0.57	0.49
Transformer and GT	0.64	0.50

Table 5: FandE Scores for the models in 6 Higher scores indicate greater redundancy.

	ExplaGraphs		WebQSP
Simple Models	$S_{\text{F}}(\text{MLP})$	$\neg S_{\text{F}}(\text{MLP})$	$S_{\text{F}}(\text{MLP})$	$\neg S_{\text{F}}(\text{MLP})$
$S_{\text{E}}(\text{GCN})$	315	85	793	97
$\neg S_{\text{E}}(\text{GCN})$	30	124	118	620
Complex Models	$S_{\text{F}}(\text{Transformer})$	$\neg S_{\text{F}}(\text{Transformer})$	$S_{\text{F}}(\text{Transformer})$	$\neg S_{\text{F}}(\text{Transformer})$
$S_{\text{E}}(\text{GT})$	352	33	807	83
$\neg S_{\text{E}}(\text{GT})$	56	113	129	609

Table 6: We explicitly compare Feature-Only (

S_{F}

) vs. Structure-Aware (

S_{E}

) models. The high values in the top-left quadrants (

S_{F}\cap S_{E}

) confirm high redundancy.

LLM Reasoning with Graphs

Large language model reasoning with graphs uses structured relationships to support multi-step inference, evidence retrieval, and planning.

See all 210 papers

On the Token Collapse Phenomenon in Textual Graph Soft Prompting
Oct 22, 2026Andrea Giuseppe Di Francesco, Rishabh Jain, Franco Maria Nardini +3Graph Prompt Learning LLM Reasoning with Graphs
Sapienza University of Rome, Rome, Italy · ISTI-CNR, Pisa, Italy · University of Cambridge, Cambridge, UK +1
Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding
May 5, 2026Zhongjian Zhang, Yue Yu, Mengmei Zhang +3LLM Evaluation LLM Reasoning with Graphs
Beijing University of Posts and Telecommunications · China Telecom Bestpay · Beihang University
When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models
Jun 2, 2026Ding Zhang, Runtao Zhou, Wenqing Zheng +3LLM Interpretability Mechanistic Interpretability
University of Virginia · Capital One

Oct 22, 2026J/K move · Enter open · S save

On the Token Collapse Phenomenon in Textual Graph Soft Prompting

Andrea Giuseppe Di Francesco, Rishabh Jain, Franco Maria Nardini, Nicola Tonellotto+2

Sapienza University of Rome, Rome, Italy · ISTI-CNR, Pisa, Italy · University of Cambridge, Cambridge, UK+1

Graph Prompt Learning LLM Reasoning with Graphs Knowledge Graph Question Answering

Open paper AskPDF arXiv

Recent methods for injecting graph structure into LLM prompts include graph soft prompting, in which a graph neural network (GNN) learns graph-specific tokens that capture relational information among entities in textual prompts. G-Retriever combines graph soft prompts with a verbalized description of the same graph to strengthen the textual prompt signal. We conduct an in-depth analysis of G-Retriever and uncover a pathological behavior we term token collapse. Across three benchmarks, three graph encoders, two LLM families, and both frozen and LoRA-adapted models, G-Retriever's graph tokens become almost indistinguishable across inputs. We provide evidence that the same failure is inherited by subsequent methods, even with multi-token pooling. Under the standard setup, swapping a token with the dataset mean, a token from another example, or one derived from a corrupted graph produces no resolved answer-level effect in accuracy. Moreover, a learned constant prompt that never accesses the graph matches or surpasses G-Retriever on all three benchmarks. These findings indicate that, rather than conveying graph content as intended, the collapsed token functions primarily as a generic soft prompt. We also examine conditions linked to this collapse and propose a two-stage procedure that yields non-collapsed tokens while maintaining competitive performance in most cases. However, token variability alone does not prove that the model uses graph information: via interventions and decoding probes, we show that depending on the configuration, a token can reflect the question, an answer prior, or graph-derived content that the LLM may or may not use. Overall, our results offer a practical approach to evaluating graph-token alignment rather than inferring it from task accuracy alone. Code to reproduce our experiments is available at https://github.com/difra100/Token_Collapse_KGQA.

Abstract

LLM Reasoning with Graphs

Explore similar work

On the Token Collapse Phenomenon in Textual Graph Soft Prompting

Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding

When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models