Degraded but Not Entirely Ineffective: PE-Based Deformable Graph Neural Networks

Positional encodings (PEs) enhance the power of graph neural networks (GNNs), both theoretically and empirically. Two of the most popular families of PEs - spectral (e.g., Laplacian eigenspaces, effective resistance) and walk-based (polynomials of the adjacency matrix) - are theoretically equivalent in expressive power, with expressivity between the 1-WL and 3-WL tests. However, this equivalence assumes the GNN uses the "complete" version of these PEs, which requires $O(n^3)$ time and space complexity. Instead, practitioners commonly use truncated variants of these encodings, such as the first $k$ eigenspaces or powers of the adjacency matrix. However, the theoretical properties of these truncated PEs are unknown. In this work, we initiate the study of these truncated PEs. Theoretically, we show that, under truncation, several families of PEs are fundamentally different in expressive power. As a corollary, we show that truncated spectral PEs are no longer stronger than the 1-WL test. We also study a family of spectral PEs, the $k$-harmonic distances, to highlight the differences in expressive power of even closely related truncated PEs. Finally, we experimentally show that a mix of truncated PEs is preferable to any single family on real-world datasets.

Pre-propagation graph neural networks (PPGNNs) decouple node feature propagation from transformation: graph diffusion is performed once as preprocessing, and training reduces to dense per-node transformations. This design enables mini-batch training without inter-node dependencies, avoids repeated sparse matrix--matrix multiplications, and better matches modern accelerators optimized for dense compute. However, their expressivity remains unclear, and empirical results show a gap between PPGNNs and their message-passing counterparts on commonly used graph benchmarks, especially heterophilic ones. In this paper, we propose a suite of robust graph diffusion operators for preprocessing and a few-shot hidden-state re-propagation scheme during training. Our methods improve the validation and test accuracy of PPGNNs, enabling them to match the accuracy of message-passing GNNs while maintaining training efficiency.

Message Passing Neural Networks (MPNNs) have achieved strong performance on tasks involving relational data. However, small perturbations to graph structure can significantly alter their outputs, raising concerns about their robustness in real-world deployment in security critical environments. In this work, we study a core vulnerability in MPNNs that explicitly consume graph topology via the adjacency matrix or Laplacian as part of their message passing mechanism. We show that this design choice exposes an extremely vulnerable attack surface, with significant effects even under minimal perturbation. Building on this observation, we propose PEA, a simple, gradient-free, black-box injection attack that requires only a single query to the target model, capitalizing on this vulnerability by constructing a perturbation aligned with a specific significant eigenvector to induce large output deviations. Unlike graph modification attacks, PEA operates under the realistic assumption that adversaries cannot alter the original graph and are limited to injecting new nodes at inference time. PEA requires no iterative optimization, parameter learning, or surrogate models---which require additional training and remain susceptible to differences in model priors and generalization capabilities---thereby avoiding significant computational overhead and the associated transferability challenges. We evaluate PEA on popular benchmark datasets across three graph learning tasks, showing consistent performance degradation under realistic attack constraints despite its simplicity. Our results reveal a fundamental security weakness in topology-driven message passing architectures and urge an implementation shift, as the worst effects of such attacks can be substantially mitigated through appropriate input filtering.