A Systematic Evaluation Protocol of Graph-Derived Signals for Tabular Machine Learning
Authors: Mario Heidrich, Jeffrey Heidemann, Rüdiger Buchkremer, Gonzalo Wandosell Fernández de Bobadilla
Organizations: Universidad Católica San Antonio de Murcia (UCAM), Murcia, Spain · Institute of IT Management and Digitization Research (IFID), FOM University of Applied Sciences in Economics and Management, Düsseldorf, Germany
While graph-derived signals are widely used in tabular learning, existing studies typically rely on limited experimental setups and average performance comparisons, leaving the statistical reliability and robustness of observed gains largely unexplored. Consequently, it remains unclear which signals provide consistent and robust improvements. This paper presents a taxonomy-driven empirical analysis of graph-derived signals for tabular machine learning. We propose a unified and reproducible evaluation protocol to systematically assess which categories of graph-derived signals yield statistically significant and robust performance improvements. The protocol provides an extensible setup for the controlled integration of diverse graph-derived signals into tabular learning pipelines. To ensure a fair and rigorous comparison, it incorporates automated hyperparameter optimization, multi-seed statistical evaluation, formal significance testing, and robustness analysis under graph perturbations. We demonstrate the protocol through an extensive case study on a large-scale, imbalanced cryptocurrency fraud detection dataset. The analysis identifies signal categories providing consistently reliable performance gains and offers interpretable insights into which graph-derived signals indicate fraud-discriminative structural patterns. Furthermore, robustness analyses reveal pronounced differences in how various signals handle missing or corrupted relational data. These findings demonstrate practical utility for fraud detection and illustrate how the proposed taxonomy-driven evaluation protocol can be applied in other application domains.
Figures & tables
Category
Method
Description
Centrality
Degree Centrality
Number of incident edges
PageRank
Recursive importance scoring
Betweenness Centrality
Fraction of shortest paths through node
Eigenvector Centrality
Importance based on neighbors’ importance
Closeness Centrality
Inverse average shortest path length
Cohesion
Clustering Coefficient
Density of direct neighbors
Table 1: Representative graph signals categorized by taxonomic family.
Figure 1: Average F1 -score improvements across graph signal categories relative to the transaction-only baseline, aggregated over classifiers and random seeds (trimmed aggregation).
Figure 2: Per-classifier F1 -scores across graph signal categories. Cell values report mean F1 -scores with standard deviations across random seeds. Color intensity indicates relative performance differences with respect to the transaction-only (TRX) baseline (green = improvement, red = degradation).
Figure 3: Aggregated McNemar test outcomes for graph signal categories across classifiers. Each cell represents the balance of statistically significant improvements ( p≤0.05 ) versus degradations. Darker green shades indicate a higher frequency of significant performance gains over the transaction-only baseline.
Figure 4: Average ΔF1 relative to the transaction-only baseline across graph signal categories under increasing edge removal.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Mean F1 -scores across classifiers and graph signals. Cell values report the mean F1 -score aggregated over the trimmed middle eight runs. Color intensity indicates relative performance differences with respect to the transaction-only baseline (green = improvement, red = degradation).
Figure 6: Standard deviation of F1 -scores across random seeds. Values report the standard deviation of F1 -scores over the trimmed middle eight runs. Darker shading indicates higher variability. Graph signals are grouped by category.
Figure 7: Signal-level aggregation of McNemar test outcomes across classifiers and runs ( p≤0.05 ). Cells show the number of statistically significant improvements versus degradations (#better / #worse) for each classifier–graph signal combination, aggregated over ten random seeds. Color intensity reflects the net balance between improvements and degradations.
Figure 8: Performance comparison under structural graph perturbations with 0% edge removal (transaction-only reference). Cell values report mean F1 -scores with standard deviations across random seeds.
Figure 9: Performance comparison under moderate structural degradation with 25 % random edge removal.
Figure 10: Performance comparison under stronger structural degradation with 50 % random edge removal.
Figure 11: Classifier-specific robustness trends illustrating how average ΔF1 improvements relative to the transaction-only baseline evolve under increasing levels of random edge removal (0%, 25%, 50%) for different graph signal categories.
Model
Parameter
Search Space
Logistic Regression
penalty
{l2,None}
C
log-uniform [0.001,10]
solver
{lbfgs,saga}
balancing_strategy
{none,oversampling,class_weights}
use_emb_pca
{False,True}
emb_pca_n
{16,32,48,64,0.95}
Appendix
Table 2: Hyperparameter search spaces for all evaluated classifiers. Only the parameters listed are optimized via Bayesian optimization (Hyperopt) with a fixed trial budget; all remaining parameters are kept at library default values.
Approximate computation for large graphs; sampled nodes: min(2000,∣V∣) ; undirected graph
Eigenvector Centrality
Computed per connected component; initialization: uniform; maximum iterations: 1000; convergence tolerance: 10−6 ; undirected graph
Closeness Centrality
Computed per connected component; undirected graph
Infomap
Directed graph; unweighted edges
Leiden
Undirected graph; resolution parameter: default
Appendix
Table 3: Graph signal generation parameters used across all evaluated signal categories. All graph-derived signals are computed once on the fixed transaction graph and reused across all evaluation runs to ensure paired comparability.
Model
Parameter
Search Space
GCN
Reduce function
{sum,max,mean}
Message function
SAFE_MSG_FUNCS (DGL message functions)
Activation
{relu,elu,gelu,leaky_relu,silu}
Bias
{True,False}
#Layers
{1,…,4}
Hidden sizes
layer2: 8–127; layer3: 8–63; layer4: 8–31
Appendix
Table 4: Hyperparameter search space for GNN-based models. The table reports the ranges and options considered during automated Bayesian hyperparameter optimization for GCN, GAT, and GCL-based models. All configurations are evaluated under identical optimization budgets and selection criteria.
Figure 12: Maximum F1 -score achieved per graph signal category across all evaluated classifiers. The dashed horizontal line denotes the best-performing transaction-only (TRX) baseline.
Industrial fraud detection often relies on costly expert-crafted features that overlook graph-structured relational signals, while GNNs often do not meet the interpretability and deployment requirements of financial risk control. We propose GraphFAS (Graph Feature Automated Selection), a distributed feature selection procedure based on Boruta that bridges this gap through: (1) a non-parametric graph feature generation module that constructs explicit, interpretable structural features via multi-hop subgraph extraction and multi-scale aggregation without learned parameters; and (2) an automated distributed feature selection algorithm extending Boruta with median-based aggregation across partitions to robustly identify informative features at scale with minimal domain expertise. Compared with end-to-end GNN pipelines, GraphFAS decouples feature aggregation from model training, enabling direct integration with tabular models and direct compatibility with TreeSHAPbased explanations. Deployed in Alipay, GraphFAS delivers orderof-magnitude improvements in engineering efficiency while showing strong performance against expert-driven and graph-learning baselines on large-scale graphs.
Yice Luo, Yun Zhu, Xi Chen +7
Ant Group Hangzhou, China · Nanjing University Nanjing, China
Financial fraud in corporate transaction networks has grown more coordinated and harder to detect with rule-based engines and with classical learning models that treat each transaction in isolation. This paper presents GTFD, a graph-transformer fraud detector that fuses structural and temporal evidence from a corporation's payment graph. GTFD encodes the graph with a multi-head graph attention network, encodes ordered transaction sequences with a gated transformer, and combines both views through a cross-modal gating layer. A conformal risk-control head converts the fused representation into threshold-free anomaly scores with finite-sample coverage guarantees, and the network is trained with self-supervised link-mask pretraining plus adversarial augmentation so it remains stable under scarce labels and under adversarial perturbation. On a corporate transaction benchmark enriched with coordinated fraud rings, GTFD reaches an AUROC of 0.990, an F1-score of 96.1% (precision 96.3%, recall 95.9%), and an accuracy of 98.4%. It reduces the false-positive rate by about 29% relative to the strongest baseline while raising coordinated fraud-ring recall from 85.1% to 96.5%. Ablations attribute roughly 2.0 AUROC points to self-supervised pretraining and 1.9 AUROC points to the conformal head, and adversarial stress tests show GTFD retains 89.2% accuracy at perturbation magnitude 0.20 where the next-best model falls to 76.4%.
Financial fraud detection in transaction networks involves modeling sparse anomalies, dynamic patterns, and severe class imbalance in the presence of temporal drift in the data. In real-world transaction systems, a suspicious transaction is rarely isolated: rather, legitimate and suspicious transactions are often connected through accounts, intermediaries or through temporal transaction sequences. Attribute-based or randomly partitioned learning pipelines are therefore insufficient to detect relationally structured fraud. STC-MixHop, a graph-based framework combining spatial multi-resolution propagation with lightweight temporal consistency modeling for anomaly and fraud detection in dynamic transaction networks. It integrates three components: a MixHop-inspired multi-scale neighborhood diffusion encoder a multi-scale neighborhood diffusion MixHop-based encoder for learning structural patterns; a spatial-temporal attention module coupling current and preceding graph snapshots to stabilize representations; and a temporally informed self-supervised pretraining strategy exploiting unlabeled transaction interactions to improve representation quality. We evaluate the framework primarily on the PaySim dataset under strict chronological splits, supplementing the analysis with Porto Seguro and FEMA data to probe cross-domain component behavior. Results show that STC-MixHop is competitive among graph methods and achieves strong screening-oriented recall under highly imbalanced conditions. The experiments also reveal an important boundary condition: when node attributes are highly informative, tabular baselines remain difficult to outperform. Graph structure contributes most clearly where hidden relational dependencies are operationally important. These findings support a stability-focused view of graph learning for financial fraud detection.
Yiming Lei, Qiannan Shen, Junhao Song
Georgia Institute of Technology, Atlanta, GA, USA · Boston University, Boston, MA, USA · Imperial College London, London, UK