Acquiring fraud labels for payment transactions is expensive and inconsistent across tasks, and high-cardinality identifiers such as merchant names make one-hot feature pipelines infeasible at scale. We present CARTA, a self-supervised encoder that learns a compact 16-dim embedding from raw transactions without labels; the frozen embedding is then reused for label-efficient fraud probing and exploratory unsupervised risk stratification without retraining the encoder. CARTA uses type-specific projections with learned embedding tables for high-cardinality identifiers, trained with token excision and InfoNCE contrastive loss. On IEEE-CIS, its linear probe at 1% of labels exceeds a raw-feature probe at 100%; on IBM TabFormer (2M rows), its frozen embedding reaches GBDT AUPRC 0.508, above frequency encoding (0.411) and the merchant-dropping baseline (0.022), though fully-supervised CatBoost is stronger. CARTA is a label-free, reusable representation that complements task-specific supervised models.
Figures & tables
Figure 1. CARTA system overview. Raw transaction features are projected into a uniform token space via type-specific transformations (continuous features to a dense projection, low-cardinality categoricals to a one-hot projection, and high-cardinality identifiers to learned embedding tables). Token excision randomly masks a subset of feature tokens per view as a self-supervised corruption. The Perceiver encoder compresses tokens into a 16-dim Variational Autoencoder (VAE) ( Kingma and Welling, 2014 ) latent. Training uses a composite loss ( LVAE+LInfoNCE with cyclical KL). At inference the frozen embedding is reused across downstream tasks.
IEEE-CIS (590,540 rows, 3.50% fraud)
TabFormer (2,000,000 rows, 0.13% fraud)
Method
Dim
Lin. AUPRC
GBDT AUPRC
GBDT AUC
Lin. AUPRC
GBDT AUPRC
GBDT AUC
Raw †
30/68
0.134 ±0.002
0.616 ±0.006
0.923
0.013 ±0.001
0.022 ±0.002
0.835
Freq. enc. ‡
35/69
—
—
—
0.023 ±0.001
0.411 ±0.056
0.871
PCA-16
16
0.112 ±0.001
0.455 ±0.005
0.882
0.012 ±0.000
0.017 ±0.001
0.818
CARTA-16 (ours)
16
0.217 ±0.034
0.465 ±0.013
0.868
0.008 ±0.009
0.508 ±0.063
0.876
CatBoost §
all
—
0.508
0.907
—
0.711
0.982
Table 1. Full-label probe performance (mean ± std); Dim = representation dimensionality. Bold marks the best frozen representation per column; the fully-supervised CatBoost baseline ( \lx@sectionsign ) is excluded. † Raw drops the high-cardinality columns (OHE infeasible); ‡ frequency encoding applies only to TabFormer. Subscript stds: CARTA-16 on IEEE-CIS is the mean over 5 independent training seeds (population std; matched inner probe scores averaged per run); Raw/PCA representations are deterministic, so their stds reflect 5 probe seeds. TabFormer results are from a single training run (5 probe seeds).
Method
1%
5%
10%
25%
100%
Raw
0.162
0.129
0.131
0.136
0.134
PCA-16
0.105
0.111
0.110
0.114
0.112
UMAP-16
0.046
0.049
0.049
0.050
0.050
CARTA-16
0.206
0.219
0.215
0.219
0.217
Table 2. Linear-probe AUPRC on IEEE-CIS vs. fraction of training labels (mean; 5 training seeds for CARTA-16, 5 probe seeds otherwise). CARTA-16 at 1% labels exceeds raw features at 100% labels; the margin (CARTA-16 at 1% =0.206 vs. raw at 100% =0.134 ) exceeds the full-label stds ( ±0.034 over training seeds for CARTA-16 and ±0.002 over probe seeds for raw, respectively).
Figure 2. Linear-probe AUPRC vs. fraction of training labels; error bars are ±σ over 5 training seeds for CARTA-16 and 5 probe seeds for Raw and PCA-16. On IEEE-CIS, CARTA-16 at 1% of labels ( 0.206 ) exceeds raw features at 100% of labels ( 0.134 ), so self-supervised pretraining reaches a given representation quality with far fewer labels.
IEEE-CIS
TabFormer
Variant
Lin. AUPRC
GBDT AUPRC
Lin. AUPRC
GBDT AUPRC
Full (cell + InfoNCE + cycl. KL)
0.199
0.492
0.014
0.401
− Cyclical KL ( β=1 )
0.208
0.481
0.219
0.431
− InfoNCE (single view)
0.151
0.467
0.005
0.365
− Token excision
0.186
0.425
0.031
0.513
Feature-wise excision (default)
0.241
0.469
0.009
0.533
Table 3. Ablation results (linear probe / GBDT AUPRC; each variant is a single trained encoder, Perceiver unless noted, evaluated as the mean over 5 probe seeds; up to 300 epochs with early stopping, batch 16,384 throughout). “Full” denotes the ablation reference configuration (cell-mode excision + InfoNCE + cyclical KL). Rows are intended for within-table comparison; the canonical CARTA numbers are those in Table 1 , whose CARTA row instead averages 5 independent training runs with per-run inner-probe averaging. The protocols therefore differ by seed population (e.g., the feature-wise row here is one training run scoring 0.241, while the Table 1 5-run mean is 0.217±0.034 ). TabFormer values are especially sensitive: the linear probe operates near the 0.13% base rate, and for the merchant-aware variants GBDT AUPRC varies by 0.02–0.08 (std) across the 5 probe seeds even on a fixed embedding.
Representation
Lin. AUPRC
LE@1%
GBDT AUPRC
Silhou.
Train (s)
PCA-16
0.112
0.105
0.455
0.152
<1
UMAP-16
0.050
0.046
0.357
0.202
1,318
Autoencoder
0.179
0.147
0.468
0.215
319
Sparse AE
0.139
0.135
0.446
0.225
303
MLP-VAE
0.083
0.083
0.443
0.717
286
SCARF
0.214
0.217
0.491
0.086
788
Table 4. Unsupervised representation comparison on IEEE-CIS. Every method produces a 16-dim label-free embedding scored with the same probes, K-means silhouette (best k∈[3,7] ), and training wall-time. Lin. AUPRC is the mean over 5 probe seeds; LE@1% is that probe at 1% of training labels; each embedding is a single timed training run (UMAP timed multi-threaded). In this single-run comparison CARTA’s full-label linear AUPRC ( 0.212 ) is similar to SCARF’s ( 0.214 ) with a shorter wall-clock time (64% of SCARF’s), while SCARF leads on LE@1% ( 0.217 vs. 0.187 ) and GBDT AUPRC ( 0.491 vs. 0.450 ); the high-silhouette methods (MLP-VAE, UMAP) have weak probes, so silhouette does not track downstream usefulness here.
Figure 3. Representation quality (linear-probe AUPRC, 5 probe seeds) versus training wall-clock time for seven label-free 16-dim encoders on IEEE-CIS (one timed training run per method). CARTA’s (blue) full-label linear AUPRC is similar to SCARF’s at 64% of SCARF’s single-run training time. Error bars are ±σ over probe seeds.
Figure 4. Unsupervised behavioral archetypes discovered by K-means ( k=3 , silhouette =0.085 ) on the full IEEE-CIS CARTA latent space (590K transactions, no fraud labels used). The highest-incidence archetype (103K transactions) shows an observed fraud rate of 8.4%, nearly 4.0 × the rate of the lowest-incidence archetype (2.1%), despite being discovered without any labeled supervision.
A total of seven categorical encoding methods were tested on the IEEE-CIS fraud benchmark dataset (590,540 records, 3.5% positives, 8 high-cardinality columns). The encoders were evaluated using a stratified 5-fold cross-validation (CV) with three repetitions. Five of the encoders had identical frozen LightGBM learners in the downstream phase, allowing for controlled comparisons of their performance to each other. CatBoost and TabNet were included as comparisons across paradigms using different learners. The entity embeddings produced the highest AUC-ROC (0.9612), with a statistically significant tie with that of CatBoost (0.9602) and statistically superior to tier group encoding (0.9548), whereas target encoding was only 0.0023 worse than tier group encoding and the auditor-friendly tier boundaries were maintained. Off-the-shelf TabNet did not outperform tree-based pipelines and collapsed under data scarcity. On AUC-PR, CatBoost leads (0.822 vs. 0.793); no encoder dominated both metrics. Per-column analysis confirmed the embedding advantage arises from joint multi-column representation.
Xiao Han, Jingjing Liu, Moxuan Zheng +2
Goizueta Business School Emory University Atlanta, GA, USA · Electrical Engineering and Computer Sciences University of California, Berkeley Kirkland, WA · Stern School of Business New York University New York, NY, USA +2
Credit card fraud detection typically relies on tabular features, while repeated attributes can also provide useful relational signals. This paper proposes THGT-FD, a Temporal Heterogeneous Graph Transformer for Fraud Detection. Each transaction is represented using one transaction token and six types of relation tokens and incorporates Time2Vec encoding into the transaction representation. A Transformer learns the interactions among these tokens within each individual transaction and then outputs a fraud probability. Experiments were conducted on 150,000 transactions sampled from the IEEE-CIS Fraud Detection dataset and chronologically partitioned according to TransactionDT. On the test set, THGT-FD achieved an AUC-ROC of 0.8536, an average precision of 0.4164, and a Recall@5% of 0.4708. The class-weighted histogram-based gradient-boosting baseline achieved an AUC-ROC of 0.8722. The results indicate that relation tokens provide useful information for fraud-risk ranking, although the current model does not yet incorporate entity-level historical aggregation.
Qinwen Yan
University of California, Los Angeles Los Angeles, California, USA
Working entirely on topologically anonymized embeddings, we perform fraud detection using iterative rounds of unsupervised filtering followed by supervised sniping. The result is an ultra-low latency privacy--preserving triage that allows institutions to flag suspicious activity without compromising Personally Identifiable Information.