We report the development of Antom 3-in-1 Transformer, a payment foundation model that integrates sequence, tabular, and graph representations for payment risk detection. The model contains approximately 7B parameters, with approximately 1.7B activated per task route. Antom 3-in-1 Transformer is pretrained to learn reusable behavioral and relational representations, primarily on Ant International's internal payment data. In a cross-business transfer experiment, pretraining improves the target business metric by 34.5% over the same architecture trained from scratch. With only 25% of the training labels, the pretrained model outperforms the full-label scratch baseline in Average Precision (AP) and [email protected]% on different merchant domains. In a four-month online experiment on Ant International's billion-scale payment platform, the model reduced the overall loss rate by 37.88% relative to XGB-Online.
Figures & tables
Figure 1. Overview of Antom 3-in-1 Transformer . (a) Sequence modality. Independent Transformer towers encode user, card, and device histories. (b) Tabular modality. Field attention captures interactions among transaction attributes and business features. (c) Graph modality. A graph attention network combines account profiles and rolling user graphs to learn relational representations. (d) Task-conditioned fusion. Task queries and sparse experts fuse sequence and tabular representations; graph embeddings augment the risk head. Sequence, tabular, and graph encoders feed task-conditioned attention and sparse experts; pretraining objectives learn behavioral and relational representations.
System
Traffic share
CB rate change
Loss rate change
XGB-Online
32.1%
base
base
SAINT-style
37.4%
− 13.21%
− 18.18%
Antom 3-in-1 Transformer
30.4%
− 37.74%
− 37.88%
Table 1. Online portfolio results relative to XGB-Online.
Table 3. Antom 3-in-1 Transformer and transaction-model baselines on the temporal test split of IBM’s synthetic dataset. R@ q is fraud recall among the top- q scored transactions.
Figure 2. Online scoring timeline of Antom 3-in-1 Transformer. Latencies are monitored values; braces mark service budgets. Stage widths are schematic. A horizontal time axis from the payment event to the risk decision, with the feature platform, the model service, and the strategy platform in sequence, their monitored latencies, and the 120 ms and 250 ms budgets.
Figure 3. Label efficiency on Antom-Merchant A and Antom-Merchant B. Points and error bars show the mean and standard deviation over three fine-tuning runs. Four plots show AP and recall at a 0.5 percent intervention budget for the pretrained model and scratch training across five training-label budgets on AliExpress and ANTOM, with a dashed line for full-label scratch.
Table 5. Sequence pretraining objectives across evaluation domains and label budgets (%). Mean ± standard deviation over three fine-tuning runs. Bold: best; underline: second best.
Method
Amazon
YelpChi
T-Finance
Elliptic
Avg. rank ↓
No graph (standalone)
.9184 ± .0066
.6553 ± .0079
.6782 ± .0647
.6578 ± .0380
7.00
Graph only
.9032 ± .0052
.6579 ± .0079
.8341 ± .0018
.3129 ± .0608
9.00
Concatenation (no dropout)
.9096 ± .0088
.7128 ± .0113
.8404 ± .0056
.6349 ± .0320
6.00
Concatenation + dropout
.9110 ± .0091
.7107 ± .0091
.8411 ± .0073
.6543 ± .0397
5.25
Graph token (no dropout)
.9122 ± .0077
.7223 ± .0124
.8356 ± .0065
.6403 ± .0402
5.25
Graph token + dropout
.9123 ± .0072
.7096 ± .0150
.8378 ± .0042
.6558 ± .0306
5.25
Table 6. Graph-augmented prediction on public graph benchmarks (AUPRC; mean ± standard deviation over ten seeds). Bold: best; underline: second best.
Sampled Industrial (pp vs. B1)
TransXion (%)
Method
AUROC
AP
R@1%
AUROC
AP
R@1%
B1
+0.00
+0.00
+0.00
82.47
13.21
39.47
B2
+0.85 ± 0.11
+0.37 ± 0.35
+1.77 ± 0.21
93.69 ± 0.07
18.85 ± 0.41
60.31 ± 0.52
Concat
+1.78 ± 0.17
+2.67 ± 0.22
+1.99 ± 0.44
94.30 ± 0.07
18.76 ± 0.37
62.64 ± 0.62
Sum-pooling
− 0.67 ± 0.13
+1.37 ± 0.31
+1.81 ± 0.21
94.60 ± 0.08
19.26 ± 0.65
62.97 ± 0.96
CE
− 0.53 ± 0.14
+1.19 ± 0.48
+1.71 ± 0.71
94.83 ± 0.06
18.89 ± 0.82
62.75 ± 1.25
Table 7. Task-conditioned fusion with fixed upstream models: Sampled Industrial (percentage-point difference from B1) and TransXion (%). Mean ± standard deviation over five seeds.
Sampled Industrial (pp vs. B1)
Method / q
0.5%
1%
5%
10%
25%
B1
+0.0
+0.0
+0.0
+0.0
+0.0
B2
+1.8
+1.8
+1.4
+0.7
+0.0
Concat
+1.2
+2.0
+1.4
+1.0
+0.3
CE
+1.6
+1.7
− 0.8
− 1.7
− 0.1
MMT
+2.1
+2.0
− 1.2
− 1.7
− 0.3
Table 8. Recall among the top- q scored transactions: Sampled Industrial (pp vs. B1) and TransXion (%).
Table 9. Modality ablation on the industrial sampled offline dataset (%).
Figure 4. Pretrained representations of 2,880 annotated histories: PCA (left) and t-SNE (right). PCA and t-SNE projections of pretrained representations, with two behavior groups and an outlined t-SNE region used for case inspection.
Figure 5. Card and device switching after rejection, from the outlined region in Figure 4 . Three payment attempts show changes in anonymous card and device identities following rejections while retaining the IP identity, ending in a successful attempt.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Domain
Scored payments
History events
Mean length
Empty history
Antom-Merchant A
501,898
14.78M
29.5
30.63%
Antom-Merchant B
371,600
2.91M
7.8
76.91%
HK
127,375
16.03M
125.8
0.50%
Appendix
Table 10. Industrial datasets for the low-label and cross-scenario studies.
Payment applications such as fraud detection, authorization prediction, and payment optimization draw on overlapping transaction information. This motivates a reusable payments intelligence layer that shares learned representations and predictive signals, rather than developing predictive models independently for each task. Realizing this shared capability requires handling heterogeneous transaction fields, high-cardinality identifiers, lifecycle-dependent attributes, long histories, and rare outcomes while meeting latency and cost constraints. We present Large Payments Model (LPM), a domain-specific foundation model designed for operational use and trained from scratch on large-scale structured transaction data to learn reusable representations for payment prediction tasks. LPM employs field-specific encodings and lifecycle-aware self-supervision in two complementary architectures: the Independent variant uses hybrid masking and supports efficient per-transaction inference, while the Sequence variant incorporates entity-linked histories through compact latent representations for richer temporal context. Optional distillation enables resource-constrained deployment. We pretrain LPM on up to 400 billion tokens and study scaling, downstream performance, and representation reuse. LPM improves over strong task-specific baselines across multiple prediction tasks in Merchant and Treasury Services, demonstrating applicability in distinct payment domains. Its representations also facilitate adaptation to a target not observed during pretraining through compact task-specific components. Together with production deployment, these results position LPM as a practical basis for reusable payments intelligence.
Ant International accumulates large volumes of transaction history across users and payment businesses. Confirmed fraud and chargeback outcomes, by contrast, are scarce, delayed, and specific to each business. A single encoder pretrained on unlabeled histories could serve all businesses, but only if pretraining captures the behavior that separates risk. Coarse activity differences can make contrastive pretraining on transaction logs too easy: transaction volume, amount, and timing can distinguish a history's continuation from arbitrary candidates without modeling how payment resources are used. We present TRIDENT-Seq, a transaction foundation model trained with Prefix-Future Contrast (PFC). PFC matches each prefix to a summary of its subsequent behavior among candidates with similar activity profiles; activity prediction and dispersion regularization act on the encoder output. Across three production payment businesses, the pretrained encoder improves average precision (AP) and recall at a 0.5% intervention budget over architecture-matched scratch training in all six business-budget settings. With a fixed validation set, it exceeds full-label scratch training in mean AP and recall using 10% of the training labels on Wallet and 25% on Shopping and SME. Pretraining only on another business's histories still helps. Replacing profile-matched candidates with random ones cuts recall from 24.62% to 5.64%. In Ant International's deployment evaluation, recorded amount-based chargeback and overall loss rates are 37.7% and 37.9% lower than those of the previous XGBoost-based system under the same intervention budget.
Weiyu Han, Haoyu Jiang, Jie Sheng +7
Ant International · East China Normal University · Tongji University +1
Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models for banking event sequences. Our approach pre-trains a Transformer-based architecture with masked modelling on a large-scale, heterogeneous banking event corpus using a self-supervised objective tailored to the discrete, variable-length nature of financial records. The resulting model supports a wide range of downstream tasks such as credit scoring, fraud detection, and lifetime value prediction: strong performance can be achieved by training a simple linear model on top of the extracted embeddings and can be further improved with lightweight fine-tuning. Through extensive evaluation on downstream tasks, we demonstrate that PRAGMA achieves superior performance across multiple domains directly from raw event sequences, providing a general-purpose representation layer for financial applications.
Maxim Ostroukhov, Ruslan Mikhailov, Vladimir Iashin +11