Payment applications such as fraud detection, authorization prediction, and payment optimization draw on overlapping transaction information. This motivates a reusable payments intelligence layer that shares learned representations and predictive signals, rather than developing predictive models independently for each task. Realizing this shared capability requires handling heterogeneous transaction fields, high-cardinality identifiers, lifecycle-dependent attributes, long histories, and rare outcomes while meeting latency and cost constraints. We present Large Payments Model (LPM), a domain-specific foundation model designed for operational use and trained from scratch on large-scale structured transaction data to learn reusable representations for payment prediction tasks. LPM employs field-specific encodings and lifecycle-aware self-supervision in two complementary architectures: the Independent variant uses hybrid masking and supports efficient per-transaction inference, while the Sequence variant incorporates entity-linked histories through compact latent representations for richer temporal context. Optional distillation enables resource-constrained deployment. We pretrain LPM on up to 400 billion tokens and study scaling, downstream performance, and representation reuse. LPM improves over strong task-specific baselines across multiple prediction tasks in Merchant and Treasury Services, demonstrating applicability in distinct payment domains. Its representations also facilitate adaptation to a target not observed during pretraining through compact task-specific components. Together with production deployment, these results position LPM as a practical basis for reusable payments intelligence.
Figures & tables
Figure 1: From shared learning to downstream use. A pretrained LPM provides a common starting point for payment tasks, reducing the need to learn transaction structure independently for each application. The reuse routes differ in the task-specific modeling they require, while shared serving infrastructure enables operational delivery.
Figure 2: LPM at a glance . LPM encodes heterogeneous transaction fields with a field-aware encoder and supports two variants: an Independent model using aggregate context and hybrid masking, and a Sequence model capturing entity-linked histories for temporal context. We evaluate LPM on Merchant Services and Treasury Services, highlighting relative improvements in Merchant Services fraud detection average precision over a task-specific baseline.
Figure 3: LPM Independent model architecture. During pretraining, hybrid masking is applied to transaction records, after which the field-aware encoder and bidirectional transformer layers produce field representations and a transaction-level representation. Field-specific heads reconstruct masked targets.
Figure 4: LPM Sequence model architecture. An entity-linked transaction history is encoded and pooled into compact latent channels, then processed by interleaved intra-transaction and causal inter-transaction blocks. The former combines cross-attention to the original field representations with self-attention; the latter applies temporal self-attention. Per-step pooling and prediction follow.
Figure 5: Payment domains and representative transaction features. The top and bottom panels schematically identify the principal participants in the Merchant Services and Treasury Services payment ecosystems, respectively. Highlighted components identify JPMorgan Chase’s roles as the acquirer, originating and receiving depository financial institution (ODFI/RDFI). The accompanying examples show selected event, entity, and behavioral attributes represented in the corresponding transaction records; they are illustrative rather than exhaustive.
Task
Metric
Independent uplift (%) ↑
Sequence uplift (%) ↑
Fraud Detection
AP
62.7
85.5
Network Response (fine-grained)
F1 Macro
80.0
89.5
Network Response (coarse-grained)
F1 Macro
4.5
8.4
Retry Response
AP
22.8
22.8
Table 1: Merchant Services task evaluation results. Values report relative uplift in the row-specific metric under the evaluation setup described in Section 5.1 .
Task
Recall Volume uplift (%) ↑
Recall Value uplift (%) ↑
ACH fraud
27.4
63.5
Wire fraud
5.6
0.0
Table 2: Treasury Services fraud-prevention results at fixed alert-review budgets. Values report relative uplift over the corresponding baseline under the same evaluation population and fixed review capacity.
Adaptation method
Backbone
Parameters
Relative Δ AP ↑
Practical value
Task-specific baseline
–
–
Reference
Dedicated solution
Linear probe on [ROW]
Frozen
<1 K
−2.1%
Reusable representations
Attention pool over field embeds
Frozen
∼ 110K
−1.3%
Learned field aggregation
LoRA with [ROW] head
LoRA
∼ 670K
+0.5%
Efficient backbone adaptation
Table 3: Adaptation to eventual retry approval, a downstream prediction target unavailable during pretraining. Relative AP change is computed against the task-specific baseline; all methods use the same downstream training set and out-of-time evaluation population. Trainable parameters exclude the frozen LPM backbone.
Figure 6: Multiple payment semantics in a shared transaction representation. Each panel shows the same UMAP projection of 30,000 [ROW] embeddings, with colors defined by payment method, merchant category, and network response. Payment-method names are anonymized. The recurring local structure across views shows that embedding neighborhoods reflect multiple facets of a transaction.
Attribute
Observed macro-F1 ↑
Permuted-label null
k=10
k=30
k=100
k=30
Payment method
0.919
0.889
0.773
0.098
Merchant category
0.660
0.604
0.488
0.079
Network Response
0.800
0.793
0.762
0.163
Table 4: Neighborhood structure of Merchant Services [ROW] representations. Observed columns report cosine k -NN macro-F1 in the original representation space. The final column reports the mean macro-F1 after permuting labels across reference embeddings while holding query labels and neighborhoods fixed.
Figure 7: Pretraining data scaling on Merchant Services fraud detection. The left and right panels show Independent and Sequence models, respectively. Each series reports the relative change in AP compared with the 18B-token Medium configuration of the same architecture. The orange result highlights additional gains from a larger Sequence model at 400B tokens.
Figure 8: Model capacity scaling at approximately 85B pretraining tokens. The left panel shows Independent model’s capacity tiers, which vary both representation width and depth; the right panel shows Sequence model’s configurations with depth fixed at 18 layers and varying representation width. Bars report relative uplift in Fraud Detection AP and fine-grained Network Response macro-F1 over their respective task-specific baselines.
Model
Supervision
Relative Δ AP (%) ↑
LPM (Medium)
Teacher
Reference
LPM (Small)
Distilled
−5.8
GBT
Distilled
−16.7
GBT
Hard label
−24.9
Table 5: Distillation results for Treasury Services fraud detection. Values report the relative change in AP from the Medium LPM teacher; all student variants use the same transfer set.
Figure 9: Sensitivity of the Sequence model to sequence length on Merchant Services tasks. The panels report Fraud Detection AP and fine- and coarse-grained Network Response macro-F1, respectively. Within each task, performance is expressed relative to the score obtained with T=32 transactions.
Figure 10: Overview of inference flow through batching, preprocessing, and the model instances.
Sequential financial transaction data are increasingly being used by financial institutions to train payment foundation models at scale, producing rich contextual embeddings that support applications such as fraud prevention, credit risk assessment, and offer personalization. However, such models are not designed for flexible instruction-following reasoning and require bespoke downstream models for each application. In contrast, large language models (LLMs) provide a natural-language interface for flexible transaction reasoning, yet naively serializing transaction histories into text representations is inefficient and costly. Multimodal fusion architectures bridge these paradigms, injecting transaction embeddings directly into flexible language reasoning pipelines. In this work, we present a systematic study of how transaction embeddings, pretrained at the scale of a global payments network, interact with LLM-based reasoning through embedding injection, transaction-language alignment, and instruction tuning. To do so, we train the Multimodal Instruction Network for Transactions (MINT), which combines a transaction encoder pretrained on billions of real-world payments with multimodal instruction tuning on millions of transaction question-answering examples. We evaluate predictive and extractive question answering, zero-shot generalization to unseen question types, and inference efficiency. Our results reveal a consistent trade-off: compact transaction embeddings improve predictive QA accuracy while substantially reducing input tokens, latency, and memory relative to a text-serialized baseline, whereas serialized histories remain highly competitive for extractive QA. Overall, our empirical findings provide a set of actionable insights and practical guidance for building multimodal transaction-aware language systems at network scale.
Parameswaran Kamalaruban, Viktor Drobnyi, Maeve Madigan +3
Risk and Security AI Lab, Visa Inc., United Kingdom
Ant International accumulates large volumes of transaction history across users and payment businesses. Confirmed fraud and chargeback outcomes, by contrast, are scarce, delayed, and specific to each business. A single encoder pretrained on unlabeled histories could serve all businesses, but only if pretraining captures the behavior that separates risk. Coarse activity differences can make contrastive pretraining on transaction logs too easy: transaction volume, amount, and timing can distinguish a history's continuation from arbitrary candidates without modeling how payment resources are used. We present TRIDENT-Seq, a transaction foundation model trained with Prefix-Future Contrast (PFC). PFC matches each prefix to a summary of its subsequent behavior among candidates with similar activity profiles; activity prediction and dispersion regularization act on the encoder output. Across three production payment businesses, the pretrained encoder improves average precision (AP) and recall at a 0.5% intervention budget over architecture-matched scratch training in all six business-budget settings. With a fixed validation set, it exceeds full-label scratch training in mean AP and recall using 10% of the training labels on Wallet and 25% on Shopping and SME. Pretraining only on another business's histories still helps. Replacing profile-matched candidates with random ones cuts recall from 24.62% to 5.64%. In Ant International's deployment evaluation, recorded amount-based chargeback and overall loss rates are 37.7% and 37.9% lower than those of the previous XGBoost-based system under the same intervention budget.
Weiyu Han, Haoyu Jiang, Jie Sheng +7
Ant International · East China Normal University · Tongji University +1
Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models for banking event sequences. Our approach pre-trains a Transformer-based architecture with masked modelling on a large-scale, heterogeneous banking event corpus using a self-supervised objective tailored to the discrete, variable-length nature of financial records. The resulting model supports a wide range of downstream tasks such as credit scoring, fraud detection, and lifetime value prediction: strong performance can be achieved by training a simple linear model on top of the extracted embeddings and can be further improved with lightweight fine-tuning. Through extensive evaluation on downstream tasks, we demonstrate that PRAGMA achieves superior performance across multiple domains directly from raw event sequences, providing a general-purpose representation layer for financial applications.
Maxim Ostroukhov, Ruslan Mikhailov, Vladimir Iashin +11