EmDT: Embedding Diffusion Transformer for Tabular Data Generation in Fraud Detection

En-Ya Kuo; Sebastien Motsch

stat.MLMar 13, 2026

EmDT: Embedding Diffusion Transformer for Tabular Data Generation in Fraud Detection

Authors: En-Ya Kuo, Sebastien Motsch

Organizations: Arizona State University – School of Mathematical and Statistical Sciences, Tempe, AZ, USA

Abstract

Imbalanced datasets pose a difficulty in fraud detection, as classifiers are often biased toward the majority class and perform poorly on rare fraudulent transactions. Synthetic data generation is therefore commonly used to mitigate this problem. In this work, we propose the Clustered Embedding Diffusion-Transformer (EmDT), a diffusion model designed to generate fraudulent samples. Our key innovation is to leverage UMAP clustering to identify distinct fraudulent patterns, and train a Transformer denoising network with sinusoidal positional embeddings to capture feature relationships throughout the diffusion process. Once the synthetic data has been generated, we employ a standard decision-tree-based classifier (e.g., XGBoost) for classification, as this type of model remains better suited to tabular datasets. Experiments on a credit card fraud detection dataset demonstrate that EmDT significantly improves downstream classification performance compared to existing oversampling and generative methods, while maintaining comparable privacy protection and preserving feature correlations present in the original data.

Figures & tables

Figure 1: Overview of the proposed EmDT architecture. Starting from an imbalanced transaction dataset, the minority samples are first projected into a two-dimensional UMAP space, where distinct fraud clusters are identified (Step 1). A separate diffusion-based generative model is then trained for each cluster (Step 2), using sinusoidal embeddings and a Transformer architecture to generate synthetic fraud samples from normal samples. The synthetic data are combined with the original dataset to train a tree-based classifier such as XGBoost (Step 3), further improving the detection of fraudulent transactions.

Figure 2: Illustration of forward and reverse processes in the diffusion model. The forward process $q(\mathbf{x}_{t}\mid\mathbf{x}_{t-1})$ progressively corrupts the original data distribution $q_{0}$ (left) by adding Gaussian noise to each sample ${\bf x}_{0}$ over timesteps, eventually transforming the data into pure Gaussian noise $\mathbf{x}_{T}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$ (right). The reverse process $p_{\theta}\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t}\right)$ learns to reverse this noising process by iteratively denoising samples, recovering the data distribution from pure noise.

L(\theta)=\frac{1}{N}\sum_{i=1}^{N}\|\boldsymbol{\epsilon}_{i}-\boldsymbol{\epsilon}_{\theta}(\mathbf{x}_{t}^{(i)},t_{i})\|^{2}

(6)

Algorithm 1 DDPM Training Process

Figure 3: An Overview of the proposed EmDT model. In the forward process, Gaussian noise is gradually added to the fraud training samples. During the reverse process, the EmDT embeds the noisy inputs into higher-dimensional spaces and applies a Transformer to better capture feature relationships. Followed by a linear projection, the EmDT model learns to denoise the data and generate synthetic fraud samples.

Figure 4: Left: UMAP visualization of the Credit Card dataset ( $N=284{,}807$ samples, $d=29$ features). Fraudulent transactions (minority class, $n=492$ , $0.17\%$ ) are shown in red, while legitimate transactions (majority class, ${\approx}99.83\%$ ) are shown in blue. The substantial overlap between classes highlights the difficulty of the classification task. Right: UMAP projection restricted to fraudulent transactions only, revealing a structure of 3 distinct clusters.

Hyperparameter	Notation	Value/Search Space
Feature dimension	$d$	29
Embedding dimension	$D$	{32, 64, 128}
Feature embedding scale	$s_{1}$	{1, 10, 50, 100, 500}
Time embedding scale	$s_{2}$	{0.5, 1, 2}
Diffusion timesteps	$T$	1,000
Learning rate	-	[0.00001, 0.003]

Table 1: Main hyperparameters for EmDT.

Figure 5: Overview of the performance evaluation workflow for generative models. The procedure is divided into four steps. First, generative models based on EmDT are trained. Second, a tree-based classifier (XGBoost) is optimized using the training set and the synthetic data. Third, the hyperparameters of the EmDT and classifier are optimized based on the F1 score on the validation set. Fourth, the classifier is evaluated on the test set.

Method	ML Efficiency				Privacy
	F1-Score $\uparrow$	Recall $\uparrow$	Precision $\uparrow$	Bal-Acc $\uparrow$	DCR ( $\approx 0.500$ )
Original	0.800 $\pm 0.035$	0.743 $\pm 0.041$	0.868 $\pm 0.039$	0.871 $\pm 0.02$	-
SMOTE	0.834 $\pm 0.025$	0.784 $\pm 0.031$	0.891 $\pm 0.031$	0.892 $\pm 0.016$	0.686 $\pm 0.01$
CTGAN	0.805 $\pm 0.033$	0.751 $\pm 0.042$	0.873 $\pm 0.074$	0.875 $\pm 0.021$	0.526 $\pm 0.06$
TVAE	0.820 $\pm 0.01$	0.778 $\pm 0.019$	0.869 $\pm 0.035$	0.889 $\pm 0.009$	0.528 $\pm 0.07$
TabDDPM	0.816 $\pm 0.019$	0.767 $\pm 0.032$	0.873 $\pm 0.036$	0.884 $\pm 0.016$	0.578 $\pm 0.04$

Table 2: Evaluation of ML efficiency using the XGBoost classifier and DCR score. ML efficiency performance metrics include F1-score, recall, precision, and balanced accuracy (Bal-Acc). Upward arrows (

\uparrow

) indicate that higher values correspond to better performance. The privacy metric includes DCR score, which represents the probability that synthetic data are more similar to the training set than to the test set. A value close to

50\%

indicates a good balance between resemblance to the training and test distributions. The Original row represents the average performance on real data without data augmentation.

Figure 6: L2 distance between correlation matrices computed from the real and synthetic data. More intense colors indicate larger differences in correlation values. Similarity is quantified using the normalized Frobenius norm, where higher values indicate better preservation of correlation structure.

Figure 7: Comparison of feature distributions between the real dataset and the synthetic data generated by different methods. The black curves represent the distributions of real data, and the colored curves indicate the distributions of synthetic data.

	ML Efficiency				Privacy
Method	F1-Score $\uparrow$	Recall $\uparrow$	Precision $\uparrow$	Bal-Acc $\uparrow$	DCR ( $\approx 0.500$ )
Original	0.800 $\pm 0.035$	0.743 $\pm 0.041$	0.868 $\pm 0.039$	0.871 $\pm 0.02$	-
EmDT w/o Cluster-training	0.829 $\pm 0.027$	0.780 $\pm 0.028$	0.885 $\pm 0.034$	0.890 $\pm 0.014$	0.538 $\pm 0.04$
EmDT w/ Cluster-training	0.849 $\pm 0.021$	0.791 $\pm 0.025$	0.916 $\pm 0.025$	0.895 $\pm 0.012$	0.555 $\pm 0.06$

Table 3: Ablation study of EmDT on classification performance and privacy assessment.

Panel of Figure 8 — Figure 8: Effect of hyperparameter settings on EmDT performance. The boxplots show the distribution of F1-scores for each hyperparameter value, with diamond markers indicating the average performance.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 623 papers

Oct 5, 2026

Usefulness of Quantile-Aware Diffusion Modeling for Highly Imbalanced Tabular Data

Classification problem in the context of highly imbalanced data is a major challenge in many real-world applications (e.g., FinTech, healthcare, etc.). In these cases, the vast majority of instances belong to a single class and a small fraction represent the minority class (often the most critical class). Recently, diffusion models have emerged as powerful approaches to reduce the degree of ``imbalanced-ness'' in the dataset; they work by generating synthetic data by capturing complex data distributions using iterative transformations. However, standard diffusion models are not inherently suited to highly skewed or heavy-tailed data, due to inbuilt quadratic error loss, which lacks the structural sensitivity to capture rare, extreme values, and minority-class nuances. We propose a novel approach, namely, Quantile-TabDDPM, based on a quantile-regularized denoising objective that combines the standard quadratic error loss with a quantile loss term to explicitly capture rare events while preserving the theoretical grounding of the original denoising objective. We extensively evaluated our approach on a real-world credit card transaction dataset characterized by extreme class imbalance. The results demonstrate that the integration of diffusion-based synthetic data generation with a quantile-regularized denoising objective provides a robust and effective framework for fraud detection in highly imbalanced datasets.

Abu Talha, Peng Liu, Souradyuti Paul

Indian Institute of Technology Bhilai, Durg, C.G. 491002, India · Loughborough University, Loughborough, LE11 3TU, United Kingdom

Class-Imbalanced Learning Synthetic Data Augmentation Tabular Diffusion Models Financial Fraud Detection Diffusion Models

Ask PDF

Jul 19, 2025

Fraud is Not Just Rarity: A Causal Prototype Attention Approach to Realistic Synthetic Oversampling

Detecting fraudulent credit card transactions remains a significant challenge, due to the extreme class imbalance in real-world data and the often subtle patterns that separate fraud from legitimate activity. Existing research commonly attempts to address this by generating synthetic samples for the minority class using approaches such as GANs, VAEs (Variational Autoencoders), or hybrid generative models. However, these techniques, particularly when applied only to minority-class data, tend to result in overconfident classifiers and poor latent cluster separation, ultimately limiting real-world detection performance. In this study, we propose the Causal Prototype Attention Classifier (CPAC), an interpretable architecture that promotes class-aware clustering and improved latent space structure through prototype-based attention mechanisms and we couple it with the encoder of a Variational Autoencoder-Generative Adversarial Network (VAE-GAN) in order to achieve improved latent cluster separation moving beyond post-hoc sample augmentation. We compared CPAC-augmented models to traditional oversamplers, such as SMOTE, as well as to state-of-the-art generative models, both with and without CPAC-based latent classifiers. Our results show that classifier-guided latent shaping with CPAC delivers superior performance, achieving an F1-score of 93.74% and recall of 92.85%, along with improved latent cluster separation. Further ablation studies and visualizations provide deeper insight into the benefits and limitations of classifier-driven representation learning for fraud detection. The codebase for this work can be found at the following link: https://github.com/claudiunderthehood/VAEGAN-CPAC.git.

Claudio Giusti, Luca Guarnera, Mirko Casu +1

Department of Mathematics and Computer Science University of Catania Catania, CT 95125

Class-Imbalanced Learning Synthetic Data Augmentation Prototype Learning Financial Fraud Detection

Ask PDF

Apr 13, 2026

Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals

We introduce behavioral fidelity -- a third evaluation dimension for synthetic tabular data that measures whether generated data preserves the temporal, sequential, and structural behavioral patterns that distinguish real-world entity activity. Existing frameworks evaluate statistical fidelity (marginal distributions and correlations) and downstream utility (classifier AUROC on synthetic-trained models), but neither tests for the behavioral signals that operational detection and analysis systems actually rely on. We formalize a taxonomy of four behavioral fraud patterns (P1-P4) covering inter-event timing, burst structure, multi-account graph motifs, and velocity-rule trigger rates; define a degradation ratio metric calibrated to a real-data noise floor (1.0 = matches real variability, k = k-times worse); and prove that row-independent generators -- the dominant paradigm -- are structurally incapable of reproducing P3 graph motifs (Proposition 1) and produce non-positive within-entity IET autocorrelation (Proposition 2), making the positive burst fingerprint of fraud sequences unachievable regardless of architecture or training data size. We benchmark CTGAN, TVAE, GaussianCopula, and TabularARGN on IEEE-CIS Fraud Detection and the Amazon Fraud Dataset. All four fail severely: on IEEE-CIS composite degradation ratios range from 24.4x (TVAE) to 39.0x (GaussianCopula); on Amazon FDB, row-independent generators score 81.6-99.7x, while TabularARGN achieves 17.2x. We document generator-specific failure modes and their resolutions. The P1-P4 framework extends to any domain with entity-level sequential tabular data, including healthcare and network security. We release our evaluation framework as open source.

Bhavana Sajja

Independent Researcher

Synthetic Tabular Data Generation Synthetic Data Evaluation

Ask PDF

Abstract

Synthetic Data Generation

Explore similar work

Usefulness of Quantile-Aware Diffusion Modeling for Highly Imbalanced Tabular Data

Fraud is Not Just Rarity: A Causal Prototype Attention Approach to Realistic Synthetic Oversampling

Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals