EmDT: Embedding Diffusion Transformer for Tabular Data Generation in Fraud Detection

En-Ya Kuo; Sebastien Motsch

stat.MLMar 13, 2026

EmDT: Embedding Diffusion Transformer for Tabular Data Generation in Fraud Detection

Authors: En-Ya Kuo, Sebastien Motsch

Organizations: Arizona State University – School of Mathematical and Statistical Sciences, Tempe, AZ, USA

Abstract

Imbalanced datasets pose a difficulty in fraud detection, as classifiers are often biased toward the majority class and perform poorly on rare fraudulent transactions. Synthetic data generation is therefore commonly used to mitigate this problem. In this work, we propose the Clustered Embedding Diffusion-Transformer (EmDT), a diffusion model designed to generate fraudulent samples. Our key innovation is to leverage UMAP clustering to identify distinct fraudulent patterns, and train a Transformer denoising network with sinusoidal positional embeddings to capture feature relationships throughout the diffusion process. Once the synthetic data has been generated, we employ a standard decision-tree-based classifier (e.g., XGBoost) for classification, as this type of model remains better suited to tabular datasets. Experiments on a credit card fraud detection dataset demonstrate that EmDT significantly improves downstream classification performance compared to existing oversampling and generative methods, while maintaining comparable privacy protection and preserving feature correlations present in the original data.

Figures & tables

Figure 1: Overview of the proposed EmDT architecture. Starting from an imbalanced transaction dataset, the minority samples are first projected into a two-dimensional UMAP space, where distinct fraud clusters are identified (Step 1). A separate diffusion-based generative model is then trained for each cluster (Step 2), using sinusoidal embeddings and a Transformer architecture to generate synthetic fraud samples from normal samples. The synthetic data are combined with the original dataset to train a tree-based classifier such as XGBoost (Step 3), further improving the detection of fraudulent transactions.

Figure 2: Illustration of forward and reverse processes in the diffusion model. The forward process $q(\mathbf{x}_{t}\mid\mathbf{x}_{t-1})$ progressively corrupts the original data distribution $q_{0}$ (left) by adding Gaussian noise to each sample ${\bf x}_{0}$ over timesteps, eventually transforming the data into pure Gaussian noise $\mathbf{x}_{T}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$ (right). The reverse process $p_{\theta}\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t}\right)$ learns to reverse this noising process by iteratively denoising samples, recovering the data distribution from pure noise.

L(\theta)=\frac{1}{N}\sum_{i=1}^{N}\|\boldsymbol{\epsilon}_{i}-\boldsymbol{\epsilon}_{\theta}(\mathbf{x}_{t}^{(i)},t_{i})\|^{2}

(6)

Algorithm 1 DDPM Training Process

Figure 3: An Overview of the proposed EmDT model. In the forward process, Gaussian noise is gradually added to the fraud training samples. During the reverse process, the EmDT embeds the noisy inputs into higher-dimensional spaces and applies a Transformer to better capture feature relationships. Followed by a linear projection, the EmDT model learns to denoise the data and generate synthetic fraud samples.

Figure 4: Left: UMAP visualization of the Credit Card dataset ( $N=284{,}807$ samples, $d=29$ features). Fraudulent transactions (minority class, $n=492$ , $0.17\%$ ) are shown in red, while legitimate transactions (majority class, ${\approx}99.83\%$ ) are shown in blue. The substantial overlap between classes highlights the difficulty of the classification task. Right: UMAP projection restricted to fraudulent transactions only, revealing a structure of 3 distinct clusters.

Hyperparameter	Notation	Value/Search Space
Feature dimension	$d$	29
Embedding dimension	$D$	{32, 64, 128}
Feature embedding scale	$s_{1}$	{1, 10, 50, 100, 500}
Time embedding scale	$s_{2}$	{0.5, 1, 2}
Diffusion timesteps	$T$	1,000
Learning rate	-	[0.00001, 0.003]

Table 1: Main hyperparameters for EmDT.

Figure 5: Overview of the performance evaluation workflow for generative models. The procedure is divided into four steps. First, generative models based on EmDT are trained. Second, a tree-based classifier (XGBoost) is optimized using the training set and the synthetic data. Third, the hyperparameters of the EmDT and classifier are optimized based on the F1 score on the validation set. Fourth, the classifier is evaluated on the test set.

Method	ML Efficiency				Privacy
	F1-Score $\uparrow$	Recall $\uparrow$	Precision $\uparrow$	Bal-Acc $\uparrow$	DCR ( $\approx 0.500$ )
Original	0.800 $\pm 0.035$	0.743 $\pm 0.041$	0.868 $\pm 0.039$	0.871 $\pm 0.02$	-
SMOTE	0.834 $\pm 0.025$	0.784 $\pm 0.031$	0.891 $\pm 0.031$	0.892 $\pm 0.016$	0.686 $\pm 0.01$
CTGAN	0.805 $\pm 0.033$	0.751 $\pm 0.042$	0.873 $\pm 0.074$	0.875 $\pm 0.021$	0.526 $\pm 0.06$
TVAE	0.820 $\pm 0.01$	0.778 $\pm 0.019$	0.869 $\pm 0.035$	0.889 $\pm 0.009$	0.528 $\pm 0.07$
TabDDPM	0.816 $\pm 0.019$	0.767 $\pm 0.032$	0.873 $\pm 0.036$	0.884 $\pm 0.016$	0.578 $\pm 0.04$

Table 2: Evaluation of ML efficiency using the XGBoost classifier and DCR score. ML efficiency performance metrics include F1-score, recall, precision, and balanced accuracy (Bal-Acc). Upward arrows (

\uparrow

) indicate that higher values correspond to better performance. The privacy metric includes DCR score, which represents the probability that synthetic data are more similar to the training set than to the test set. A value close to

50\%

indicates a good balance between resemblance to the training and test distributions. The Original row represents the average performance on real data without data augmentation.

Figure 6: L2 distance between correlation matrices computed from the real and synthetic data. More intense colors indicate larger differences in correlation values. Similarity is quantified using the normalized Frobenius norm, where higher values indicate better preservation of correlation structure.

Figure 7: Comparison of feature distributions between the real dataset and the synthetic data generated by different methods. The black curves represent the distributions of real data, and the colored curves indicate the distributions of synthetic data.

	ML Efficiency				Privacy
Method	F1-Score $\uparrow$	Recall $\uparrow$	Precision $\uparrow$	Bal-Acc $\uparrow$	DCR ( $\approx 0.500$ )
Original	0.800 $\pm 0.035$	0.743 $\pm 0.041$	0.868 $\pm 0.039$	0.871 $\pm 0.02$	-
EmDT w/o Cluster-training	0.829 $\pm 0.027$	0.780 $\pm 0.028$	0.885 $\pm 0.034$	0.890 $\pm 0.014$	0.538 $\pm 0.04$
EmDT w/ Cluster-training	0.849 $\pm 0.021$	0.791 $\pm 0.025$	0.916 $\pm 0.025$	0.895 $\pm 0.012$	0.555 $\pm 0.06$

Table 3: Ablation study of EmDT on classification performance and privacy assessment.

Panel of Figure 8 — Figure 8: Effect of hyperparameter settings on EmDT performance. The boxplots show the distribution of F1-scores for each hyperparameter value, with diamond markers indicating the average performance.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 623 papers

Usefulness of Quantile-Aware Diffusion Modeling for Highly Imbalanced Tabular Data
Oct 5, 2026Abu Talha, Peng Liu, Souradyuti PaulClass-Imbalanced Learning Synthetic Data Augmentation
Indian Institute of Technology Bhilai, Durg, C.G. 491002, India · Loughborough University, Loughborough, LE11 3TU, United Kingdom
Fraud is Not Just Rarity: A Causal Prototype Attention Approach to Realistic Synthetic Oversampling
Jul 19, 2025Claudio Giusti, Luca Guarnera, Mirko Casu +1Class-Imbalanced Learning Synthetic Data Augmentation
Department of Mathematics and Computer Science University of Catania Catania, CT 95125
Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals
Apr 13, 2026Bhavana SajjaSynthetic Tabular Data Generation Synthetic Data Evaluation
Independent Researcher

Oct 5, 2026J/K move · Enter open · S save

Usefulness of Quantile-Aware Diffusion Modeling for Highly Imbalanced Tabular Data

Abu Talha, Peng Liu, Souradyuti Paul

Indian Institute of Technology Bhilai, Durg, C.G. 491002, India · Loughborough University, Loughborough, LE11 3TU, United Kingdom

Class-Imbalanced Learning Synthetic Data Augmentation Tabular Diffusion Models Financial Fraud Detection Diffusion Models

Open paper AskPDF arXiv

Classification problem in the context of highly imbalanced data is a major challenge in many real-world applications (e.g., FinTech, healthcare, etc.). In these cases, the vast majority of instances belong to a single class and a small fraction represent the minority class (often the most critical class). Recently, diffusion models have emerged as powerful approaches to reduce the degree of ``imbalanced-ness'' in the dataset; they work by generating synthetic data by capturing complex data distributions using iterative transformations. However, standard diffusion models are not inherently suited to highly skewed or heavy-tailed data, due to inbuilt quadratic error loss, which lacks the structural sensitivity to capture rare, extreme values, and minority-class nuances. We propose a novel approach, namely, Quantile-TabDDPM, based on a quantile-regularized denoising objective that combines the standard quadratic error loss with a quantile loss term to explicitly capture rare events while preserving the theoretical grounding of the original denoising objective. We extensively evaluated our approach on a real-world credit card transaction dataset characterized by extreme class imbalance. The results demonstrate that the integration of diffusion-based synthetic data generation with a quantile-regularized denoising objective provides a robust and effective framework for fraud detection in highly imbalanced datasets.

Abstract

Synthetic Data Generation

Explore similar work

Usefulness of Quantile-Aware Diffusion Modeling for Highly Imbalanced Tabular Data

Fraud is Not Just Rarity: A Causal Prototype Attention Approach to Realistic Synthetic Oversampling

Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals