f-Divergence Self-Play for Tabular Anomaly Detection via Large Language Models
Authors: Hoang Tran Vuong, Linh Ngo Van, Dang Nguyen, Thin Nguyen, Phuoc Nguyen, Mehrtash Harandi, Trung Le
Organizations: Hanoi University of Science and Technology, Hanoi, Vietnam · Applied Artificial Intelligence Initiative, Deakin University, Australia · Monash University, Clayton, VIC 3800, Australia
Anomaly detection in tabular data poses significant challenges due to heterogeneous feature types-mixing numerical, categorical, and textual attributes, which complicate learning meaningful representations of normality. Recent work has applied large language models (LLMs) to this problem by serializing table rows as text sequences, yet these approaches rely on one-shot supervised fine-tuning that offers limited signal to tighten the model's description of normality. We propose DiSPaT, a self-play fine-tuning framework that strengthens the model's understanding of normal data. Building on the theoretical foundation of f-divergence minimization, we derive a tight approximation connecting our training objective to reducing the distributional gap between real normal data and model-generated samples. DiSPaT operates through an alternating optimization: at each iteration, the current policy generates synthetic samples that serve as pseudo-anomalies, while a critic discriminator learns to distinguish these from real normal samples; this signal drives policy updates that progressively align the model distribution with the true normal-data distribution. Extensive experiments on diverse benchmarks demonstrate that DiSPaT consistently outperforms prior LLM-based methods, deep learning approaches, and classical unsupervised detectors for tabular anomaly detection.
Figures & tables
Methods \ Datasets
Fakejob
Lymphography
Seismic
Vehicle
20 Newsgroups
Ecoli
Average
Classical methods
Iforest
75.5%
67.3%
69.2%
49.6%
62.3%
85.6%
68.3%
PCA
72.4%
82.6%
69.2%
50.9%
62.3%
85.6%
70.5%
KNN
63.6%
86.0%
73.8%
52.4%
60.5%
87.7%
70.7%
ECOD
51.2%
83.0%
69.2%
50.9%
62.0%
77.6%
65.7%
Deep Learning based methods
Table 1: Detailed AUC-ROC scores (%) comparing DiSPaT against baselines on the six-dataset benchmark. Best and second-best results per dataset are highlighted in bold and underlined , respectively. Values marked with ↑ and ↓ indicate where DiSPaT improves or degrades compared to AnoLLM with the same model size. All baseline results are reported directly from ( Tsai et al., 2025 ) .
Methods \ Datasets
Fakejob
Lymphography
Seismic
Vehicle
20 Newsgroups
Ecoli
Average
Classical methods
Iforest
27.4%
23.3%
25.1%
11.0%
13.7%
75.6 %
29.4%
PCA
25.6%
56.7%
26.6%
12.4%
13.3%
77.8%
35.4%
KNN
16.3%
66.7%
29.1%
13.5%
15.6%
77.8%
36.5%
ECOD
16.5%
40.0%
28.2%
11.2%
13.2%
31.1%
23.4%
Deep Learning based methods
Table 2: Detailed F1 scores (%) comparing DiSPaT against baselines on six-dataset benchmark. Best and second-best per dataset are highlighted in bold and underlined ; ↑ and ↓ indicate where DiSPaT improves or degrades compared to AnoLLM with same model size. Baselines from ( Tsai et al., 2025 ) , except AnoLLM (SmolLM-1.7B) obtained via official implementation (not reported in original paper).
Methods \ Datasets
Fakejob
Lymphography
Seismic
Vehicle
20 Newsgroups
Ecoli
Average
Classical methods
Iforest
22.7%
23.2%
23.5%
11.2%
14.6%
8.6%
17.3%
PCA
19.4%
62.4%
21.6%
11.7%
14.3%
16.0%
24.2%
KNN
13.8%
72.0%
25.6%
12.3%
14.8%
73.9%
35.4%
ECOD
13.0%
36.5%
24.4%
11.6%
14.1%
18.9%
19.8%
Deep Learning based methods
Table 3: Detailed AUC-PR scores (%) comparing DiSPaT against baselines on six-dataset benchmark. Best and second-best per dataset are highlighted in bold and underlined ; ↑ and ↓ indicate where DiSPaT improves or degrades compared to AnoLLM with same model size. Baselines from ( Tsai et al., 2025 ) , except AnoLLM (SmolLM-1.7B) obtained via official implementation (not reported in original paper).
Figure 1: AUC-ROC scores (%) of DiSPaT across self-play iterations on four datasets using SmolLM-135M. SFT indicates initial model, and iter- k refers to model after k iterations of training.
Figure 2: Effect of clamping threshold ϵ on AUC-ROC scores (%) on two datasets: Ecoli and Vehicle, using SmolLM-360M.
f∗
Metric
Fakejob
Lymph.
Vehicle
Ecoli
Avg.
v
ROC
84.2%
99.3%
59.0%
92.3%
83.7%
F1
34.9%
83.3%
16.9%
44.4%
44.9%
PR
30.9%
94.4%
15.4%
37.9%
44.6%
ev−1
ROC
83.9%
100.0%
63.8%
94.9%
85.7%
F1
35.1%
100.0%
20.2%
55.6%
52.7%
PR
30.7%
100.0%
17.5%
55.6%
50.5%
Table 4: Performance comparison for different f -divergence choices on four datasets using SmolLM-135M.
Benchmark
SPIN
DiSPaT
ARC
34.90±1.39
36.52±1.41
TruthfulQA
39.59±1.45
39.71±1.46
Winogrande
61.09±1.37
62.33±1.37
GSM8K
29.57±1.26
33.89±1.30
HellaSwag
43.66±0.49
45.11±0.50
MMLU
44.75±10.05
45.23±9.83
Table 5: Comparison with SPIN in the standard alignment setting. We fine-tune Qwen1.5-1.8B on 50,000 UltraChat200k prompts and evaluate on HuggingFace Open LLM Leaderboard benchmarks. Results are reported as mean ± standard deviation.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
AnoLLM Time
DiSPaT Time
AnoLLM FLOPs
DiSPaT FLOPs
Ecoli
4.1
28.5
1.5×1015
3.7×1016
Lymphography
4.4
30.9
3.1×1015
1.7×1016
20 Newsgroups
4.7
35.2
3.2×1015
4.1×1016
Vehicle
6.6
48.3
6.4×1015
8.4×1016
Seismic
9.7
67.6
9.8×1015
1.2×1017
Fakejob
98.8
671.3
8.4×1016
7.7×1017
Appendix
Table 6: Training time and total FLOPs of AnoLLM and DiSPaT on six datasets using SmolLM-135M on a single NVIDIA A40 48GB GPU. Training time is reported in minutes.
Dataset
DiSPaT-360M
MCM
NPT-AD
DRL
LLM-DAS
CausalTAD
Annthyroid
93.1
68.9
86.8
92.4
90.7
89.3
Arr
84.3
81.1
71.9
77.4
77.6
83.6
Breast
99.6
99.6
98.5
99.7
62.0
99.7
Glass
82.3
72.3
78.8
69.7
91.1
67.1
Ionosphere
92.7
97.3
97.4
98.4
99.9
25.2
Lymphography
100.0
92.6
99.9
100.0
90.7
99.3
Appendix
Table 7: Detailed AUC-ROC scores (%) comparing DiSPaT against recent state-of-the-art baselines across 21 overlapping datasets. The best and second-best results in each row are highlighted in red and blue, respectively. The “Win/Tie” row indicates the number of datasets where DiSPaT outperforms or ties with the respective baseline. All baseline results are taken directly from the reported results in LLM-DAS ( Ye et al., 2026 ) and CausalTAD ( Wang et al., 2026a ) .
Datasets
Classical Methods
Deep-learning based Methods
LLM-based Methods
Iforest
PCA
KNN
ECOD
DeepSVDD
RCA
SLAD
GOAD
NeuTral
ICL
DTE
REPEN
135M
360M
1.7B
AnoLLM
DiSPaT
AnoLLM
DiSPaT
AnoLLM
DiSPaT
Annthyroid
92.2
83.9
81.1
79.0
74.2
71.8
76.1
57.2
81.3
84.2
97.7
73.6
92.7
92.9
93.1
93.1
93.0
94.0
Arrhythmia
82.7
79.6
78.6
81.1
76.5
78.6
78.4
68.1
76.0
78.5
77.1
68.4
82.5
81.4
82.2
84.3
82.4
82.4
BreastW
99.4
98.8
99.2
99.2
97.4
98.7
98.6
99.4
98.3
99.2
98.2
95.5
99.2
99.6
99.3
99.6
99.1
99.4
Cardio
94.8
96.6
92.1
93.5
84.2
94.8
84.0
52.4
85.9
89.4
92.0
82.9
94.0
94.9
87.3
91.7
86.7
87.7
Appendix
Table 8: Detailed AUC-ROC scores comparing DiSPaT against baselines over 30 datasets in ODDS. The best and second-best results in each row are indicated in red and blue, respectively. All baseline results are reported directly from ( Tsai et al., 2025 ) .
Datasets
Classical Methods
Deep-learning based Methods
LLM-based Methods
Iforest
PCA
KNN
ECOD
DeepSVDD
RCA
SLAD
GOAD
NeuTral
ICL
DTE
REPEN
135M
360M
1.7B
AnoLLM
DiSPaT
AnoLLM
DiSPaT
AnoLLM
DiSPaT
Annthyroid
57.4
48.7
44.0
38.8
43.6
36.7
41.8
25.7
46.8
50.1
78.9
33.8
58.4
59.2
59.7
62.0
62.6
64.6
Arrhythmia
61.2
54.2
55.4
59.1
53.3
54.2
53.6
50.3
51.5
53.3
52.1
45.2
61.2
60.4
60.0
62.1
62.1
60.7
BreastW
96.9
95.9
96.3
95.4
93.6
95.9
95.1
96.6
96.7
95.9
96.3
93.5
95.8
97.1
96.6
97.1
96.7
96.7
Cardio
71.5
80.8
67.6
66.6
56.4
72.6
60.2
29.4
56.8
68.9
64.4
56.1
73.4
72.2
66.5
65.9
63.4
64.2
Appendix
Table 9: Detailed F1 scores comparing DiSPaT against baselines over 30 datasets in ODDS. The best and second-best results in each row are indicated in red and blue, respectively. Baseline results are reported from ( Tsai et al., 2025 ) , except for AnoLLM (SmolLM-1.7B) which is obtained using the official implementation as they were not available in the original report.
Datasets
Classical Methods
Deep-learning based Methods
LLM-based Methods
Iforest
PCA
KNN
ECOD
DeepSVDD
RCA
SLAD
GOAD
NeuTral
ICL
DTE
REPEN
135M
360M
1.7B
AnoLLM
DiSPaT
AnoLLM
DiSPaT
AnoLLM
DiSPaT
Annthyroid
64.6
55.0
46.3
40.6
44.1
38.3
46.1
28.6
43.5
55.5
83.5
34.3
63.1
64.4
64.8
69.0
69.4
70.8
Arrhythmia
66.2
61.7
55.6
62.2
56.3
56.2
55.6
51.8
50.7
55.6
56.6
41.9
63.6
65.1
64.2
70.3
60.2
63.7
BreastW
99.4
98.5
99.2
99.2
96.6
98.6
98.3
99.4
97.0
99.1
96.7
92.5
99.1
99.6
99.2
99.6
99.2
99.5
Cardio
78.6
84.4
73.7
71.2
60.6
74.5
66.7
32.5
61.0
75.0
67.8
56.7
81.1
79.1
72.6
72.2
69.2
69.6
Appendix
Table 10: Detailed AUC-PR scores comparing DiSPaT against baselines over 30 datasets in ODDS. The best and second-best results in each row are indicated in red and blue, respectively. Baseline results are reported from ( Tsai et al., 2025 ) , except for AnoLLM (SmolLM-1.7B) which is obtained using the official implementation as they were not available in the original report.
f∗
Metric
Fakejob
Lymphography
Vehicle insurance
Ecoli
Seismic
20 Newsgroups
Average
v
AUC-ROC
84.2%
99.3%
59.0%
92.3%
73.2%
91.3%
83.2%
F1
34.9%
83.3%
16.9%
44.4%
32.4%
46.4%
43.1%
AUC-PR
30.9%
94.4%
15.4%
37.9%
26.6%
46.7%
42.0%
ev−1
AUC-ROC
83.9%
100.0%
63.8%
94.9%
73.2%
91.9%
84.6%
F1
35.1%
100.0%
20.2%
55.6%
33.5%
49.4%
49.0%
AUC-PR
30.7%
100.0%
17.5%
53.6%
26.9%
49.8%
46.4%
Appendix
Table 11: Performance comparison of SmolLM-135M with different f -divergence choices across datasets. Metrics reported are AUC-ROC, F1-score, and AUC-PR.
Figure 3: F1 scores (%) and AUC-PR scores (%) of DiSPaT across self-play iterations on four datasets using SmolLM-135M. SFT indicates initial model, and iter- k refers to model after k iterations of training.
Figure 4: Effect of clamping threshold ϵ on F1 scores (%) and AUC-PR scores (%) on two datasets using SmolLM-360M.
Dataset
Samples
Textual
Numerical
Categorical
Anomalies (%)
20 Newsgroup
11905
1
0
0
591 (5.0%)
Annthyroid
7200
0
6
0
534 (7.42%)
Arrhythmia
452
0
274
0
66 (15%)
BreastW
683
0
9
0
239 (35%)
Cardio
1831
0
21
0
176 (9.6%)
Ecoli
336
0
7
0
9 (2.6%)
Appendix
Table 12: Statistics for all datasets used in this work. Datasets are drawn from the AnoLLM benchmark ( Tsai et al., 2025 ) and the ODDS library ( Rayana, 2016 ) .
Figure 5: Iterative refinement of synthetic samples through self-play (Fakejob dataset). At iteration 0, the model generates severely oversimplified values (e.g., benefits = “$11” instead of a detailed description) and incorrect values (e.g., has questions = -1.0 instead of 1.0), clearly marking them as pseudo-anomalies. By iteration 1, the model generates structured values but lacks detail and context coherence (e.g., company profile = “Planning Centre” vs. a detailed company description; industry = “Computer Games” which mismatches the Design job context). By iteration 2, some features are correctly generated (e.g., function , telecommuting ), but textual fields remain generic rather than job-specific. This progression demonstrates how self-play enables the model to learn increasingly subtle distinctions between synthetic and real normal samples.
Figure 6: Iterative refinement on numerical features (Ecoli dataset). At iteration 0, the model generates malformed numerical values with formatting errors (e.g., “0 .” with trailing punctuation, or “0.1 . Both partial least squares…” with appended text), clearly marking them as pseudo-anomalies. By iteration 1, the model generates properly formatted numbers but with incorrect magnitudes (e.g., 1.7 instead of 1.2, 0.5 instead of -0.2). By iteration 2, while formatting is correct, subtle numerical discrepancies persist (e.g., 1.8 instead of 0.2, 1.7 instead of 1.2), demonstrating progressive refinement in learning numerical distributions.
Figure 7: Iterative refinement on mixed categorical and numerical features (Vehicle dataset). At iteration 0, the model generates incorrect categorical values (e.g., “3Sum” instead of “Sedan - Collision” for policy type, “1 to 2 years” instead of “5 years” for age of vehicle) and malformed numerical values (e.g., “0.4” instead of “Wednesday” for day of week, “0.6” instead of 0.2 for week of month claimed), clearly marking them as pseudo-anomalies. By iteration 1, categorical values improve but still contain errors (e.g., “No” instead of “Sedan - Collision” for policy type, “50” instead of “5 years” for age of vehicle, “Male” instead of “External” for agent type). By iteration 2, most categorical values are correct, but subtle numerical discrepancies persist (e.g., “7 years” instead of “5 years” for age of vehicle, “0.2” instead of -0.6 for week of month), demonstrating progressive refinement in learning both categorical and numerical distributions.