Anomalies as Residuals: Cauchy Displacement Networks
Authors: Yang Cao, Zihao Guo
Organizations: School of Computing and Information Technology, Great Bay University · Tsinghua Shenzhen International Graduate School, Tsinghua University, China · Institute for Advanced Research, Great Bay University, China
Neighbourhood-based detectors such as KNN, LOF and LUNAR remain among the most accurate methods for tabular anomaly detection, but they compare every query with the stored training set, so their inference cost and memory grow with the data and the data must be kept at deployment. We ask whether the neighbourhood geometry these detectors exploit can be compiled into a small network that scores a query with a single forward pass. We propose the Cauchy Displacement Network (CDN), which regresses each normal point onto the mean of its k nearest training neighbours with a single-hidden-layer network of Cauchy activations and scores a query by the length of its predicted displacement. This displacement is an empirical mean-shift vector, whose fixed-radius population counterpart locally approximates a scaled gradient of the log density. We show that Cauchy units exactly represent the score fields of a structured family of Student-t expert densities, and establish a constructive approximation bound for the corresponding population neighbourhood-mean map using a single-hidden-layer Cauchy network. We further show that the bounded network output guarantees asymptotically linear residual-score growth along every fixed direction. On 46 benchmark datasets against 28 baselines, CDN obtains the best mean rank under both AUC-ROC and AUC-PR, while scoring 104 queries in a few milliseconds on a single CPU thread, up to three orders of magnitude faster than neighbourhood-based detectors, with a model whose size does not depend on the training set.
Figures & tables
Fig. 1: Overview of CDN. Training: each normal point is paired with the mean of its k nearest training neighbours, and a single-hidden-layer network with Cauchy activations regresses the point onto this target. Inference: a query is scored by the length of the displacement between the query and the network output, using only the trained parameters.
Fig. 2: Anomaly scores on a two-dimensional example. White dots are normal training points from two clusters; red stars are example anomalies. The colour at each location is the score a query there would receive, rescaled to [0,1] within each panel (dark: low, judged normal; bright: high, judged anomalous). (a) Nonparametric mean shift ∥x−t(x)∥ , which needs the training data and a neighbour search for every query. (b) The same Cauchy network trained to reconstruct its input: the low-score region covers both clusters and a large empty area around them. (c) CDN: the low-score region follows the clusters as in (a), using only the network. All networks use the configuration of Section IV-E .
xi
N2(xi)
ti
mi=ti−xi
∣mi∣
r2(xi)
0
{1,2}
1.5
−1.5
1.5
2
1
{0,2}
1.0
−0.0
0.0
1
2
{1,0}
0.5
−1.5
1.5
2
5
{2,1}
1.5
−3.5
3.5
4
TABLE I: Neighbour-mean targets on four one-dimensional points with k=2 . r2 is the distance to the second nearest neighbour, i.e. the score of the KNN detector.
Method
Time per query
Memory at inference
KNN [ 6 ]
O(nd)
O(nd)
LOF [ 7 ]
O(nd)
O(nd)
MOD [ 50 ]
O(Ind)
O(nd)
LUNAR [ 8 ]
O(nd+∣θ∣)
O(nd+∣θ∣)
AE [ 10 ]
O(∣θ∣)
O(∣θ∣)
CDN
O(hd)
O(hd)
TABLE II: Cost of scoring one query and memory kept after training, assuming brute-force neighbour search. n : training size; d : dimensionality; h : hidden width of CDN; I : number of shift iterations of MOD; ∣θ∣ : number of network parameters.
Fig. 3: Anomaly score far from the data on the two-cluster example of Fig. 2 . Queries lie on 360 rays from the centre of the standardised data; the horizontal axis is the distance along the ray. (a) Median score over the rays. (b) Minimum score over the rays. All networks use the configuration of Section IV-E ; the dotted line is the distance itself.
Classical
Deep
Dataset
N
d
KNN
LOF
PCA
GMM
iForest
iNNE
ADERH
Sampl.
HBOS
ECOD
COPOD
QMCD
MOD
AE
VAE
DSVDD
DAGMM
ALAD
GANom.
SO-GAAL
LUNAR
DIF
GOAD
ICL
SLAD
DDPM
DTE-C
TCCM
CDN
ALOI
49534
27
51.0
48.8
54.0
54.6
51.3
54.1
53.5
52.8
52.2
51.7
49.5
51.0
50.3
53.2
53.5
49.3
51.9
50.7
52.4
55.4
50.7
53.8
47.3
47.5
50.9
50.6
50.6
50.5
52.9
annthyroid
7200
6
92.8
88.6
85.2
84.9
91.0
92.8
89.1
88.5
66.0
78.4
76.8
73.9
91.8
82.0
89.2
67.4
71.3
55.7
60.7
76.7
87.5
74.6
81.2
78.4
90.4
75.6
97.5
95.8
91.7
backdoor
95329
196
93.6
93.8
50.0
91.9
74.2
72.9
77.8
74.8
75.5
83.9
79.5
58.0
94.4
92.8
93.8
93.6
67.1
48.2
82.8
87.9
94.4
91.6
65.8
92.6
50.0
64.6
90.2
95.3
93.7
breastw
683
9
99.2
97.2
99.4
98.7
99.8
98.0
99.2
98.9
99.6
99.5
99.8
44.4
98.9
98.9
99.3
98.1
95.5
62.1
95.2
10.1
99.2
77.6
98.5
97.5
99.5
94.5
91.0
98.1
99.2
campaign
41188
62
78.5
70.6
50.0
79.9
74.2
77.3
71.4
75.2
77.1
76.9
78.1
79.6
77.3
81.0
76.0
73.3
61.0
53.8
69.0
65.8
71.9
67.4
41.2
81.2
76.2
71.2
79.1
76.1
83.2
TABLE III: AUC-ROC ( ×100 ) on 46 datasets with their total sample size N and dimensionality d . Best in bold, second best underlined.
Classical
Deep
Dataset
KNN
LOF
PCA
GMM
iForest
iNNE
ADERH
Sampl.
HBOS
ECOD
COPOD
QMCD
MOD
AE
VAE
DSVDD
DAGMM
ALAD
GANom.
SO-GAAL
LUNAR
DIF
GOAD
ICL
SLAD
DDPM
DTE-C
TCCM
CDN
ALOI
6.0
6.5
6.5
6.5
5.9
6.5
6.4
6.3
6.4
6.1
5.7
6.0
6.0
6.2
6.4
5.7
6.3
6.2
7.6
6.9
6.7
7.2
5.6
5.5
5.9
6.0
5.8
5.9
6.3
annthyroid
68.1
53.5
56.4
56.0
61.2
69.2
59.5
62.8
39.0
40.0
29.6
35.5
65.2
58.5
58.5
37.0
29.7
17.6
22.7
32.3
55.7
46.1
59.8
41.8
66.9
47.1
82.9
74.8
63.6
backdoor
48.0
37.4
4.8
85.8
9.0
15.7
14.4
12.3
10.5
15.5
13.1
5.8
50.5
86.0
85.9
83.3
9.0
4.9
16.2
25.1
69.8
56.1
13.3
79.4
4.8
10.3
60.7
67.9
55.6
breastw
99.0
94.5
99.4
98.3
99.7
97.8
98.9
98.6
99.6
99.6
99.8
63.7
98.5
98.4
99.2
97.5
95.5
65.2
94.9
36.9
99.0
68.3
98.6
96.3
99.5
95.2
87.6
98.1
99.2
campaign
49.0
40.2
20.2
51.4
45.3
47.3
46.0
48.7
49.7
49.5
51.0
48.4
46.1
49.4
44.4
44.0
32.8
23.5
36.9
40.6
41.8
37.8
20.9
49.4
47.8
45.1
48.4
46.6
53.7
TABLE IV: AUC-PR ( ×100 ) on 46 datasets. Best in bold, second best underlined.
Fig. 4: Rank distributions over 46 datasets (lower is better). Methods are sorted by mean rank, shown below each name.
AUC-ROC
AUC-PR
Variant
Avg.
Rank
Avg.
Rank
CDN
81.2
2.49
57.4
2.39
ReLU activation
80.3
2.74
53.9
3.04
tanh activation
80.5
2.74
55.9
2.76
Self-reconstruction target
80.1
3.10
56.7
2.72
Skip connection x+gθ(x)
72.3
3.93
38.5
4.09
TABLE V: Ablation of CDN on 46 datasets. Avg.: average ( ×100 ); Rank: mean rank among the five variants.
Fig. 5: CDN on the four types of anomalies of ADBench in two dimensions: (a) global, (b) local, (c) clustered, (d) dependency. Background: anomaly score (brighter is higher, as in Fig. 2 ). Orange: region judged normal, with the threshold at the largest anomaly score of a normal test point. White: normal test points; black: anomalies. The AUC-ROC is computed on 500 normal test points and the anomalies shown.
Fig. 6: Running time on synthetic Gaussian data (log scales). (a) Time to score 104 queries as the training size n grows with d=20 . (b) Training time for the same settings. (c) Time to score 104 queries as d grows with n=104 . CPU methods use a single thread; dashed lines are deep baselines run on the GPU.
Fig. 7: Accuracy against inference cost. Vertical axis: mean rank over the 46 datasets among all 29 methods (lower is better). Horizontal axis: time to score 104 queries with n=105 and d=20 (log scale). Squares are deep baselines run on the GPU.
AUC-ROC
AUC-PR
Setting (default)
Value
Avg.
Rank
Pos.
Avg.
Rank
Pos.
Default
81.2
8.65
1
57.4
8.20
1
h (128)
32
80.6
9.32
3
55.7
9.39
3
64
81.0
8.93
1
56.6
8.61
2
β0 (0.2)
0.01
81.1
8.55
1
57.2
8.39
1
0.1
81.2
8.65
1
57.2
8.11
1
TABLE VI: Sensitivity of CDN to its settings. Each row changes one setting from its default value, given in parentheses, and keeps the others at their defaults. Avg.: average over 46 datasets ( ×100 ); Rank: mean rank among the 29 methods; Pos.: position of CDN in that ranking.