Hierarchical Reference Sets for Robust Unsupervised Detection of Scattered and Clustered Outliers

Yiqun Zhang; Zexi Tan; Xiaopeng Luo; Yunlin Liu

cs.LGMar 13, 2026

Hierarchical Reference Sets for Robust Unsupervised Detection of Scattered and Clustered Outliers

Authors: Yiqun Zhang, Zexi Tan, Xiaopeng Luo, Yunlin Liu

Organizations: School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China

Abstract

Most real-world IoT data analysis tasks, such as clustering and anomaly event detection, are unsupervised and highly susceptible to the presence of outliers. In addition to sporadic scattered outliers caused by factors such as faulty sensor readings, IoT systems often exhibit clustered outliers. These occur when multiple devices or nodes produce similar anomalous measurements, for instance, owing to localized interference, emerging security threats, or regional false alarms, forming micro-clusters. These clustered outliers can be easily mistaken for normal behavior because of their relatively high local density, thereby obscuring the detection of both scattered and contextual anomalies. To address this, we propose a novel outlier detection paradigm that leverages the natural neighboring relationships using graph structures. This facilitates multi-perspective anomaly evaluation by incorporating reference sets at both local and global scales derived from the graph. Our approach enables the effective recognition of scattered outliers without interference from clustered anomalies, whereas the graph structure simultaneously helps reflect and isolate clustered outlier groups. Extensive experiments, including comparative performance analysis, ablation studies, validation on downstream clustering tasks, and evaluation of hyperparameter sensitivity, demonstrate the efficacy of the proposed method. The source code is available at https://github.com/gordonlok/DROD.

Figures & tables

Fig. 1: Illustration of scatterliers (gray cross), clusterliers (red square), collective outliers (red cross) and the masking effect in outlier detection (red dotted arrow) of a dataset with normal clustered samples (dotted circle).

Section	Symbol	Definition
III	$x_{i}$	Data sample
	$NB(x_{i})$	Natural neighbor set of $x_{i}$
	$kNN_{\lambda}(x_{i})$	$k$ -nearest neighbors with scope $\lambda$
	$NBG$	Natural Neighborhood Graph
	$NBG_{ij}$	Edge indicator in $NBG$ between $x_{i},x_{j}$
IV-A	$\rho(x_{i})$	Local natural density of $x_{i}$

TABLE I: Explanation of Symbols.

Fig. 2: Pipeline of the proposed method. Natural neighbor subsets containing natural neighboring samples are constructed based on the NB definition in Section III . Then, the Subset Anomaly Index (SAI) and Local Anomaly Index (LAI) are computed to comprehensively reflect the sample abnormality under the complex co-occurrence of scatterliers and clusterliers. SAI is derived from the degree of adjacency (i.e., link strength) among subsets, indicating the subset-level abnormality of samples. LAI measures the local abnormality of samples within each subset based on distribution density. These two indices collectively enable the detection of outliers without bias and clusterliers’ masking effect.

Case	SAI	LAI	Anomaly Type	Description
1	High	High	Scatterlier	Clear abnormality with dual evidence
2	High	Low	Clusterlier	Core member of anomalous micro-cluster
3	Low	High	Potential noise	Local isolation, global normality
4	Low	Low	Normal sample	No significant anomaly evidence

TABLE II: Sample types under different SAI and LAI combinations.

Real dataset	Samples	Features	Anomaly(%)	Reference
PageBlocks	5393	10	510 (9.46)	[ 36 ]
WPBC	198	33	47 (23.74)	[ 37 ]
mnist	7603	100	700 (9.21)	[ 38 ]
musk	3062	166	97 (3.17)	[ 39 ]
Ionosphere	351	33	126 (35.90)	[ 40 ]
Waveform	3443	21	100 (2.90)	[ 41 ]

TABLE III: Statistics of 20 real benchmark datasets.

Fig. 3: Two basic synthetic datasets C1 and C2 for generating various types of outliers.

Fig. 4: Visualization of synthetic datasets D1-D12.

Dataset	Normal Samples	Scatterliers	Clusterliers
D1	1000	0	(100,100)
D2	266	0	(26,26)
D3	1000	50	(50)
D4	1000	100	(50,50)
D5	1000	150	(30,50,70)
D6	266	20	(20)

TABLE IV: Statistics of 12 synthetic datasets and the counts of scatterliers and clusterliers.

Datasets	COPOD	ECOD	OCSVM	IFOREST	$k$ NN+KFC	LOF+KFC	DGOF+KFC	CBLOF	DROD
	[ 57 ] ’2020	[ 58 ] ’2022	[ 55 ] ’2023	[ 56 ] ’2023	[ 53 ] ’2024	[ 53 ] ’2024	[ 53 ] ’2024	[ 54 ] ’2025	ours
D1	0.5070	0.4149	0.4932	0.6759	0.1266	0.4887	0.3562	0.2415	0.8330
D2	0.5227	0.3758	0.5000	0.5454	0.8228	0.5856	0.6076	0.0753	0.9180
Avg. AUC	0.5149	0.3954	0.4966	0.6107	0.4747	0.5732	0.4819	0.1573	0.8755

TABLE V: AUC comparison of 9 methods on D1 and D2 with only clusteriers . The best results are emphasized in bold .

Panel of Figure 10 — Fig. 5: AUC comparison on D1-D12. D1 and D2 are with only clusterliers . The ten datasets D3-D12 are with both scatterliers and clusterliers .

Datasets	COPOD	ECOD	OCSVM	IFOREST	$k$ NN+KFC	LOF+KFC	DGOF+KFC	CBLOF	DROD
	[ 57 ] ’2020	[ 58 ] ’2022	[ 55 ] ’2023	[ 56 ] ’2023	[ 53 ] ’2024	[ 53 ] ’2024	[ 53 ] ’2024	[ 54 ] ’2025	ours
Pageblocks	0.8754(6)	0.9139(2)	0.8903(5)	0.8977(4)	0.8061(8)	0.7512(9)	0.8413(7)	0.9083(3)	0.9154(1)
WPBC	0.5233(4)	0.4813(7)	0.4743(8)	0.4938(5)	0.5323(2)	0.4932(6)	0.5600(1)	0.4686(9)	0.5249(3)
mnist	0.7739(8)	0.7463(9)	0.8204(3)	0.8008(5)	0.8655(1)	0.8019(4)	0.7829(7)	0.7913(6)	0.8603(2)
musk	0.9463(7)	0.9559(6)	0.8094(8)	0.9996(4)	1.0000(2)	0.8014(9)	1.0000(2)	1.0000(2)	0.9888(5)
Ionosphere	0.7895(7)	0.7284(9)	0.7395(8)	0.8503(4)	0.8300(6)	0.8889(2)	0.8379(5)	0.9013(1)	0.8598(3)

TABLE VI: AUC of 9 methods on 20 real benchmark datasets. The performance rank (the lower, the better) is shown in parentheses, and the best results are emphasized in bold .

Datasets	COPOD	ECOD	OCSVM	IFOREST	$k$ NN+KFC	LOF+KFC	DGOF+KFC	CBLOF	DROD
	[ 57 ] ’2020	[ 58 ] ’2022	[ 55 ] ’2023	[ 56 ] ’2023	[ 53 ] ’2024	[ 53 ] ’2024	[ 53 ] ’2024	[ 54 ] ’2025	ours
Pageblocks	0.3353(9)	0.4314(4)	0.3670(8)	0.4186(6)	0.4310(5)	0.3892(7)	0.4324(3)	0.6078(1)	0.5235(2)
WPBC	0.2128(3)	0.1277(9)	0.1702(6.5)	0.1553(8)	0.1773(5)	0.1809(4)	0.2199(2)	0.1702(6.5)	0.2340(1)
mnist	0.2357(8)	0.1800(9)	0.3710(3)	0.2990(6)	0.4210(2)	0.2502(7)	0.3510(4)	0.3329(5)	0.5443(1)
musk	0.3608(6)	0.4948(5)	0.0223(9)	0.9495(2)	0.5052(4)	0.0464(7)	0.0395(8)	1.0000(1)	0.5361(3)
Ionosphere	0.5873(8)	0.5317(9)	0.8479(1)	0.6706(7)	0.7315(5)	0.7659(4)	0.7262(6)	0.8254(2)	0.7698(3)

TABLE VII: Precision-

s

of 9 methods on 20 benchmark datasets. The performance rank (the lower, the better) is shown in parentheses, and the best results are emphasized in bold .

Counterparts	R+	R-	p-value	null hypothesis ( $\alpha$ =0.05)
DROD vs $k$ NN+KFC	165	45	0.0240	Reject
DROD vs LOF+KFC	201	9	0.0001	Reject
DROD vs DGOF+KFC	183	27	0.0023	Reject
DROD vs IFOREST	191	19	0.0006	Reject
DROD vs CBLOF	180	30	0.0037	Reject
DROD vs OCSVM	203	7	0.0001	Reject

TABLE VIII: The Wilcoxon signed-rank test results on real datasets using AUC as the metric.

Fig. 6: Runtime performance evaluation under two scalability scenarios: Left : varying the sample size from 1,000 to 6,000 with fixed dimensionality

d

=36; Right : varying the dimensionality from 40 to 400 with fixed sample size

n

=3,686.

Datasets	NRS (ours)	GB	GNAN	HDIOD
Pageblocks	0.9154	0.4562	0.5249	0.8919 ( $k$ =6)
WPBC	0.5249	0.4584	0.4760	0.5942 ( $k$ =4)
Ionosphere	0.8598	0.7271	0.7586	0.5867 ( $k$ =6)
Cardiotocography	0.7431	0.5687	0.5107	0.7831 ( $k$ =7)
cardio	0.9368	0.5275	0.6151	0.9407 ( $k$ =8)
landsat	0.6290	0.4889	0.4849	0.6241 ( $k$ =3)

TABLE IX: AUC Results of Replacing DROD’s Component with NRS and Other Competing Methods. The best results are highlighted in bold and the value of

k

in HDIOD represents the parameter setting for the method.

Datasets	Scales (n $\times$ d)	Euclidean	Mahalanobis	Chebyshev
Pageblocks	5393 $\times$ 10	0.9154	0.9129	0.9096
WPBC	198 $\times$ 33	0.5249	0.4999	0.4841
Ionosphere	351 $\times$ 33	0.8598	0.8602	0.8241
Cardiotocography	351 $\times$ 33	0.7431	0.6942	0.7829
Cardio	1831 $\times$ 21	0.9368	0.9370	0.9407
Landsat	6435 $\times$ 36	0.6290	0.4499	0.6221

TABLE X: Comparative AUC performance of DROD with different distance metrics on datasets of various scales. The best result for each dataset is highlighted in bold .

Datasets	LOF+KFC	DROD-L	DROD-S	DROD-0	DROD
Pageblocks	0.7263	0.7250	0.9020	0.9116	0.9154
WPBC	0.4932	0.4973	0.5545	0.5546	0.5249
mnist	0.8019	0.4361	0.7863	0.7599	0.8603
musk	0.8014	0.7270	0.9604	0.9481	0.9888
Ionosphere	0.8889	0.7543	0.8702	0.8504	0.8598
Waveform	0.7826	0.3864	0.8220	0.8182	0.8755

TABLE XI: AUC performance comparison of baseline and three ablated versions of DROD. The best performance is marked in bold .

Detectors	AUC	DBI ( $\Delta$ )
Random	-	1.9578 (0.0000)
$k$ NN+KFC	0.5249	1.9132 ( -0.0446 )
LOF+KFC	0.3921	1.9227 ( -0.0351 )
DGOF+KFC	0.2831	1.9117 ( -0.0461 )
IFOREST	0.7140	1.9341 ( -0.0237 )
CBLOF	0.7547	2.0125 ( +0.0547 )

TABLE XII: DBI of

K

-means on “optdigits” dataset with outlier removal using the nine compared methods as detectors. The AUC performance x for reference, and the symbol “

\Delta

” denotes boosted DBI based on the DBI performance without outlier removal. Results indicating better or worse performance are marked in green or red , respectively.

Fig. 7: AUC of DROD on 20 datasets with sampling rate

\eta

from

[0.1,1]

with step size 0.1 under sampling times

T=100

Fig. 8: AUC of DROD on 20 datasets with sampling times

T

from

[10,100]

with step size 10 under sampling rate

\eta=0.8

Panel of Figure 21 — Fig. 9: Visualization of DAI mechanism through t-SNE embedding: (a) Distribution of normal data and ground-truth anomalies; (b) DAI values of predicted anomalies; (c) LAI values of predicted anomalies; (d) SAI values of predicted anomalies, demonstrating the synergistic relationship between local and global anomaly indicators.

Explore similar work

Clustering-Based Collective Anomaly Detection in IoT Systems: A Graph Neural Network Approach
Aug 26, 2026Dalila Khettaf, Djamel Djenouri, Zeinab Rezaeifar +1Anomaly Detection Unsupervised Anomaly Detection
University of the West of England(UWE) Bristol BS16 1QY, England · University of South-Eastern Norway, Kongsberg, 3616 Norway
Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection
Feb 28, 2026Li Sun, Lanxu Yang, Jiayu Tian +6OOD Detection Synthetic Anomaly Generation
North China Electric Power University · Beijing University of Posts and Telecommunications · Tsinghua University +4
An Efficient Unsupervised Federated Learning Approach for Anomaly Detection in Heterogeneous IoT Networks
Feb 27, 2026Mohsen Tajgardan, Atena Shiranzaei, Mahdi Rabbani +2Unsupervised Anomaly Detection Anomaly Detection
Faculty of Electrical and Computer Engineering, Qom University of Technology, Qom, Iran · Department of Computer Engineering, Faculty of Industry and Mining (Khash), Sistan and Baluchestan University, Zahedan, Iran · Canadian Institute for Cybersecurity, University of New Brunswick, NB, Canada +1

Aug 26, 2026J/K move · Enter open · S save

Clustering-Based Collective Anomaly Detection in IoT Systems: A Graph Neural Network Approach

Dalila Khettaf, Djamel Djenouri, Zeinab Rezaeifar, Youcef Djenouri

University of the West of England(UWE) Bristol BS16 1QY, England · University of South-Eastern Norway, Kongsberg, 3616 Norway

Anomaly Detection Unsupervised Anomaly Detection IoT Intrusion Detection

Open paper AskPDF arXiv

The rapid advancement of Internet of Things (IoT) technology has led to the widespread deployment of smart, interconnected devices across a range of domains. However, this expansion has also resulted in a substantial increase in network traffic, creating more opportunities for malicious actors to launch cyberattacks and compromise sensitive information, thereby increasing the need for effective anomaly detection. The state-of-the-art in anomaly detection has predominantly focused on point anomalies. In contrast, the detection of collective anomalies remains relatively under-explored in the literature. In this paper, we introduce Unsupervised Graph Collective Anomaly Detection (UGCAD), a novel frame- work designed to identify collective anomalies in IoT network traffic. Unlike many existing methods, UGCAD operates on graph-structured data without any prior knowledge of group labels or membership. It leverages a variational graph autoencoder (VGAE) to learn the graph representation, which is subsequently used to enhance a clustering algorithm for effective grouping of nodes. To detect collective anomalies, clusters identified as normal are first aggregated and refined, after which anomaly scores are applied to detect collective anomalies. Extensive experiments conducted on the CICIoT2023 and ToN-IoT network datasets demonstrate the effectiveness of UGCAD in both clustering and collective anomaly detection (CAD). Furthermore, comparative evaluations against several traditional and state-of-the-art clustering-based CAD approaches confirm the superiority of UGCAD in accurately detecting collective anomalies.

Abstract

Explore similar work

Clustering-Based Collective Anomaly Detection in IoT Systems: A Graph Neural Network Approach

Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection

An Efficient Unsupervised Federated Learning Approach for Anomaly Detection in Heterogeneous IoT Networks