A Feature Shuffling and Restoration Strategy for Universal Unsupervised Anomaly Detection

Wei Luo; Haiming Yao; Zhenfeng Qiang; Xiaotian Zhang; Weihang Zhang

cs.CVMar 24, 2026

A Feature Shuffling and Restoration Strategy for Universal Unsupervised Anomaly Detection

Authors: Wei Luo, Haiming Yao, Zhenfeng Qiang, Xiaotian Zhang, Weihang Zhang

Organizations: State Key Laboratory of Precision Measurement Technology and Instruments, Tsinghua University, Beijing, 100084, China · School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China

Abstract

Unsupervised anomaly detection is vital in industrial fields, with reconstruction-based methods favored for their simplicity and effectiveness. However, reconstruction methods often encounter an identical shortcut issue, where both normal and anomalous regions can be well reconstructed and fail to identify outliers. The severity of this problem increases with the complexity of the normal data distribution. Consequently, existing methods may exhibit excellent detection performance in a specific scenario, but their performance sharply declines when transferred to another scenario. This paper focuses on establishing a universal model applicable to anomaly detection tasks across different settings, termed as universal anomaly detection. In this work, we introduce a novel, straightforward yet efficient framework for universal anomaly detection: \uline{F}eature \uline{S}huffling and \uline{R}estoration (FSR), which can alleviate the identical shortcut issue across different settings. First and foremost, FSR employs multi-scale features with rich semantic information as reconstruction targets, rather than raw image pixels. Subsequently, these multi-scale features are partitioned into non-overlapping feature blocks, which are randomly shuffled and then restored to their original state using a restoration network. This simple paradigm encourages the model to focus more on global contextual information. Additionally, we introduce a novel concept, the shuffling rate, to regulate the complexity of the FSR task, thereby alleviating the identical shortcut across different settings. Furthermore, we provide theoretical explanations for the effectiveness of FSR framework from two perspectives: network structure and mutual information. Extensive experimental results validate the superiority and efficiency of the FSR framework across different settings.Code is available at https://github.com/luow23/FSR.

Figures & tables

Figure 1: Illustration of universal anomaly detection. The universal anomaly detection refers to comprehensive consideration of a model’s detection performance across few-shot, separate, and unified settings. (a) The few-shot setting involves training a model using only a limited number of samples from a specific category of product. (b) In the separate setting, a model is trained using a significant number of samples from a single category of product. (c) The unified setting entails training a single model using a substantial number of samples from multiple categories of products.

Figure 2: Comparison between reconstruction (Rec) task and our strategy across few-shot, separate, and unified settings. (a) Training loss (red) as well as the image/pixel AUROC ( blue / green ). In the Rec task, transitioning from a few-shot setting (Fs) to a separate setting (Ss) and subsequently to a unified setting (Us), the issue of identical shortcut becomes increasingly pronounced due to the growing complexity of the normal data distribution. On the contrary, our strategy is capable of addressing this issue across different settings. (b) Visualization of the shortcut problem, wherein the anomalous regions can be well reconstructed, making them difficult to distinguish from normal ones. In contrast, our strategy successfully addresses this issue, adeptly reconstructing anomalies as normal samples under various settings. Notably, all models are trained for feature reconstruction. We utilize an additional decoder to visualize the reconstructed features as images. This decoder is only designed for visualization.

Figure 3: The overall architecture of FSR framework. (a) The training stage comprises three steps: pre-trained feature extraction, random shuffling, and feature restoration. We use the difference between the input normal feature and restored feature as the feature restoration loss. (b) The testing stage incorporates only two steps: pre-trained feature extraction and feature restoration. We compute the anomaly score by comparing the difference between the input anomalous feature and restored feature, followed by an upsampling operation.

Figure 4: Illustration of multi-scale feature extraction.

Figure 5: The intuitive explanation of feature shuffling and restoration (FSR) effectiveness from the perspective of network structure. In reconstruction (Rec) task, shortcut learning issues exist, while there are no shortcut learning issues in FSR task.

Figure 6: The mathematical explanation of FSR effectiveness from the perspective of mutual informaton.

Category

PSVDD

PaDiM

MKD

DRAEM

SimpleNet

PatchCore

RD4AD

UniAD

Ours

[ 4 ]

[ 51 ]

[ 43 ]

[ 36 ]

[ 55 ]

[ 24 ]

[ 32 ]

[ 9 ]

[ 52 ]

(

\tau=0.1

)

(

\tau=0.9

)

Texture

Carpet

86.6/88.7

97.4/98.5

93.8/97.6

69.8/95.5

98.0/98.6

95.9/92.4

97.0/98.1

97.1/ 98.8

99.8 /98.5

99.0 / 98.8

99.0 / 99.0

Grid

69.2/64.5

99.1/97.5

73.9/71.0

83.8/82.3

99.3 / 98.7

49.8/46.7

91.4/98.4

99.7 / 99.2

98.2/96.5

96.8/98.3

98.7/98.3

Leather

97.2/95.4

95.1/98.1

99.9 /84.8

93.6/96.7

98.7/97.3

93.9/96.9

100 / 99.2

100 / 99.4

100 /98.8

100 /99.0

100 /99.1

Tile

93.7/82.7

97.3/82.5

93.3/80.5

89.5/85.3

99.8 / 98.0

93.7/93.1

96.0/90.3

97.5/ 95.6

99.3/91.8

100 /95.1

99.6/94.5

Wood

90.6/83.3

99.8 /92.6

98.4/89.1

93.4/80.5

99.8 / 96.0

95.2/84.8

93.8/90.8

99.2/ 96.0

98.6/93.2

98.1/ 94.5

99.7 /93.8

Table 2: Anomaly detection and localization results in terms of image/pixel level AUROC on MVTec AD dataset [ 3 ] under the unified setting.

k	TDG	DiffNet	PaDiM †	UniAD †	RegAD	Ours
k	[ 40 ]	[ 33 ]	[ 43 ]	[ 52 ]	[ 20 ]	( $\tau=0.1$ )
2	73.2/-	80.6/-	76.4/89.9	81.9/91.3	85.7 / 94.6	88.4 / 95.9
4	74.4/-	81.3/-	79.6/91.6	86.1/93.5	88.2 / 95.8	91.9 / 96.8
8	76.6/-	83.2/-	82.8/93.3	88.8/94.5	91.2 / 96.7	93.5 / 97.8

Table 3: Anomaly detection and localization results in terms of image/pixel level AUROC on MVTec AD dataset [ 3 ] under the few-shot setting. k represents the number of samples.

Setting		PaDiM	DRAEM	FastFlow	PatchCore	RD4AD	UniAD	Ours	Ours	Ours	Ours
Setting		[ 43 ]	[ 55 ]	[ 53 ]	[ 32 ]	[ 9 ]	[ 52 ]	( $\tau=0.1$ )	( $\tau=0.3$ )	( $\tau=0.9$ )	(Optimal)
MVTec AD	Separate	95.5/97.5	98.0/97.3	97.5 † / 98.4 †	99.1 /98.1	98.5/97.8	96.6/96.6	99.1 / 98.4	99.2 / 98.4	98.9/ 98.3	99.2 / 98.4
	Unified	84.2/89.5	88.1/87.2	91.1 † /96.2 †	96.4/95.7	93.4/96.0	96.5 / 96.8	96.6/ 97.9	97.1 / 98.0	98.3 / 98.0	98.3 / 98.0
	Few-shot-8	82.8 † /93.3 †	81.2 † /91.5 †	92.4 † / 97.0 †	92.1 † /96.5 †	90.5 † /95.9 †	88.8 † /94.5 †	93.5 / 97.8	92.5 / 97.4	90.4/96.6	93.5 / 97.8
	Mean	87.5/93.4	89.1/92.0	93.7/ 97.2	95.9 /96.8	94.1/96.6	94.0/96.0	96.4 / 98.0	96.3/97.9	95.9/97.6	97.0 / 98.1
BTAD	Separate	93.9 † /97.2 †	89.0/87.1	90.1/96.3	94.3 † /96.9 †	94.4 † / 97.5 †	93.8 † /97.0 †	95.6 / 97.6	95.9 /97.4	95.5/ 97.5	95.9 / 97.6

Table 4: Comprehensive comparative analysis of the proposed FSR method and existing approaches on MVTec AD [ 3 ] and BTAD [ 30 ] datasets. The best result of the existing methods is indicated by a wavy underline .

Figure 7: The qualitative segmentation results of the proposed model and the comparative methods under the unified setting on MVTec AD [ 3 ] benchmark. In contrast to alternative approaches, our method exhibits superior segmentation accuracy across various samples and defect types.

Figure 8: The reconstruction and anomaly segmentation results of our model. From top to bottom: input anomalous image, ground truth, reconstructed image, and predicted anomaly map. Notably, we utilize an additional decoder to visualize the reconstructed features as images. This decoder is solely designed for visualization purposes.

Method $\rightarrow$	FastRecon [ 12 ]	FOD [ 50 ]	DiAD [ 18 ]	Ours	Ours
Setting $\downarrow$	ICCV’23	ICCV’23	AAAI’24	( $\tau=0.1$ )	(Optimal)
Separate	95.7 † /97.8 †	99.2 / 98.3	90.8 † /95.7 †	99.1 / 98.4	99.2 / 98.4
Unified	72.0 † /87.5 †	92.6 † / 98.0 †	97.2 /96.8	96.6/ 97.9	98.3 / 98.0
Few-shot-8	95.2 / 97.3	83.4 † /94.5 †	85.0 † /92.2 †	93.5 / 97.8	93.5 / 97.8
Mean	87.6/94.2	95.1/97.0	91.0/94.9	96.4 / 98.0	97.0 / 98.1

Table 5: Comprehensive comparison with the latest SOTA methods on the MVTec-AD [ 3 ] dataset.

Figure 9: The qualitative segmentation results of proposed method on BTAD [ 30 ] dataset. Top row: The anomalous image. Mid row: The anomaly scoring map. Bottom row: The ground truth.

Setting $\rightarrow$	Unified	Seperate	Few-Shot-4
Seed $\downarrow$	Unified	Seperate	Few-Shot-4
1	96.6/97.9	99.1/98.4	91.9/96.8
42	96.6/97.9	99.1/98.4	91.8/96.8
123	96.5/97.8	99.1/98.5	91.7/96.9
2025	96.7/97.9	99.1/98.4	91.8/96.8
Mean	96.60/97.88	99.10/98.43	91.80/96.83

Table 6: Performance of our method under different settings across multiple random seeds on the MVTec-AD [ 3 ] dataset.

Figure 10: The impact of FSR strategy. (a) Quantitative ablation results. (b) Qualitative ablation results. First row: The anomalous image. Second row: The ground truth. Third row: The reconstruction result without FSR strategy. Last row: The reconstruction result with FSR strategy.

Figure 11: (a) The ablation results regarding positional embedding (PE). (b) The ablation results regarding Vision Transformer (ViT).

Figure 12: Ablation studies with image AUROC metric on MVTec AD [ 3 ] . (a) Anomaly detection performance with different shuffling rate $\tau$ . (b) Anomaly detection performance with different patch size $P$ . (c) Anomaly detection performance with different layer number $N$ .

Figure 13: Visualization of attention maps in reconstruction (Rec) task and feature shuffling and restoration (FSR) task. Each red box in the figure represents an image block, and we visualize its discrete similarity distribution with all the other image blocks.

Figure 14: Failure cases of our method. From top to bottom: input anomalous image, ground truth, and predicted anomaly map.

Explore similar work

Mar 24, 2026

URA-Net: Uncertainty-Integrated Anomaly Perception and Restoration Attention Network for Unsupervised Anomaly Detection

Unsupervised anomaly detection plays a pivotal role in industrial defect inspection and medical image analysis, with most methods relying on the reconstruction framework. However, these methods may suffer from over-generalization, enabling them to reconstruct anomalies well, which leads to poor detection performance. To address this issue, instead of focusing solely on normality reconstruction, we propose an innovative Uncertainty-Integrated Anomaly Perception and Restoration Attention Network (URA-Net), which explicitly restores abnormal patterns to their corresponding normality. First, unlike traditional image reconstruction methods, we utilize a pre-trained convolutional neural network to extract multi-level semantic features as the reconstruction target. To assist the URA-Net learning to restore anomalies, we introduce a novel feature-level artificial anomaly synthesis module to generate anomalous samples for training. Subsequently, a novel uncertainty-integrated anomaly perception module based on Bayesian neural networks is introduced to learn the distributions of anomalous and normal features. This facilitates the estimation of anomalous regions and ambiguous boundaries, laying the foundation for subsequent anomaly restoration. Then, we propose a novel restoration attention mechanism that leverages global normal semantic information to restore detected anomalous regions, thereby obtaining defect-free restored features. Finally, we employ residual maps between input features and restored features for anomaly detection and localization. The comprehensive experimental results on two industrial datasets, MVTec AD and BTAD, along with a medical image dataset, OCT-2017, unequivocally demonstrate the effectiveness and superiority of the proposed method.

Wei Luo, Peng Xing, Yunkang Cao +3

State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instrument, Tsinghua University, Beijing, China · School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China

Unsupervised Anomaly Detection Industrial Anomaly Detection Anomaly Localization Reconstruction-Based Anomaly Detection Medical Image Anomaly Detection

Ask PDF

Mar 24, 2026

Semantic Iterative Reconstruction: One-Shot Universal Anomaly Detection

Unsupervised medical anomaly detection is severely limited by the scarcity of normal training samples. Existing methods typically train dedicated models for each dataset or disease, requiring hundreds of normal images per task and lacking cross-modality generalization. We propose Semantic Iterative Reconstruction (SIR), a framework that enables a single universal model to detect anomalies across diverse medical domains using extremely few normal samples. SIR leverages a pretrained teacher encoder to extract multi-scale deep features and employs a compact up-then-down decoder with multi-loop iterative refinement to enforce robust normality priors in deep feature space. The framework adopts a one-shot universal design: a single model is trained by mixing exactly one normal sample from each of nine heterogeneous datasets, enabling effective anomaly detection on all corresponding test sets without task-specific retraining. Extensive experiments on nine medical benchmarks demonstrate that SIR achieves state-of-the-art under all four settings -- one-shot universal, full-shot universal, one-shot specialized, and full-shot specialized -- consistently outperforming previous methods. SIR offers an efficient and scalable solution for multi-domain clinical anomaly detection.

Ning Zhu

Medical Image Anomaly Detection Unsupervised Anomaly Detection Anomaly Detection Reconstruction-Based Anomaly Detection

Ask PDF

Jul 16, 2026

SwinAD: Multi-stage feature reconstruction for unsupervised industrial anomaly detection

Industrial anomaly detection aims to identify and localize defective regions without relying on exhaustive annotations of all possible defect types. Although recent unsupervised methods have achieved strong performance, most are primarily designed for single-class settings and often struggle in multi-class scenarios, where diverse normal patterns may lead to over-generalization and reduce the discriminative capability between normal and anomalous regions. In this paper, we propose SwinAD, a reconstruction-based framework for multi-class unsupervised anomaly detection that leverages a frozen pretrained Swin Transformer V2 encoder and a feature diversity-preserving reconstruction decoder. The hierarchical encoder provides semantically rich multi-scale features, while stage-wise bottleneck modules with dropout prevent trivial identity mapping and encourage robust reconstruction of normal patterns. To further improve localization, we introduce a feature diversity-preserving reconstruction framework that maintains complementary reconstruction hypotheses instead of relying on a single decoding branch. The discrepancies between encoder features and the two reconstructed features are then aggregated across multiple scales to produce the final anomaly map. Experiments conducted on three industrial anomaly detection benchmarks, including MVTec AD, VisA, and Real-IAD, demonstrate that SwinAD achieves competitive image-level performance and strong pixel-level localization accuracy, with particularly notable improvements in pixel-level AP and 1 on MVTec AD. These results indicate that combining hierarchical Swin features with diverse multi-scale reconstruction substantially improve pixel-level localization in multi-class unsupervised anomaly setting.

Huong Ninh, Chien Thai, Mai Xuan Trang +3

Institute of Artificial Intelligence, University of Engineering and Technology, Vietnam National University · Applied AI Lab, Phenikaa School of Computing, Phenikaa University

Anomaly Localization Reconstruction-Based Anomaly Detection Anomaly Detection

Ask PDF