One-to-More: High-Fidelity Training-Free Anomaly Generation with Attention Control

Haoxiang Rao; Zhao Wang; Chenyang Si; Yan Lyu; Yuanyi Duan; Fang Zhao; Caifeng Shan

cs.CVMar 18, 2026

One-to-More: High-Fidelity Training-Free Anomaly Generation with Attention Control

Authors: Haoxiang Rao, Zhao Wang, Chenyang Si, Yan Lyu, Yuanyi Duan, Fang Zhao, Caifeng Shan

Organizations: Nanjing University · China Mobile Zijin Innovation Institute · Shandong University of Science and Technology

Abstract

Industrial anomaly detection (AD) is characterized by an abundance of normal images but a scarcity of anomalous ones. Although numerous few-shot anomaly synthesis methods have been proposed to augment anomalous data for downstream AD tasks, most existing approaches require time-consuming training and struggle to learn distributions that are faithful to real anomalies, thereby restricting the efficacy of AD models trained on such data. To address these limitations, we propose a training-free few-shot anomaly generation method, namely O2MAG, which leverages the self-attention in One reference anomalous image to synthesize More realistic anomalies, supporting effective downstream anomaly detection. Specifically, O2MAG manipulates three parallel diffusion processes via self-attention grafting and incorporates the anomaly mask to mitigate foreground-background query confusion, synthesizing text-guided anomalies that closely adhere to real anomalous distributions. To bridge the semantic gap between the encoded anomaly text prompts and the true anomaly semantics, Anomaly-Guided Optimization is further introduced to align the synthesis process with the target anomalous distribution, steering the generation toward realistic and text-consistent anomalies. Moreover, to mitigate faint anomaly synthesis inside anomaly masks, Dual-Attention Enhancement is adopted during generation to reinforce both self- and cross-attention on masked regions. Extensive experiments validate the effectiveness of O2MAG, demonstrating its superior performance over prior state-of-the-art methods on downstream AD tasks.

Figures & tables

Figure 2: Overview of the proposed O2MAG. Our method synthesize anomalies by coordinating self-attention in three parallel diffusion process. DAE operates on the target branch’s self-/cross-attention to enforce full-mask filling, while AGO refines the anomaly text embedding to align semantics with reference anomaly image to make generation more realistic.

Figure 3: (a) The intermediate reconstruction during the iterative denoising process, and (b) visualization of the self-attention map A at the 30th sampling step. Principal components preserve the image layout and regions rendered with the same color share semantic attributes.

Figure 4: Anomaly-Guided Optimization (AGO) pipeline. Numbers indicate different synthesized samples from $\boldsymbol{Z}_{T}^{ref}$ using the current optimized prompt embedding.

Category	DFMGAN [ 9 ]				AnomalyDiffusion [ 18 ]				DualAnoDiff [ 20 ]				SeaS [ 8 ]				Ours
Category	AP-I	AUC-P	AP-P	F1-P	AP-I	AUC-P	AP-P	F1-P	AP-I	AUC-P	AP-P	F1-P	AP-I	AUC-P	AP-P	F1-P	AP-I	AUC-P	AP-P	F1-P
bottle	99.8	98.9	90.2	83.9	99.9	99.4	94.1	87.3	100	99.5	93.4	85.7	99.9	99.7	95.9	88.8	100	99.7	95.4	88.4
cable	97.8	97.2	81.0	75.4	100	99.2	90.8	83.5	98.3	98.5	82.6	76.9	98.8	96.0	83.1	77.7	100	99.4	91.2	85.0
capsule	98.5	79.2	26.0	35.0	99.9	98.8	57.2	59.8	99.2	99.5	73.2	67.0	99.2	93.7	41.9	47.3	99.8	97.0	60.6	59.0
carpet	98.5	90.6	33.4	38.1	98.8	98.6	81.2	74.6	99.9	99.4	89.1	80.2	99.0	99.3	86.4	78.1	100	99.5	88.5	80.0
grid	90.4	75.2	14.3	20.5	99.5	98.3	52.9	54.6	99.7	98.5	57.2	54.9	99.9	99.7	76.3	70.0	100	99.6	78.6	71.6

Table 1: Comparison on a trained U-Net segmentation model [ 31 ] for anomaly detection and localization on MVTec-AD dataset.

Category	DFMGAN [ 9 ]		AnoDiff [ 18 ]		DualAno [ 20 ]		SeaS [ 8 ]		Ours
Category	KID $\downarrow$	IC-L $\uparrow$	KID $\downarrow$	IC-L $\uparrow$	KID $\downarrow$	IC-L $\uparrow$	KID $\downarrow$	IC-L $\uparrow$	KID $\downarrow$	IC-L $\uparrow$
bottle	70.90	0.12	137.71	0.17	151.09	0.31	148.29	0.21	42.78	0.17
cable	61.68	0.25	88.44	0.40	86.01	0.45	108.62	0.43	28.93	0.41
capsule	40.63	0.11	111.30	0.18	72.82	0.32	183.11	0.29	22.71	0.17
carpet	25.14	0.13	107.87	0.21	56.80	0.27	127.62	0.26	45.10	0.29
grid	101.04	0.13	51.55	0.45	64.13	0.44	60.85	0.41	7.04	0.38

Table 2: Comparison on KID and IC-LPIPS on MVTec dataset. Bold and underline represent optimal and sub-optimal results, respectively. Our method generates anomaly images closest to the real distribution, achieving the lowest KID. Although our IC-LPIPS is lower than some methods, their higher IC-LPIPS partly results from corrupting the normal background.

Figure 5: Qualitative comparison of generated results on MVTec-AD. The sub-image in the lower right corner is anomaly mask.

Metric	AUC-I	F1-I	AUC-P	F1-P
AnomalyAny	98.4	96.9	97.4	65.1
Ours	99.6 (+ 1.2 )	99.2 (+ 2.3 )	99.2 (+ 1.8 )	80.6 (+ 15.5 )

Table 3: Comparison to training-free AnomalyAny on MVTec-AD.

Category	DFMGAN [ 9 ]	AnoDiff [ 18 ]	TF 2 [ 41 ]	DualAno [ 20 ]	SeaS [ 8 ]	Ours
bottle	56.59	88.37	79.36	67.44	81.40	86.05
cable	45.31	76.56	60.00	57.81	48.44	76.56
capsule	37.23	44.00	53.12	50.67	33.33	77.33
carpet	47.31	58.06	55.05	62.90	38.71	74.19
grid	40.83	60.00	47.36	67.50	47.50	80.00
hazelnut	81.94	81.25	88.88	79.17	81.25	95.83

Table 4: Anomaly classification accuracy (%) with ResNet-34 [ 13 ] trained on images synthesized by each method.

Figure 6: Qualitative comparison of generated hazelnut-hole .

Method	Image-level			Pixel-level
Method	AUC-I	AP-I	F1-I	AUC-P	AP-P	F1-P
DREAM	99.6	99.1	94.7	99.3	74.8	73.3
AnomalyAny	100.0	100.0	100.0	98.6	78.8	75.0
SeaS-zero	100.0	100.0	100.0	97.5	78.6	73.7
Ours-zero	100.0	100.0	100.0	99.1	86.8	80.2
SeaS-origin	100.0	100.0	100.0	99.2	87.3	80.0

Table 5: Comparison between zero-shot and original settings on hazelnut–hole subset of MVTec-AD for anomaly detection.

Method			Metric
TriAG	DAE	AGO	AUROC $\uparrow$	AP $\uparrow$	F1-max $\uparrow$
✓			99.0	81.9	77.6
✓	✓		99.0	83.3	77.9
✓		✓	99.1	82.7	77.7
✓	✓	✓	99.2	86.3	80.8

Table 6: Ablation on components. ✓ denotes enabled.

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: (a) The Demonstration of O2MAG Anomaly Generation, and (b) Self-attention feature maps visualized at the 30th denoising step. In each triplet of subfigures, the maps correspond (left $\rightarrow$ right) to the normal image, the reference anomaly image, and the generated image.

Figure 8: The intermediate reconstruction during the iterative denoising process. For both texture and object categories, the global image layout is already formed in the early denoising steps.

Setting	AUC-I	AP-I	F1-I	AUC-P	AP-P	F1-P
Origin	99.9	100.0	99.0	99.8	96.2	90.1
Augment	99.9	99.9	98.9	99.8	95.9	91.0

Appendix

Table 7: Comparison between the original and augmented settings.

Figure 9: Analysis of the optimization step in AGO.

Figure 10: Analysis of the scaling factor $C$ in DAE.

Methods	Training Overall Time	Inference Time (per image)
DFMGAN [ 9 ]	464 hours	0.9 s
AnomalyDiffusion [ 18 ]	310 hours	7.4 s
DualAnoDiff [ 20 ]	197 hours	32.4 s
SeaS [ 8 ]	124 hours	10.7 s
AnomalyAny [ 41 ]	0 hours	120s
O2MAG (ours)	0 hours	28s

Appendix

Table 8: Comparison of training cost and inference speed.

Figure 11: Anomaly generation under zero-shot settings. We aim to transfer anomalous features of the same anomaly type from “Reference Anomaly” images of other categories to the target category, synthesizing realistic defects that are consistent with ”Real Anomaly”.

Figure 12: Qualitative ablation of TriAG, AGO, and DAE for anomaly realism and localization.

Mask Source	AUROC-I	AP-I	F1-I	AUROC-P	AP-P	F1-P	Pro
Perlin noise	100	100	100	99.8	95.5	89.8	94.0
Nano Banana	100	100	100	99.9	97.3	92.1	95.1
AnomalyDiffusion	100	100	100	99.9	96.3	90.6	95.2
SeaS	100	100	100	99.6	94.8	89.3	95.9

Appendix

Table 9: Robustness to different mask source for our method.

Figure 13: Qualitative comparison of generated results on VisA. The sub-image in the lower right corner is anomaly mask.

Category	AnomalyDiffusion [ 18 ]		DualAnoDiff [ 20 ]		SeaS [ 8 ]		Ours
Category	KID $\downarrow$	IC-L $\uparrow$	KID $\downarrow$	IC-L $\uparrow$	KID $\downarrow$	IC-L $\uparrow$	KID $\downarrow$	IC-L $\uparrow$
capsules	103.91	0.56	126.19	0.64	111.32	0.61	20.53	0.54
candle	183.21	0.17	180.22	0.38	47.65	0.10	41.91	0.12
cashew	78.22	0.38	52.46	0.48	10.37	0.35	10.29	0.35
chewinggum	39.63	0.34	29.76	0.43	24.19	0.29	33.36	0.33
fryum	238.41	0.27	102.00	0.41	82.20	0.24	24.93	0.18

Appendix

Table 10: Comparison of different methods on VisA. Bold and underlined text indicate the best and second-best results, respectively.

Category	AnomalyAny				Ours
Category	I-AUC	I-F1	P-AUC	P-F1	I-AUC	I-F1	P-AUC	P-F1
candle	95.6	90.0	99.3	40.1	94.9	86.1	98.8	42.4
capsules	96.2	93.8	99.1	60.1	89.6	83.2	99.2	68.7
cashew	97.4	94.5	99.2	70.4	94.0	91.7	99.9	92.3
chewinggum	98.7	97.0	99.5	75.3	98.5	94.8	99.7	78.1
fryum	98.4	97.5	97.4	53.6	94.4	91.5	98.6	70.8

Appendix

Table 11: Comparison to training-free AnomalyAny on VisA.

Figure 14: Limitations of Self-Attention for Tiny Anomaly Generation. The middle columns show the three leading principal components of the self-attention matrix for the real reference anomaly on the left at the 30th denoising step, while the right columns present our synthesized anomalous images and the corresponding anomaly masks. For small anomalous regions, the anomaly is almost invisible in the self-attention feature maps spanned by the top three PCA components, which makes it challenging to synthesize anomalies within small masks.

Method	AUROC-I	AP-I	F1-I	AUROC-P	AP-P	F1-P	Pro
DualAnoDiff	72.5	77.8	75.8	91.3	25.7	31.9	75.7
SeaS	76.7	80.4	77.2	87.7	31.1	36.5	76.9
Ours	79.3	84.0	80.4	93.6	37.3	41.9	82.9

Appendix

Table 12: One-reference AD results on Real-IAD (trained U-Net).

Methods	Training Time (single GPU)	Inference Time (per image)	GPU Memory
DualAnoDiff	119 hours	23 s	12G
SeaS	160 hours	11 s	24G
O2MAG Ours	0 hours	28 s	32G

Appendix

Table 13: Comparison of runtime and memory cost.

Category	AnomalyDiffusion			DualAnoDiff			SeaS			Ours
Category	AUC-I	AP-I	F1-I	AUC-I	AP-I	F1-I	AUC-I	AP-I	F1-I	AUC-I	AP-I	F1-I
candle	95.1	95.3	90.1	96.7	95.6	90.4	78.7	77.9	68.5	94.9	93.3	86.1
capsules	91.9	93.8	85.9	91.0	93.0	85.7	84.7	89.0	78.6	89.6	91.9	83.2
cashew	95.2	96.0	92.2	99.2	99.4	96.4	94.8	95.2	92.4	94.0	94.6	91.7
chewinggum	94.4	97.2	93.1	99.6	99.7	97.8	98.9	99.3	95.0	98.5	99.0	94.8
fryum	81.3	87.2	79.2	86.7	90.7	84.1	91.2	94.4	86.7	94.4	96.7	91.5

Appendix

Table 14: Comparison on a trained U-Net segmentation model for image-level anomaly detection and localization on VisA dataset.

Category	AnomalyDiffusion			DualAnoDiff			SeaS			Ours
Category	AUC-P	AP-P	F1-P	AUC-P	AP-P	F1-P	AUC-P	AP-P	F1-P	AUC-P	AP-P	F1-P
candle	99.2	24.7	33.0	98.9	48.8	49.7	96.5	29.9	37.5	98.8	39.8	42.4
capsules	99.6	60.2	61.2	98.4	60.7	63.8	99.5	59.1	62.3	99.2	70.2	68.7
cashew	98.4	68.8	63.9	99.9	98.0	94.5	99.9	98.2	95.3	99.9	96.4	92.3
chewinggum	98.7	78.3	72.4	99.9	86.6	78.2	99.5	84.3	77.6	99.7	85.5	78.1
fryum	91.0	28.2	31.8	97.9	76.7	74.7	97.4	74.5	69.5	98.6	80.6	70.8

Appendix

Table 15: Comparison on a trained U-Net segmentation model for pixel-level anomaly detection and localization on VisA dataset.

Category	DFMGAN			AnomalyDiffusion			DualAnoDiff			SeaS			Ours
Category	AUC-I	AP-I	F1-I	AUC-I	AP-I	F1-I	AUC-I	AP-I	F1-I	AUC-I	AP-I	F1-I	AUC-I	AP-I	F1-I
bottle	99.3	99.8	97.7	99.8	99.9	98.9	100	100	100	99.9	99.9	98.9	100	100	100
cable	95.9	97.8	93.8	100	100	100	99.8	99.8	98.5	98.0	98.8	96.1	100	100	100
capsule	92.8	98.5	94.5	99.7	99.9	98.7	96.3	99.2	94.7	97.1	99.2	95.4	99.3	99.8	98.0
carpet	67.9	87.9	87.3	96.7	98.8	94.3	98.6	99.9	96.7	97.4	99.0	96.7	100	100	100
grid	73.0	90.4	85.4	98.4	99.5	98.7	100	99.7	100	99.9	99.9	98.8	100	100	100

Appendix

Table 16: Comparison on a trained U-Net segmentation model for image-level anomaly detection and localization on MVTec-AD dataset.

Category	DFMGAN			AnomalyDiffusion			DualAnoDiff			SeaS			Ours
Category	AUC-P	AP-P	F1-P	AUC-P	AP-P	F1-P	AUC-P	AP-P	F1-P	AUC-P	AP-P	F1-P	AUC-P	AP-P	F1-P
bottle	98.9	90.2	83.9	99.4	94.1	87.3	99.5	93.4	85.7	99.7	95.9	88.8	99.7	95.4	88.4
cable	97.2	81.0	75.4	99.2	90.8	83.5	98.5	82.6	76.9	96.0	83.1	77.7	99.4	91.2	85.0
capsule	79.2	26.0	35.0	98.8	57.2	59.8	99.5	73.2	67.0	93.7	41.9	47.3	97.0	60.6	59.0
carpet	90.6	33.4	38.1	98.6	81.2	74.6	99.4	89.1	80.2	99.3	86.4	78.1	99.5	88.5	80.0
grid	75.2	14.3	20.5	98.3	52.9	54.6	98.5	57.2	54.9	99.7	76.3	70.0	99.6	78.6	71.6

Appendix

Table 17: Comparison on a trained U-Net segmentation model for pixel-level anomaly detection and localization on MVTec-AD dataset.

Figure 15: Generalization to real-world scenes.

Figure 17: bottle qualitative results on MVTec-AD.

Figure 18: cable qualitative results on MVTec-AD.

Figure 19: capsule qualitative results on MVTec-AD.

Figure 20: carpet qualitative results on MVTec-AD.

Figure 21: AnomalyDiffusion fails to synthesize the intended defects within the anomaly mask; DualAnoDiff and SeaS do not preserve background appearance and produce anomalies with distribution shift; AnomalyAny struggles to capture precise anomaly semantics. In contrast, our method preserves background fidelity and synthesizes diverse, realistic anomalies.

Figure 22: hazelnut qualitative results on MVTec-AD.

Figure 23: leather qualitative results on MVTec-AD.

Figure 24: metal_nut qualitative results on MVTec-AD.

Figure 25: pill qualitative results on MVTec-AD.

Figure 26: screw qualitative results on MVTec-AD.

Figure 27: tile qualitative results on MVTec-AD.

Figure 28: toothbrush qualitative results on MVTec-AD.

Figure 29: transistor qualitative results on MVTec-AD.

Figure 30: wood qualitative results on MVTec-AD.

Figure 31: zipper qualitative results on MVTec-AD.

Explore similar work

AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learning
Apr 9, 2026Jiaming Su, Tengchao Yang, Ruikang Zhang +3Synthetic Anomaly Generation Tool-Using Agents
Shanghai Jiao Tong University Shanghai, China · Tongji University Shanghai, China · Fudan University Shanghai, China
GroundingAnomaly: Spatially-Grounded Diffusion for Few-Shot Anomaly Synthesis
Apr 9, 2026Yishen Liu, Hongcang Chen, Pengcheng Zhao +8Synthetic Anomaly Generation Diffusion-Based Image Generation
Beijing Institute of Technology · Beijing Jiaotong University · Li Auto +1
CORE: Contrast-and-Reflect for Self-Supervised Prompt Optimization in Anomaly Synthesis
Oct 14, 2026Haoxiang Rao, Fang Zhao, Zhao Wang +3Synthetic Anomaly Generation Synthetic Data Augmentation
School of Intelligence Science and Technology, Nanjing University · China Mobile Zijin Innovation Institute

Apr 9, 2026J/K move · Enter open · S save

AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learning

Jiaming Su, Tengchao Yang, Ruikang Zhang, Zhengan Yan+2

Shanghai Jiao Tong University Shanghai, China · Tongji University Shanghai, China · Fudan University Shanghai, China

Synthetic Anomaly Generation Tool-Using Agents Industrial Anomaly Detection RL for Generative Models

Open paper AskPDF arXiv

Industrial anomaly generation is a crucial method for alleviating the data scarcity problem in anomaly detection tasks. Most existing anomaly synthesis methods rely on single-step generation mechanisms, lacking complex reasoning and iterative optimization capabilities, making it difficult to generate anomaly samples with high semantic realism. We propose AnomalyAgent, an anomaly synthesis agent with self-reflection, knowledge retrieval, and iterative refinement capabilities, aiming to generate realistic and diverse anomalies. Specifically, AnomalyAgent is equipped with five tools: Prompt Generation (PG), Image Generation (IG), Quality Evaluation (QE), Knowledge Retrieval (KR), and Mask Generation (MG), enabling closed-loop optimization. To improve decision-making and self-reflection, we construct structured trajectories from real anomaly images and design a two-stage training framework: supervised fine-tuning followed by reinforcement learning. This process is driven by a three-part reward mechanism: (1) task rewards to supervise the quality and location rationality of generated anomalies; (2) reflection rewards to train the model's ability to improve anomaly synthesis prompt; (3) behavioral rewards to ensure adherence to the trajectory. On the MVTec-AD dataset, AnomalyAgent achieves IS/IC-L of 2.10/0.33 for anomaly generation, 57.0% classification accuracy using ResNet34, and 99.3%/74.2% AP at the image/pixel level using a simple UNet, surpassing all zero-shot SOTA methods. The code is publicly available at https://github.com/sssjmmm/AnomalyAgent.

Abstract

Explore similar work

AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learning

GroundingAnomaly: Spatially-Grounded Diffusion for Few-Shot Anomaly Synthesis

CORE: Contrast-and-Reflect for Self-Supervised Prompt Optimization in Anomaly Synthesis