Spatiotemporal Dynamics-Aware Industrial Anomaly Detection and Reasoning

S-Wei Chen; Siangyu Yang; Yi-Chen Jan; Chun-Yi Lee; Shao-Yuan Lo

cs.CVNov 5, 2026

Spatiotemporal Dynamics-Aware Industrial Anomaly Detection and Reasoning

Authors: S-Wei Chen, Siangyu Yang, Yi-Chen Jan, Chun-Yi Lee, Shao-Yuan Lo

Organizations: National Taiwan University

Abstract

Industrial Anomaly Detection (IAD) is a long-established problem that aims to detect defects in industrial products during manufacturing for quality control. Most existing studies focus on static visual patterns as the basis for detection. However, in real manufacturing environments, many anomalies are dynamics-based, such as irregular movements that cannot be perceived from static cues alone. We observe that even recent Multimodal Large Language Model (MLLM)-based approaches fall short in identifying such dynamic anomalies due to limited temporal modeling capability. To address this, we introduce MUSTARD: MUltimodal SpatioTemporal Anomaly Reasoning and Detection, a framework that equips MLLMs with fine-grained spatiotemporal dynamics awareness for detecting and reasoning about dynamics-based defects. MUSTARD introduces optical-flow-guided keyframe selection and augmented modalities to capture critical motion irregularities through local motion patterns and global dynamic evolution. Its normal pattern-based reasoning pipeline requires only a few normal reference samples as the detection basis, avoiding any model updates. This enables efficient adaptation across various products and provides high decision interpretability. MUSTARD is the first MLLM-based framework for dynamics-based IAD. Experiments demonstrate its superior performance in detecting spatiotemporal anomalies and providing plausible explanations. The code will be made publicly available.

Figures & tables

Figure 1: Comparison between the direct use of an MLLM and the proposed method on an anomalous test video (a degaussing magnet). (1) Directly Use MLLM: fails to recognize that the magnet falls off the metal surface. (2) Our Method: leverages spatiotemporal information to correctly interpret the magnet’s drop from the surface.

Figure 2: An overview of the proposed MUSTARD framework. It consists of three main stages. (1) Spatiotemporal Dynamics Awareness: Leverages optical flow to select keyframes and derive three complementary descriptive modalities, including raw frames for visual context, trajectory frames for visualizing local motion, and a flow magnitude curve for representing global temporal dynamics. (2) Normal Pattern Extraction: Extracts multi-faceted normal patterns from a small set of reference videos. (3) Pattern Reasoning (Inference): Compares the test video’s description with the extracted normal patterns to detect deviations, producing a final anomaly judgment and explanation.

Figure 3: Illustration of the three derived modalities: raw frames, trajectory frames, and the flow magnitude curve. Different sources excel in different categories of test cases. Integrating them enables the model to leverage complementary strengths across modalities.

Category	Unsupervised / Weakly-supervised				General-purpose MLLMs				Specialized VAD			Ours
Category	MNAD.p	MNAD.r	VADClip	S3R	V-Chat	V-Llama	V-Llava	Qwen2.5	LAVAD	ARuler	O-VAD	MUSTARD
Car	49.2	94.4	58.1	60.6	50.0	67.8	52.2	50.0	55.7	40.7	57.5	56.5
Fan	81.0	54.2	62.4	64.0	54.9	59.2	61.1	50.0	51.0	49.4	66.9	47.4
Rolling Bearing	35.2	80.0	58.9	60.1	30.0	50.0	50.0	50.0	53.2	40.0	65.0	94.2
Spherical Bearing	96.2	81.3	50.0	68.2	45.0	55.0	50.0	50.0	43.5	62.0	64.1	76.4
Servo	97.5	87.8	51.8	59.2	50.6	68.3	46.4	50.0	50.2	49.3	49.5	69.9

Table 1: Video-level AUROC (%) across 22 Phys-AD categories. Scores are scaled by 100, with category-wise best results in bold . Average denotes the unweighted mean across categories.

Method	AUROC $\uparrow$
Video-LLaMA [ 39 ]	45.5
Qwen2.5-VL-7B [ 1 ]	47.2
Video-ChatGPT [ 20 ]	47.9
Video-LLaVA [ 15 ]	50.0
O-VAD [ 35 ]	56.5
Ours (MUSTARD)	57.2

Table 2: Generalization Results on IPAD Dataset (AUROC %)

Method	PAEval $\uparrow$	Win Rate vs. Ours
Video-LLaMA [ 39 ]	30.3	6.81
Video-LLaVA [ 15 ]	28.2	9.28
Qwen2.5-VL-7B [ 1 ]	22.3	22.63
AnomalyRuler [ 32 ]	26.3	34.65
Ours (MUSTARD)	45.6	–

Table 3: Anomaly Explanation Quality and Pairwise Win Rates versus Ours (%). A lower win rate indicates that the baseline is less competitive than our method. All baselines achieve win rates below 50%, demonstrating the superiority of our method.

Figure 4: Example outputs from three methods: direct MLLM usage, a single-source static pattern-reasoning pipeline, and our proposed MUSTARD method.

Raw Frames	Traj. Frames	Flow Mag.	AUROC $\uparrow$	PAEval $\uparrow$
✓			53.4	15.79
	✓		57.0	11.20
✓		✓	64.7	45.88
	✓	✓	67.1	44.63
✓	✓	✓	68.1	45.62

Table 4: Ablation Study of Spatiotemporal Modalities on Detection and Reasoning. Scores are scaled by 100 for readability.

Normal representation	Rules	Pattern (MUSTARD)
AUC	0.527	0.681

Table 5: AUROC on different normal representation Strategies

Stage	Model	GB
Raw-frame Desc.	Qwen2.5-VL-7B	7.7
Flow-mag Desc.	Qwen2.5-VL-7B	8.0
Reasoning + Fusion	Gemma3-12B	14.7
Peak	–	14.7

Table 6: GPU memory footprint of the major MUSTARD inference stages.

Panel of Figure 5 — Figure 5: Computational efficiency on the 220-video subset.

Panel of Figure 6 — Figure 6: An example of three different data sources for the Rolling Bearing object.

Figure 7: An example of three different data sources for the Lock object.

Figure 8: An example of three different data sources for the Clip object.

Figure 9: An example of the trajectory-frame normal pattern and corresponding reasoning output.

Figure 10: An example of the flow-magnitude-curve normal pattern and corresponding reasoning output.

Figure 11: Prompts of each source fed to Qwen2.5-VL-7B to generate frame descriptions.

Figure 12: Prompts fed to Gemma3-12B to capture normal patterns of trajectory-frame descriptions and flow-magnitude curves.

Figure 13: Prompts fed to Gemma3-12B to compare frame descriptions with the normal patterns.

Figure 14: Prompt provided to Gemma3-12B for fusing reasoning from trajectory frames, the flow-magnitude curve, and raw-frame descriptions.

Industrial Anomaly Detection

Industrial anomaly detection finds defective products and abnormal equipment behavior from visual, sensor, audio, and other production data.

See all 269 papers

Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning
Feb 10, 2026Peng Chen, Wenqiang Wang, Yunkang Cao +7Industrial Anomaly Detection Multimodal Anomaly Detection
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, China · Department of Computer and Information Science, University of Macau, Macau, China
Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space
Sep 24, 2026Jaron Yeh, Yen-Wei Chang, Jiang Liu +1Industrial Anomaly Detection Anomaly Localization
National Taiwan University · AMD GenAI
O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning
Jul 20, 2026Mei Yuan, Qi Long, Qifeng Wu +5Industrial Anomaly Detection Anomaly Localization
Carnegie Mellon University · University of Alabama at Birmingham · Griffith University

Feb 10, 2026J/K move · Enter open · S save

Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning

Peng Chen, Wenqiang Wang, Yunkang Cao, Chengliang Liu+6

School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, China · Department of Computer and Information Science, University of Macau, Macau, China

Industrial Anomaly Detection Multimodal Anomaly Detection Interpretable Anomaly Detection Latent Reasoning

Open paper AskPDF arXiv

Industrial anomaly detection demands precise reasoning over fine-grained defect patterns. However, existing multimodal large language models (MLLMs), pretrained on general-domain data, often struggle to capture category-specific anomalies, thereby limiting both detection accuracy and interpretability. To address these limitations, we propose Reason-IAD, a knowledge-guided dynamic latent reasoning framework for explainable industrial anomaly detection. Reason-IAD comprises two core components. First, a retrieval-augmented knowledge module incorporates category-specific textual descriptions into the model input, enabling context-aware reasoning over domain-specific defects. Second, an entropy-driven latent reasoning mechanism conducts iterative exploration within a compact latent space using optimizable latent think tokens, guided by an entropy-based reward that encourages confident and stable predictions. Furthermore, a dynamic visual injection strategy selectively incorporates the most informative image patches into the latent sequence, directing the reasoning process toward regions critical for anomaly detection. Extensive experimental results demonstrate that Reason-IAD consistently outperforms state-of-the-art methods across multiple tasks. The code will be publicly available at https://github.com/chenpeng052/Reason-IAD.

Abstract

Industrial Anomaly Detection

Explore similar work

Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning

Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space

O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning