Spatiotemporal Dynamics-Aware Industrial Anomaly Detection and Reasoning

S-Wei Chen; Siangyu Yang; Yi-Chen Jan; Chun-Yi Lee; Shao-Yuan Lo

cs.CVNov 5, 2026

Spatiotemporal Dynamics-Aware Industrial Anomaly Detection and Reasoning

Authors: S-Wei Chen, Siangyu Yang, Yi-Chen Jan, Chun-Yi Lee, Shao-Yuan Lo

Organizations: National Taiwan University

Abstract

Industrial Anomaly Detection (IAD) is a long-established problem that aims to detect defects in industrial products during manufacturing for quality control. Most existing studies focus on static visual patterns as the basis for detection. However, in real manufacturing environments, many anomalies are dynamics-based, such as irregular movements that cannot be perceived from static cues alone. We observe that even recent Multimodal Large Language Model (MLLM)-based approaches fall short in identifying such dynamic anomalies due to limited temporal modeling capability. To address this, we introduce MUSTARD: MUltimodal SpatioTemporal Anomaly Reasoning and Detection, a framework that equips MLLMs with fine-grained spatiotemporal dynamics awareness for detecting and reasoning about dynamics-based defects. MUSTARD introduces optical-flow-guided keyframe selection and augmented modalities to capture critical motion irregularities through local motion patterns and global dynamic evolution. Its normal pattern-based reasoning pipeline requires only a few normal reference samples as the detection basis, avoiding any model updates. This enables efficient adaptation across various products and provides high decision interpretability. MUSTARD is the first MLLM-based framework for dynamics-based IAD. Experiments demonstrate its superior performance in detecting spatiotemporal anomalies and providing plausible explanations. The code will be made publicly available.

Figures & tables

Figure 1: Comparison between the direct use of an MLLM and the proposed method on an anomalous test video (a degaussing magnet). (1) Directly Use MLLM: fails to recognize that the magnet falls off the metal surface. (2) Our Method: leverages spatiotemporal information to correctly interpret the magnet’s drop from the surface.

Figure 2: An overview of the proposed MUSTARD framework. It consists of three main stages. (1) Spatiotemporal Dynamics Awareness: Leverages optical flow to select keyframes and derive three complementary descriptive modalities, including raw frames for visual context, trajectory frames for visualizing local motion, and a flow magnitude curve for representing global temporal dynamics. (2) Normal Pattern Extraction: Extracts multi-faceted normal patterns from a small set of reference videos. (3) Pattern Reasoning (Inference): Compares the test video’s description with the extracted normal patterns to detect deviations, producing a final anomaly judgment and explanation.

Figure 3: Illustration of the three derived modalities: raw frames, trajectory frames, and the flow magnitude curve. Different sources excel in different categories of test cases. Integrating them enables the model to leverage complementary strengths across modalities.

Category	Unsupervised / Weakly-supervised				General-purpose MLLMs				Specialized VAD			Ours
Category	MNAD.p	MNAD.r	VADClip	S3R	V-Chat	V-Llama	V-Llava	Qwen2.5	LAVAD	ARuler	O-VAD	MUSTARD
Car	49.2	94.4	58.1	60.6	50.0	67.8	52.2	50.0	55.7	40.7	57.5	56.5
Fan	81.0	54.2	62.4	64.0	54.9	59.2	61.1	50.0	51.0	49.4	66.9	47.4
Rolling Bearing	35.2	80.0	58.9	60.1	30.0	50.0	50.0	50.0	53.2	40.0	65.0	94.2
Spherical Bearing	96.2	81.3	50.0	68.2	45.0	55.0	50.0	50.0	43.5	62.0	64.1	76.4
Servo	97.5	87.8	51.8	59.2	50.6	68.3	46.4	50.0	50.2	49.3	49.5	69.9

Table 1: Video-level AUROC (%) across 22 Phys-AD categories. Scores are scaled by 100, with category-wise best results in bold . Average denotes the unweighted mean across categories.

Method	AUROC $\uparrow$
Video-LLaMA [ 39 ]	45.5
Qwen2.5-VL-7B [ 1 ]	47.2
Video-ChatGPT [ 20 ]	47.9
Video-LLaVA [ 15 ]	50.0
O-VAD [ 35 ]	56.5
Ours (MUSTARD)	57.2

Table 2: Generalization Results on IPAD Dataset (AUROC %)

Method	PAEval $\uparrow$	Win Rate vs. Ours
Video-LLaMA [ 39 ]	30.3	6.81
Video-LLaVA [ 15 ]	28.2	9.28
Qwen2.5-VL-7B [ 1 ]	22.3	22.63
AnomalyRuler [ 32 ]	26.3	34.65
Ours (MUSTARD)	45.6	–

Table 3: Anomaly Explanation Quality and Pairwise Win Rates versus Ours (%). A lower win rate indicates that the baseline is less competitive than our method. All baselines achieve win rates below 50%, demonstrating the superiority of our method.

Figure 4: Example outputs from three methods: direct MLLM usage, a single-source static pattern-reasoning pipeline, and our proposed MUSTARD method.

Raw Frames	Traj. Frames	Flow Mag.	AUROC $\uparrow$	PAEval $\uparrow$
✓			53.4	15.79
	✓		57.0	11.20
✓		✓	64.7	45.88
	✓	✓	67.1	44.63
✓	✓	✓	68.1	45.62

Table 4: Ablation Study of Spatiotemporal Modalities on Detection and Reasoning. Scores are scaled by 100 for readability.

Normal representation	Rules	Pattern (MUSTARD)
AUC	0.527	0.681

Table 5: AUROC on different normal representation Strategies

Stage	Model	GB
Raw-frame Desc.	Qwen2.5-VL-7B	7.7
Flow-mag Desc.	Qwen2.5-VL-7B	8.0
Reasoning + Fusion	Gemma3-12B	14.7
Peak	–	14.7

Table 6: GPU memory footprint of the major MUSTARD inference stages.

Panel of Figure 5 — Figure 5: Computational efficiency on the 220-video subset.

Panel of Figure 6 — Figure 6: An example of three different data sources for the Rolling Bearing object.

Figure 7: An example of three different data sources for the Lock object.

Figure 8: An example of three different data sources for the Clip object.

Figure 9: An example of the trajectory-frame normal pattern and corresponding reasoning output.

Figure 10: An example of the flow-magnitude-curve normal pattern and corresponding reasoning output.

Figure 11: Prompts of each source fed to Qwen2.5-VL-7B to generate frame descriptions.

Figure 12: Prompts fed to Gemma3-12B to capture normal patterns of trajectory-frame descriptions and flow-magnitude curves.

Figure 13: Prompts fed to Gemma3-12B to compare frame descriptions with the normal patterns.

Figure 14: Prompt provided to Gemma3-12B for fusing reasoning from trajectory frames, the flow-magnitude curve, and raw-frame descriptions.

Industrial Anomaly Detection

Industrial anomaly detection finds defective products and abnormal equipment behavior from visual, sensor, audio, and other production data.

See all 269 papers

Feb 10, 2026

Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning

Industrial anomaly detection demands precise reasoning over fine-grained defect patterns. However, existing multimodal large language models (MLLMs), pretrained on general-domain data, often struggle to capture category-specific anomalies, thereby limiting both detection accuracy and interpretability. To address these limitations, we propose Reason-IAD, a knowledge-guided dynamic latent reasoning framework for explainable industrial anomaly detection. Reason-IAD comprises two core components. First, a retrieval-augmented knowledge module incorporates category-specific textual descriptions into the model input, enabling context-aware reasoning over domain-specific defects. Second, an entropy-driven latent reasoning mechanism conducts iterative exploration within a compact latent space using optimizable latent think tokens, guided by an entropy-based reward that encourages confident and stable predictions. Furthermore, a dynamic visual injection strategy selectively incorporates the most informative image patches into the latent sequence, directing the reasoning process toward regions critical for anomaly detection. Extensive experimental results demonstrate that Reason-IAD consistently outperforms state-of-the-art methods across multiple tasks. The code will be publicly available at https://github.com/chenpeng052/Reason-IAD.

Peng Chen, Wenqiang Wang, Yunkang Cao +7

School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, China · Department of Computer and Information Science, University of Macau, Macau, China

Industrial Anomaly Detection Multimodal Anomaly Detection Interpretable Anomaly Detection Latent Reasoning

Ask PDF

Sep 24, 2026

Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space

Industrial anomaly detection (IAD) is evolving beyond conventional detection and localization toward multimodal inspection systems that can describe, explain, and reason about fine-grained defects. Although recent multimodal large language model (MLLM)-based methods improve anomaly understanding through textual reasoning and visual guidance, they face two limitations in fine-grained inspection. First, their visual refinement often requires iteratively revisiting local image regions or augmenting with additional tools. Second, the resulting local defect evidence may not be reliably preserved throughout subsequent reasoning. To address these, we propose Anomaly-LR, a defect-grounded latent reasoning framework that first forms a global understanding of the input and then progressively refines anomaly-relevant representations directly in the visual latent space. We further construct IAD-LR-22K, the first IAD instruction dataset designed for latent reasoning, containing 22,228 image-question instances from 4,523 industrial images, with global textual reasoning traces and region-level visual annotations. Extensive experiments show that Anomaly-LR achieves state-of-the-art performance among comparable-scale methods across multiple IAD benchmarks, without requiring external references or tools. The code and data will be released at https://github.com/Yen666/Anomaly-LR.

Jaron Yeh, Yen-Wei Chang, Jiang Liu +1

National Taiwan University · AMD GenAI

Industrial Anomaly Detection Anomaly Localization Latent Visual Reasoning Multimodal Anomaly Detection

Ask PDF

Jul 20, 2026

O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning

Industrial Video Anomaly Detection (IVAD) aims to identify anomalous objects and events in an industrial process, which is crucial for modern manufacturing and quality control systems. Existing VLM-based anomaly reasoning methods are capable of detecting open-ended anomalies in general domains. However, their performance declines in industrial settings characterized by intricate object transformations, strict physics, and procedural constraints. To tackle the complexity of such interaction-intensive detection, we introduce a training-free agentic framework for anomaly detection free of domain-specific knowledge, emphasizing object state evolution like humans inspectors. It is designed to track spatial-temporal dynamics and underlying transformations of detected objects over time, and then reason over the object-wise temporal state trajectories to identify abnormal objects in grounded frames. Our method overcomes limitations of prior approaches that rely on retraining on normal clips or injecting domain knowledge as context for test-time inference. Extensive experiments on three IVAD datasets demonstrate that our method outperforms frontier VLMs, agentic frameworks, and traditional VAD methods fine-tuned on the respective datasets, while providing interpretable reports over anomaly processes and types.

Mei Yuan, Qi Long, Qifeng Wu +5

Carnegie Mellon University · University of Alabama at Birmingham · Griffith University

Industrial Anomaly Detection Anomaly Localization Video Anomaly Detection

Ask PDF

Abstract

Industrial Anomaly Detection

Explore similar work

Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning

Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space

O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning