ATSS: Detecting AI-Generated Videos via Anomalous Temporal Self-Similarity
Authors: Hang Wang, Chao Shen, Lei Zhang, Zhi-Qi Cheng
Organizations: Xi’an Jiaotong University, Xi’an, China · The Hong Kong Polytechnic University, Hong Kong, China · School of Engineering and Technology, University of Washington, Tacoma, WA 98402, USA
AI-generated videos (AIGVs) have achieved unprecedented photorealism, posing severe threats to digital forensics. Existing AIGV detectors focus mainly on localized artifacts or short-term temporal inconsistencies, thus often fail to capture the underlying generative logic governing global temporal evolution, limiting AIGV detection performance. In this paper, we identify a distinctive fingerprint in AIGVs, termed anomalous temporal self-similarity (ATSS). Unlike real videos that exhibit stochastic natural dynamics, AIGVs follow deterministic anchor-driven trajectories (e.g., text or image prompts), inducing unnaturally repetitive correlations across visual and semantic domains. To exploit this, we propose the ATSS method, a multimodal detection framework that exploits this insight via a triple-similarity representation and a cross-attentive fusion mechanism. Specifically, ATSS reconstructs semantic trajectories by leveraging frame-wise descriptions to construct visual, textual, and cross-modal similarity matrices, which jointly quantify the inherent temporal anomalies. These matrices are encoded by dedicated Transformer encoders and integrated via a bidirectional cross-attentive fusion module to effectively model intra- and inter-modal dynamics. Extensive experiments on four large-scale benchmarks, including GenVideo, EvalCrafter, VideoPhy, and VidProM, demonstrate that ATSS significantly outperforms state-of-the-art methods in terms of AP, AUC, and ACC metrics, exhibiting superior generalization across diverse video generation models. Code and models of ATSS will be released at https://github.com/hwang-cs-ime/ATSS.
Figures & tables
Fig. 1 : Motivation of the proposed ATSS framework. Real videos are characterized by stochastic spatiotemporal dynamics, resulting in diffuse and low-magnitude self-similarity patterns. In contrast, AI-generated videos exhibit systemic temporal regularity caused by their anchor-driven generation, which yields denser and higher-intensity correlation matrices across multiple modalities. This anomalous self-similarity serves as a distinctive forensic fingerprint, enabling ATSS to effectively distinguish AIGVs from natural sequences.
Fig. 2 : The overall framework of ATSS. Given a video with T sampled frames, an image captioning model is first employed to generate frame-wise textual descriptions. Visual and textual features are then extracted to construct visual, textual, and cross-modal self-similarity matrices. Each matrix is processed by a dedicated Transformer to capture temporal dynamics. Finally, a cross-attentive fusion module integrates these modality-specific cues into a unified representation for binary classification.
Method
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
LaVie
Sora
WS
mean
STIL † [ 22 ] , ACM MM 2021
88.21
88.68
73.07
56.91
62.07
83.96
64.87
65.54
49.86
63.60
69.68
FTCN † [ 23 ] , ICCV 2021
70.01
83.59
97.07
87.42
93.30
91.86
91.72
84.16
44.48
84.46
82.81
X-CLIP † [ 51 ] , ECCV 2022
79.84
87.54
95.53
90.71
94.54
88.69
93.50
86.28
64.23
88.54
89.62
TALL † [ 24 ] , ICCV 2023
51.11
63.63
92.09
44.00
51.06
93.47
87.85
59.07
15.82
64.43
62.25
FID † [ 52 ] , NeurIPS 2024
91.50
92.24
93.67
86.10
90.61
93.27
92.41
83.68
74.95
82.24
88.07
NPR † [ 53 ] , CVPR 2024
84.67
96.53
96.79
40.17
21.61
96.35
97.02
22.37
90.55
66.51
71.26
TABLE I : Comparison of detection AP ( % ) between ATSS and 13 state-of-the-art baselines on GenVideo. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
LaVie
Sora
WS
mean
STIL † [ 22 ] , ACM MM 2021
86.48
88.75
82.77
60.00
68.71
86.67
73.56
72.67
54.18
69.94
74.37
FTCN † [ 23 ] , ICCV 2021
69.76
83.25
97.18
88.69
93.45
93.33
91.62
84.70
38.30
85.74
82.60
X-CLIP † [ 51 ] , ECCV 2022
80.51
86.14
95.12
91.51
94.69
88.96
92.84
85.04
64.09
87.93
88.87
TALL † [ 24 ] , ICCV 2023
58.46
60.54
83.24
45.21
46.24
73.30
66.37
48.40
66.36
53.84
60.20
FID † [ 52 ] , NeurIPS 2024
90.94
91.93
93.70
85.77
91.19
93.16
92.02
82.57
73.45
81.30
87.60
NPR † [ 53 ] , CVPR 2024
93.92
99.38
99.92
26.83
18.99
98.78
99.24
42.32
97.56
76.12
75.31
TABLE II : Comparison of detection AUC ( % ) between ATSS and 13 state-of-the-art baselines on GenVideo. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
Floor33
Gen2
Gen2-D
HS-XL
LaVie-B
LaVie-I
Mix-SR
MS
MV
PKL
PKL-V1
Show-1
VC
ZS
mean
STIL †
90.42
90.59
68.07
59.36
65.96
66.02
65.41
89.20
74.71
95.86
79.97
60.21
67.14
89.73
75.90
FTCN †
82.85
88.50
94.72
88.25
82.71
81.91
92.74
71.16
96.65
96.36
93.14
93.55
92.23
69.09
87.42
X-CLIP †
87.32
87.56
90.72
89.67
85.58
83.55
94.22
79.45
96.04
97.44
96.15
95.68
91.42
77.63
90.28
TALL †
63.25
70.75
77.04
46.93
52.87
52.53
78.16
62.11
83.63
65.33
70.98
48.00
60.50
51.73
63.13
FID †
96.40
97.36
98.68
89.90
92.92
84.19
98.51
95.74
98.29
99.49
99.17
96.77
95.71
95.18
95.59
NPR †
99.77
99.34
99.95
47.39
76.45
72.23
99.67
98.54
99.96
99.97
99.93
69.82
99.68
98.21
90.07
TABLE III : Comparison of detection AP ( % ) between ATSS and 13 state-of-the-art baselines on EvalCrafter. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
Floor33
Gen2
Gen2-D
HS-XL
LaVie-B
LaVie-I
Mix-SR
MS
MV
PKL
PKL-V1
Show-1
VC
ZS
mean
STIL †
90.14
91.40
79.25
61.62
73.63
71.23
74.22
87.28
84.22
97.82
88.50
66.68
74.49
90.77
80.80
FTCN †
82.78
90.76
95.11
90.29
84.51
83.00
92.31
70.99
96.65
96.73
94.01
93.62
91.27
70.82
88.06
X-CLIP †
86.10
86.70
90.93
90.80
84.06
83.01
93.59
79.69
95.56
97.45
95.79
95.29
90.71
80.53
89.38
TALL †
61.46
72.04
74.60
43.88
49.64
50.02
77.50
58.99
83.19
67.13
72.51
46.81
55.69
46.03
61.39
FID †
96.26
97.46
98.58
90.05
92.32
83.42
98.23
95.27
98.28
99.48
99.24
96.79
95.26
95.29
95.42
NPR †
99.38
97.74
99.80
26.83
50.23
36.00
99.22
93.92
99.92
99.92
99.71
18.99
99.25
92.60
79.54
TABLE IV : Comparison of detection AUC ( % ) between ATSS and 13 state-of-the-art baselines on EvalCrafter. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
CVX
CVX-5B
DM
Gen-2
LaVie
OpenSora
Pika
SVD-T2I2V
VC2
ZS
mean
STIL † [ 22 ] , ACM MM 2021
66.43
69.61
65.31
69.63
63.68
61.94
97.07
67.51
67.54
58.56
68.73
FTCN † [ 23 ] , ICCV 2021
74.24
74.83
67.98
93.99
69.16
66.95
93.94
82.81
83.39
75.64
78.29
X-CLIP † [ 51 ] , ECCV 2022
87.35
83.72
75.54
92.09
79.52
90.44
96.13
85.26
85.35
88.88
86.66
TALL † [ 24 ] , ICCV 2023
39.59
50.72
62.36
70.78
40.40
37.30
62.69
52.62
52.66
50.66
51.98
FID † [ 52 ] , NeurIPS 2024
93.34
91.41
97.50
98.35
96.51
87.90
99.55
95.66
96.03
90.60
94.69
NPR † [ 53 ] , CVPR 2024
81.37
81.99
99.86
99.90
63.72
88.78
99.91
99.54
60.21
78.23
85.35
TABLE V : Comparison of detection AP ( % ) between ATSS and 13 state-of-the-art baselines on VideoPhy. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
CVX
CVX-5B
DM
Gen-2
LaVie
OpenSora
Pika
SVD-T2I2V
VC2
ZS
mean
STIL † [ 22 ] , ACM MM 2021
71.73
73.94
73.01
80.75
68.66
63.42
98.47
77.82
75.18
55.85
73.88
FTCN † [ 23 ] , ICCV 2021
80.91
81.01
70.20
95.21
72.88
76.37
95.90
87.07
86.53
74.88
82.10
X-CLIP † [ 51 ] , ECCV 2022
88.99
84.84
78.57
93.35
76.51
92.68
96.22
84.92
86.27
88.06
86.88
TALL † [ 24 ] , ICCV 2023
30.05
46.36
56.54
64.80
32.94
24.82
63.65
45.40
42.48
39.44
44.65
FID † [ 52 ] , NeurIPS 2024
93.37
91.72
97.62
98.11
96.03
86.03
99.59
95.81
95.68
89.38
94.33
NPR † [ 53 ] , CVPR 2024
72.10
73.60
99.70
99.80
42.90
83.50
99.80
99.50
47.20
52.90
77.10
TABLE VI : Comparison of detection AUC ( % ) between ATSS and 13 state-of-the-art baselines on VideoPhy. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
MS
OpenSora
Pika
ST2V
T2VZ
VC2
mean
STIL † [ 22 ] , ACM MM 2021
43.68
63.15
95.28
55.46
48.68
61.49
61.29
FTCN † [ 23 ] , ICCV 2021
74.65
90.05
94.26
83.53
46.25
87.27
79.33
X-CLIP † [ 51 ] , ECCV 2022
82.24
93.47
95.80
86.42
70.92
91.60
88.05
TALL † [ 24 ] , ICCV 2023
50.93
54.50
63.47
51.50
60.99
59.70
56.85
FID † [ 52 ] , NeurIPS 2024
91.35
87.68
99.59
97.87
68.51
85.92
88.49
NPR † [ 53 ] , CVPR 2024
87.04
89.85
99.98
89.88
88.93
70.79
87.75
TABLE VII : Comparison of detection AP ( % ) between ATSS and 13 state-of-the-art baselines on VidProM. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
MS
OpenSora
Pika
ST2V
T2VZ
VC2
mean
STIL † [ 22 ] , ACM MM 2021
41.05
72.01
97.42
67.12
47.66
67.98
65.54
FTCN † [ 23 ] , ICCV 2021
70.98
90.76
95.23
87.81
50.70
88.43
80.65
X-CLIP † [ 51 ] , ECCV 2022
80.59
93.02
95.85
87.47
75.86
91.24
87.34
TALL † [ 24 ] , ICCV 2023
45.29
56.39
66.91
50.31
57.80
53.48
55.03
FID † [ 52 ] , NeurIPS 2024
89.86
87.36
99.62
98.10
67.01
85.65
87.93
NPR † [ 53 ] , CVPR 2024
82.61
98.56
99.84
98.92
93.32
56.70
88.33
TABLE VIII : Comparison of detection AUC ( % ) between ATSS and 13 state-of-the-art baselines on VidProM. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Method
GenVideo
EvalCrafter
VideoPhy
VidProM
STIL † [ 22 ] , ACM MM 2021
59.90
66.78
56.62
56.58
FTCN † [ 23 ] , ICCV 2021
70.52
73.87
62.43
67.23
X-CLIP † [ 51 ] , ECCV 2022
75.34
76.82
69.26
73.37
TALL † [ 24 ] , ICCV 2023
57.47
58.42
48.78
54.02
FID † [ 52 ] , NeurIPS 2024
54.57
63.59
65.01
54.44
NPR † [ 53 ] , CVPR 2024
65.41
71.36
57.00
68.04
TABLE IX : Comparison of average detection ACC (%) of ATSS and 13 state-of-the-art baselines on the GenVideo, EvalCrafter, VideoPhy, and VidProM datasets. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.
Visual
Textual
Cross-Modal
Cross-Attentive
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
LaVie
Sora
WS
mean
✓
×
×
×
45.39
44.62
43.70
45.61
43.76
43.12
44.18
44.24
47.47
43.84
44.59
×
✓
×
×
85.69
90.30
94.54
94.45
94.99
96.02
95.50
93.81
72.59
88.52
90.64
×
×
✓
×
87.17
89.89
95.34
95.01
93.12
95.85
95.98
93.49
86.05
88.58
92.05
×
✓
✓
✓
83.85
89.67
88.67
94.21
93.64
93.15
94.38
89.18
82.08
88.23
89.71
✓
×
✓
✓
92.55
97.27
99.08
98.82
99.36
99.38
99.55
96.02
86.03
97.49
96.55
✓
✓
×
✓
90.07
95.40
97.29
96.04
97.22
97.09
98.42
95.26
81.77
93.11
94.17
TABLE X : Ablation performance of individual components in ATSS on GenVideo, reported in terms of AP (%).
Visual
Textual
Cross-Modal
Cross-Attentive
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
LaVie
Sora
WS
mean
✓
×
×
×
50.75
49.90
47.57
52.03
47.87
45.75
48.48
48.88
53.44
48.10
49.28
×
✓
×
×
84.56
89.66
93.81
93.74
94.45
95.73
94.90
93.04
72.40
87.06
89.93
×
×
✓
×
87.14
91.24
95.32
95.09
94.45
95.68
95.97
93.51
84.08
88.11
92.06
×
✓
✓
✓
85.22
91.53
93.47
95.73
96.26
96.03
96.42
92.39
84.24
89.14
92.04
✓
×
✓
✓
90.06
96.84
98.89
98.47
99.26
99.25
99.44
95.02
85.85
96.54
95.96
✓
✓
×
✓
90.04
95.39
96.93
96.73
96.93
97.37
98.21
95.34
84.28
93.21
94.44
TABLE XI : Ablation performance of individual components in ATSS on GenVideo, reported in terms of AUC (%).
Visual
Textual
Cross-Modal
Cross-Attentive
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
LaVie
Sora
WS
mean
✓
×
×
×
63.02
66.27
68.26
69.84
66.03
65.06
69.38
66.07
62.75
64.75
66.14
×
✓
×
×
76.51
81.90
85.90
86.75
86.51
88.41
87.49
84.92
66.67
77.76
82.28
×
×
✓
×
78.97
84.05
87.41
87.38
87.38
88.12
87.81
86.11
71.57
80.11
83.89
×
✓
✓
✓
78.41
84.29
86.88
89.21
90.00
89.90
89.04
86.07
79.41
82.16
85.54
✓
×
✓
✓
82.70
88.33
92.46
91.59
92.14
93.48
92.02
87.22
78.43
89.47
88.78
✓
✓
×
✓
83.10
89.13
87.68
89.21
88.25
89.33
90.23
87.70
76.47
85.44
86.65
TABLE XII : Ablation performance of individual components in ATSS on GenVideo, reported in terms of ACC (%).
Caption Model
Visual & Textual Encoder
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
Lavie
Sora
WS
mean
BLIP-base
CLIP-P16
92.25
97.25
98.69
98.56
99.32
99.18
99.56
95.49
84.90
96.00
96.12
CLIP-P32
94.00
97.70
99.02
98.69
99.25
99.22
99.63
97.31
87.80
97.07
96.97
XCLIP-P16
76.37
79.21
80.76
82.16
82.66
83.29
83.77
79.08
63.37
83.25
79.39
XCLIP-P32
91.92
96.68
99.09
98.23
98.72
99.35
99.15
97.94
72.66
95.57
94.93
BLIP-large
CLIP-P16
86.86
93.35
94.76
94.09
95.89
94.27
96.16
94.15
73.97
88.62
91.21
CLIP-P32
82.97
89.53
93.64
92.36
91.87
93.07
93.72
90.50
74.32
86.47
88.84
TABLE XIII : Ablation results of ATSS using various image captioning backbones and visual-textual encoders on the GenVideo dataset, measured by AP (%).
Caption Model
Visual & Textual Encoder
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
Lavie
Sora
WS
mean
BLIP-base
CLIP-P16
90.09
96.86
98.22
98.25
99.21
98.94
99.45
93.53
83.20
94.65
95.24
CLIP-P32
93.15
97.74
98.91
98.69
99.33
99.15
99.62
96.50
87.77
96.24
96.71
XCLIP-P16
83.08
86.75
89.55
89.61
90.10
90.14
91.73
86.04
71.32
88.43
86.67
XCLIP-P32
90.30
96.04
98.92
98.84
98.64
99.43
99.02
97.32
76.66
94.40
94.96
BLIP-large
CLIP-P16
85.25
91.50
93.74
93.42
95.28
93.02
95.20
93.33
69.24
86.43
89.64
CLIP-P32
85.06
91.01
95.46
93.72
93.86
94.54
95.47
91.14
78.66
88.51
90.74
TABLE XIV : Ablation results of ATSS using various image captioning backbones and visual-textual encoders on the GenVideo dataset, measured by AUC (%).
Caption Model
Visual & Textual Encoder
MS
MPS
MV
HotShot
Show-1
Gen2
Crafter
Lavie
Sora
WS
mean
BLIP-base
CLIP-P16
82.86
87.70
89.45
89.52
90.71
90.18
90.43
86.07
77.45
86.62
87.10
CLIP-P32
84.84
92.86
96.19
95.24
96.83
96.78
97.26
91.67
73.53
91.26
91.64
XCLIP-P16
74.52
76.03
76.33
75.63
75.16
76.41
77.24
74.72
64.71
76.58
74.73
XCLIP-P32
80.63
89.68
94.68
95.95
93.41
96.70
96.07
92.14
64.71
85.44
88.94
BLIP-large
CLIP-P16
76.27
80.40
81.83
81.75
84.13
81.44
83.76
83.10
68.63
77.63
79.89
CLIP-P32
76.03
81.59
89.18
86.19
86.35
87.44
88.24
83.45
71.57
79.12
82.92
TABLE XV : Ablation results of ATSS using various image captioning backbones and visual-textual encoders on the GenVideo dataset, measured by ACC (%).
Fig. 3 : t-SNE visualizations of 10 subsets on the GenVideo dataset: (a) Crafter, (b) Gen2, (c) HotShot, (d) Lavie, (e) ModelScope, (f) MoonValley, (g) MorphStudio, (h) Show-1, (i) Sora, and (j) WildScrape.
Fig. 4 : Visualization of Attention Density Maps. Each row displays the attention weights for the visual, textual, and cross-modal branches of a real video and ten AI-generated samples, which are randomly selected from MSR-VTT and each generator subset of GenVideo, respectively.
The proliferation of advanced AI video synthesis techniques poses an unprecedented challenge to digital video authenticity. Existing AI-generated video (AIGV) detection methods primarily focus on uni-modal or spatiotemporal artifacts, but they overlook the rich cues within the visual-textual cross-modal space, especially the temporal stability of semantic alignment. In this work, we identify a distinctive fingerprint in AIGVs, termed cross-modal temporal artifact (CMTA). Unlike real videos that exhibit natural temporal fluctuations in cross-modal alignment due to semantic variations, AIGVs display unnaturally stable semantic trajectories governed by given input prompts. To bridge this gap, we propose the CMTA framework, a cross-modal detection approach that captures these unique temporal artifacts through joint cross-modal embedding and multi-grained temporal modeling. Specifically, CMTA leverages BLIP to generate frame-level image captions and utilizes CLIP to extract corresponding visual-textual representations. A coarse-grained temporal modeling branch is then designed to characterize temporal fluctuations in cross-modal alignment with a GRU. In parallel, a fine-grained branch is constructed to capture intricate inter-frame variations from integrated visual-textual features with a Transformer encoder. Extensive experiments on 40 subsets across four large-scale datasets, including GenVideo, EvalCrafter, VideoPhy, and VidProM, validate that our approach sets a new state-of-the-art while exhibiting superior cross-generator generalization. Code and models of CMTA will be released at https://github.com/hwang-cs-ime/CMTA
Hang Wang, Chao Shen, Chenhao Lin +3
Xi’an Jiaotong University, Xi’an, China · The Hong Kong Polytechnic University, Hong Kong, China · Guangdong OPPO Mobile Communications Co., Ltd. +1
AI-generated videos are becoming increasingly realistic and difficult to distinguish from authentic ones, which facilitates malicious misuse and poses growing threats to cybersecurity and social governance. Attributing AI-generated videos to their specific generative sources is therefore of critical importance for forensic investigation and legal regulation. However, most existing visual attribution methods focus on images and particularly rely on the image generation model, thereby lacking the ability to generalize to large-scale AI-generated video data. To address these limitations, we introduce an training-free AI-generated video attribution paradigm. Specifically, we formulates AI-generated video attribution as an instance retrieval task, and design a generative fingerprint-based pipeline. This pipeline consists of an adapted orthogonal color transformation, multi-scale quantized residual generation, and temporal-semantic aggregation, progressively capturing and integrating artifacts introduced by generative models across video frames. Extensive experiments on the GenVidBench benchmark demonstrate that our method achieves strong performance in both AI-generated video detection and attribution, outperforming existing state-of-the-art methods with a Rank-1 accuracy of 20.5% and a mean Average Precision of 16.6%. The code is at https://github.com/renxi-seu/Video_Attribution.
Renxi Cheng, Chaolei Han, Jie Gui +1
School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China · Purple Mountain Laboratories, Nanjing 210000, China · Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, China +2
Recent advances in generative video models have enabled the synthesis of visually realistic content, posing significant challenges to synthetic video detection. Existing detectors often rely on appearance artifacts, semantic inconsistencies, and temporal patterns that may be generator-specific, limitating generalization to unseen synthesis models. We investigate whether responses to a pretrained generative model provide more transferable forensic cues. Our key observation is that real and AI-generated videos exhibit distinct velocity responses under a pretrained Flow Matching video model. This distinction persists when different pretrained video-generation backbones are used as probes, suggesting that velocity responses offer transferable forensic signals beyond visual artificts. Motivated by this observation, we propose TRACE (Trajectory Representation and Consistency Estimation), a generation-process-aware framework for AI-generated video detection. TRACE leverages a pretrained video DiT as a velocity-field probe to extract representations at multiple flow time points, and models cross-frame consistency through velocity differences between adjacent frames. We further introduce a Real-Centered Trajectory Optimization objective that encourages generator-invariant representation learning. Extensive experiments on AIGVDBench demonstrate that TRACE generalizes effectively across diverse generators, substantially outperforming prior state-of-the-art methods on unseen open- and closed-source video generation models.
Huangsen Cao, Hongkang chu, Siyao Yu +3
Zhejiang University · University of the Chinese Academy of Sciences · Nanjing University of Information Science and Technology +1