ATSS: Detecting AI-Generated Videos via Anomalous Temporal Self-Similarity

Hang Wang; Chao Shen; Lei Zhang; Zhi-Qi Cheng

cs.CVApr 5, 2026

ATSS: Detecting AI-Generated Videos via Anomalous Temporal Self-Similarity

Authors: Hang Wang, Chao Shen, Lei Zhang, Zhi-Qi Cheng

Organizations: Xi’an Jiaotong University, Xi’an, China · The Hong Kong Polytechnic University, Hong Kong, China · School of Engineering and Technology, University of Washington, Tacoma, WA 98402, USA

Abstract

AI-generated videos (AIGVs) have achieved unprecedented photorealism, posing severe threats to digital forensics. Existing AIGV detectors focus mainly on localized artifacts or short-term temporal inconsistencies, thus often fail to capture the underlying generative logic governing global temporal evolution, limiting AIGV detection performance. In this paper, we identify a distinctive fingerprint in AIGVs, termed anomalous temporal self-similarity (ATSS). Unlike real videos that exhibit stochastic natural dynamics, AIGVs follow deterministic anchor-driven trajectories (e.g., text or image prompts), inducing unnaturally repetitive correlations across visual and semantic domains. To exploit this, we propose the ATSS method, a multimodal detection framework that exploits this insight via a triple-similarity representation and a cross-attentive fusion mechanism. Specifically, ATSS reconstructs semantic trajectories by leveraging frame-wise descriptions to construct visual, textual, and cross-modal similarity matrices, which jointly quantify the inherent temporal anomalies. These matrices are encoded by dedicated Transformer encoders and integrated via a bidirectional cross-attentive fusion module to effectively model intra- and inter-modal dynamics. Extensive experiments on four large-scale benchmarks, including GenVideo, EvalCrafter, VideoPhy, and VidProM, demonstrate that ATSS significantly outperforms state-of-the-art methods in terms of AP, AUC, and ACC metrics, exhibiting superior generalization across diverse video generation models. Code and models of ATSS will be released at https://github.com/hwang-cs-ime/ATSS.

Figures & tables

Fig. 1 : Motivation of the proposed ATSS framework. Real videos are characterized by stochastic spatiotemporal dynamics, resulting in diffuse and low-magnitude self-similarity patterns. In contrast, AI-generated videos exhibit systemic temporal regularity caused by their anchor-driven generation, which yields denser and higher-intensity correlation matrices across multiple modalities. This anomalous self-similarity serves as a distinctive forensic fingerprint, enabling ATSS to effectively distinguish AIGVs from natural sequences.

Fig. 2 : The overall framework of ATSS. Given a video with $T$ sampled frames, an image captioning model is first employed to generate frame-wise textual descriptions. Visual and textual features are then extracted to construct visual, textual, and cross-modal self-similarity matrices. Each matrix is processed by a dedicated Transformer to capture temporal dynamics. Finally, a cross-attentive fusion module integrates these modality-specific cues into a unified representation for binary classification.

Method	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	LaVie	Sora	WS	mean
STIL † [ 22 ] , ACM MM 2021	88.21	88.68	73.07	56.91	62.07	83.96	64.87	65.54	49.86	63.60	69.68
FTCN † [ 23 ] , ICCV 2021	70.01	83.59	97.07	87.42	93.30	91.86	91.72	84.16	44.48	84.46	82.81
X-CLIP † [ 51 ] , ECCV 2022	79.84	87.54	95.53	90.71	94.54	88.69	93.50	86.28	64.23	88.54	89.62
TALL † [ 24 ] , ICCV 2023	51.11	63.63	92.09	44.00	51.06	93.47	87.85	59.07	15.82	64.43	62.25
FID † [ 52 ] , NeurIPS 2024	91.50	92.24	93.67	86.10	90.61	93.27	92.41	83.68	74.95	82.24	88.07
NPR † [ 53 ] , CVPR 2024	84.67	96.53	96.79	40.17	21.61	96.35	97.02	22.37	90.55	66.51	71.26

TABLE I : Comparison of detection AP (

\%

) between ATSS and 13 state-of-the-art baselines on GenVideo. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.

Method	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	LaVie	Sora	WS	mean
STIL † [ 22 ] , ACM MM 2021	86.48	88.75	82.77	60.00	68.71	86.67	73.56	72.67	54.18	69.94	74.37
FTCN † [ 23 ] , ICCV 2021	69.76	83.25	97.18	88.69	93.45	93.33	91.62	84.70	38.30	85.74	82.60
X-CLIP † [ 51 ] , ECCV 2022	80.51	86.14	95.12	91.51	94.69	88.96	92.84	85.04	64.09	87.93	88.87
TALL † [ 24 ] , ICCV 2023	58.46	60.54	83.24	45.21	46.24	73.30	66.37	48.40	66.36	53.84	60.20
FID † [ 52 ] , NeurIPS 2024	90.94	91.93	93.70	85.77	91.19	93.16	92.02	82.57	73.45	81.30	87.60
NPR † [ 53 ] , CVPR 2024	93.92	99.38	99.92	26.83	18.99	98.78	99.24	42.32	97.56	76.12	75.31

TABLE II : Comparison of detection AUC (

\%

Method	Floor33	Gen2	Gen2-D	HS-XL	LaVie-B	LaVie-I	Mix-SR	MS	MV	PKL	PKL-V1	Show-1	VC	ZS	mean
STIL †	90.42	90.59	68.07	59.36	65.96	66.02	65.41	89.20	74.71	95.86	79.97	60.21	67.14	89.73	75.90
FTCN †	82.85	88.50	94.72	88.25	82.71	81.91	92.74	71.16	96.65	96.36	93.14	93.55	92.23	69.09	87.42
X-CLIP †	87.32	87.56	90.72	89.67	85.58	83.55	94.22	79.45	96.04	97.44	96.15	95.68	91.42	77.63	90.28
TALL †	63.25	70.75	77.04	46.93	52.87	52.53	78.16	62.11	83.63	65.33	70.98	48.00	60.50	51.73	63.13
FID †	96.40	97.36	98.68	89.90	92.92	84.19	98.51	95.74	98.29	99.49	99.17	96.77	95.71	95.18	95.59
NPR †	99.77	99.34	99.95	47.39	76.45	72.23	99.67	98.54	99.96	99.97	99.93	69.82	99.68	98.21	90.07

TABLE III : Comparison of detection AP (

\%

) between ATSS and 13 state-of-the-art baselines on EvalCrafter. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.

Method	Floor33	Gen2	Gen2-D	HS-XL	LaVie-B	LaVie-I	Mix-SR	MS	MV	PKL	PKL-V1	Show-1	VC	ZS	mean
STIL †	90.14	91.40	79.25	61.62	73.63	71.23	74.22	87.28	84.22	97.82	88.50	66.68	74.49	90.77	80.80
FTCN †	82.78	90.76	95.11	90.29	84.51	83.00	92.31	70.99	96.65	96.73	94.01	93.62	91.27	70.82	88.06
X-CLIP †	86.10	86.70	90.93	90.80	84.06	83.01	93.59	79.69	95.56	97.45	95.79	95.29	90.71	80.53	89.38
TALL †	61.46	72.04	74.60	43.88	49.64	50.02	77.50	58.99	83.19	67.13	72.51	46.81	55.69	46.03	61.39
FID †	96.26	97.46	98.58	90.05	92.32	83.42	98.23	95.27	98.28	99.48	99.24	96.79	95.26	95.29	95.42
NPR †	99.38	97.74	99.80	26.83	50.23	36.00	99.22	93.92	99.92	99.92	99.71	18.99	99.25	92.60	79.54

TABLE IV : Comparison of detection AUC (

\%

Method	CVX	CVX-5B	DM	Gen-2	LaVie	OpenSora	Pika	SVD-T2I2V	VC2	ZS	mean
STIL † [ 22 ] , ACM MM 2021	66.43	69.61	65.31	69.63	63.68	61.94	97.07	67.51	67.54	58.56	68.73
FTCN † [ 23 ] , ICCV 2021	74.24	74.83	67.98	93.99	69.16	66.95	93.94	82.81	83.39	75.64	78.29
X-CLIP † [ 51 ] , ECCV 2022	87.35	83.72	75.54	92.09	79.52	90.44	96.13	85.26	85.35	88.88	86.66
TALL † [ 24 ] , ICCV 2023	39.59	50.72	62.36	70.78	40.40	37.30	62.69	52.62	52.66	50.66	51.98
FID † [ 52 ] , NeurIPS 2024	93.34	91.41	97.50	98.35	96.51	87.90	99.55	95.66	96.03	90.60	94.69
NPR † [ 53 ] , CVPR 2024	81.37	81.99	99.86	99.90	63.72	88.78	99.91	99.54	60.21	78.23	85.35

TABLE V : Comparison of detection AP (

\%

) between ATSS and 13 state-of-the-art baselines on VideoPhy. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.

Method	CVX	CVX-5B	DM	Gen-2	LaVie	OpenSora	Pika	SVD-T2I2V	VC2	ZS	mean
STIL † [ 22 ] , ACM MM 2021	71.73	73.94	73.01	80.75	68.66	63.42	98.47	77.82	75.18	55.85	73.88
FTCN † [ 23 ] , ICCV 2021	80.91	81.01	70.20	95.21	72.88	76.37	95.90	87.07	86.53	74.88	82.10
X-CLIP † [ 51 ] , ECCV 2022	88.99	84.84	78.57	93.35	76.51	92.68	96.22	84.92	86.27	88.06	86.88
TALL † [ 24 ] , ICCV 2023	30.05	46.36	56.54	64.80	32.94	24.82	63.65	45.40	42.48	39.44	44.65
FID † [ 52 ] , NeurIPS 2024	93.37	91.72	97.62	98.11	96.03	86.03	99.59	95.81	95.68	89.38	94.33
NPR † [ 53 ] , CVPR 2024	72.10	73.60	99.70	99.80	42.90	83.50	99.80	99.50	47.20	52.90	77.10

TABLE VI : Comparison of detection AUC (

\%

Method	MS	OpenSora	Pika	ST2V	T2VZ	VC2	mean
STIL † [ 22 ] , ACM MM 2021	43.68	63.15	95.28	55.46	48.68	61.49	61.29
FTCN † [ 23 ] , ICCV 2021	74.65	90.05	94.26	83.53	46.25	87.27	79.33
X-CLIP † [ 51 ] , ECCV 2022	82.24	93.47	95.80	86.42	70.92	91.60	88.05
TALL † [ 24 ] , ICCV 2023	50.93	54.50	63.47	51.50	60.99	59.70	56.85
FID † [ 52 ] , NeurIPS 2024	91.35	87.68	99.59	97.87	68.51	85.92	88.49
NPR † [ 53 ] , CVPR 2024	87.04	89.85	99.98	89.88	88.93	70.79	87.75

TABLE VII : Comparison of detection AP (

\%

) between ATSS and 13 state-of-the-art baselines on VidProM. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.

Method	MS	OpenSora	Pika	ST2V	T2VZ	VC2	mean
STIL † [ 22 ] , ACM MM 2021	41.05	72.01	97.42	67.12	47.66	67.98	65.54
FTCN † [ 23 ] , ICCV 2021	70.98	90.76	95.23	87.81	50.70	88.43	80.65
X-CLIP † [ 51 ] , ECCV 2022	80.59	93.02	95.85	87.47	75.86	91.24	87.34
TALL † [ 24 ] , ICCV 2023	45.29	56.39	66.91	50.31	57.80	53.48	55.03
FID † [ 52 ] , NeurIPS 2024	89.86	87.36	99.62	98.10	67.01	85.65	87.93
NPR † [ 53 ] , CVPR 2024	82.61	98.56	99.84	98.92	93.32	56.70	88.33

TABLE VIII : Comparison of detection AUC (

\%

Method	GenVideo	EvalCrafter	VideoPhy	VidProM
STIL † [ 22 ] , ACM MM 2021	59.90	66.78	56.62	56.58
FTCN † [ 23 ] , ICCV 2021	70.52	73.87	62.43	67.23
X-CLIP † [ 51 ] , ECCV 2022	75.34	76.82	69.26	73.37
TALL † [ 24 ] , ICCV 2023	57.47	58.42	48.78	54.02
FID † [ 52 ] , NeurIPS 2024	54.57	63.59	65.01	54.44
NPR † [ 53 ] , CVPR 2024	65.41	71.36	57.00	68.04

TABLE IX : Comparison of average detection ACC (%) of ATSS and 13 state-of-the-art baselines on the GenVideo, EvalCrafter, VideoPhy, and VidProM datasets. † Results are reproduced using the officially released code. †† Results are reproduced from our implementation based on the original paper, as no official code is available.

Visual	Textual	Cross-Modal	Cross-Attentive	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	LaVie	Sora	WS	mean
$\checkmark$	$\times$	$\times$	$\times$	45.39	44.62	43.70	45.61	43.76	43.12	44.18	44.24	47.47	43.84	44.59
$\times$	$\checkmark$	$\times$	$\times$	85.69	90.30	94.54	94.45	94.99	96.02	95.50	93.81	72.59	88.52	90.64
$\times$	$\times$	$\checkmark$	$\times$	87.17	89.89	95.34	95.01	93.12	95.85	95.98	93.49	86.05	88.58	92.05
$\times$	$\checkmark$	$\checkmark$	$\checkmark$	83.85	89.67	88.67	94.21	93.64	93.15	94.38	89.18	82.08	88.23	89.71
$\checkmark$	$\times$	$\checkmark$	$\checkmark$	92.55	97.27	99.08	98.82	99.36	99.38	99.55	96.02	86.03	97.49	96.55
$\checkmark$	$\checkmark$	$\times$	$\checkmark$	90.07	95.40	97.29	96.04	97.22	97.09	98.42	95.26	81.77	93.11	94.17

TABLE X : Ablation performance of individual components in ATSS on GenVideo, reported in terms of AP (%).

Visual	Textual	Cross-Modal	Cross-Attentive	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	LaVie	Sora	WS	mean
$\checkmark$	$\times$	$\times$	$\times$	50.75	49.90	47.57	52.03	47.87	45.75	48.48	48.88	53.44	48.10	49.28
$\times$	$\checkmark$	$\times$	$\times$	84.56	89.66	93.81	93.74	94.45	95.73	94.90	93.04	72.40	87.06	89.93
$\times$	$\times$	$\checkmark$	$\times$	87.14	91.24	95.32	95.09	94.45	95.68	95.97	93.51	84.08	88.11	92.06
$\times$	$\checkmark$	$\checkmark$	$\checkmark$	85.22	91.53	93.47	95.73	96.26	96.03	96.42	92.39	84.24	89.14	92.04
$\checkmark$	$\times$	$\checkmark$	$\checkmark$	90.06	96.84	98.89	98.47	99.26	99.25	99.44	95.02	85.85	96.54	95.96
$\checkmark$	$\checkmark$	$\times$	$\checkmark$	90.04	95.39	96.93	96.73	96.93	97.37	98.21	95.34	84.28	93.21	94.44

TABLE XI : Ablation performance of individual components in ATSS on GenVideo, reported in terms of AUC (%).

Visual	Textual	Cross-Modal	Cross-Attentive	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	LaVie	Sora	WS	mean
$\checkmark$	$\times$	$\times$	$\times$	63.02	66.27	68.26	69.84	66.03	65.06	69.38	66.07	62.75	64.75	66.14
$\times$	$\checkmark$	$\times$	$\times$	76.51	81.90	85.90	86.75	86.51	88.41	87.49	84.92	66.67	77.76	82.28
$\times$	$\times$	$\checkmark$	$\times$	78.97	84.05	87.41	87.38	87.38	88.12	87.81	86.11	71.57	80.11	83.89
$\times$	$\checkmark$	$\checkmark$	$\checkmark$	78.41	84.29	86.88	89.21	90.00	89.90	89.04	86.07	79.41	82.16	85.54
$\checkmark$	$\times$	$\checkmark$	$\checkmark$	82.70	88.33	92.46	91.59	92.14	93.48	92.02	87.22	78.43	89.47	88.78
$\checkmark$	$\checkmark$	$\times$	$\checkmark$	83.10	89.13	87.68	89.21	88.25	89.33	90.23	87.70	76.47	85.44	86.65

TABLE XII : Ablation performance of individual components in ATSS on GenVideo, reported in terms of ACC (%).

Caption Model	Visual & Textual Encoder	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	Lavie	Sora	WS	mean
BLIP-base	CLIP-P16	92.25	97.25	98.69	98.56	99.32	99.18	99.56	95.49	84.90	96.00	96.12
	CLIP-P32	94.00	97.70	99.02	98.69	99.25	99.22	99.63	97.31	87.80	97.07	96.97
	XCLIP-P16	76.37	79.21	80.76	82.16	82.66	83.29	83.77	79.08	63.37	83.25	79.39
	XCLIP-P32	91.92	96.68	99.09	98.23	98.72	99.35	99.15	97.94	72.66	95.57	94.93
BLIP-large	CLIP-P16	86.86	93.35	94.76	94.09	95.89	94.27	96.16	94.15	73.97	88.62	91.21
	CLIP-P32	82.97	89.53	93.64	92.36	91.87	93.07	93.72	90.50	74.32	86.47	88.84

TABLE XIII : Ablation results of ATSS using various image captioning backbones and visual-textual encoders on the GenVideo dataset, measured by AP (%).

Caption Model	Visual & Textual Encoder	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	Lavie	Sora	WS	mean
BLIP-base	CLIP-P16	90.09	96.86	98.22	98.25	99.21	98.94	99.45	93.53	83.20	94.65	95.24
	CLIP-P32	93.15	97.74	98.91	98.69	99.33	99.15	99.62	96.50	87.77	96.24	96.71
	XCLIP-P16	83.08	86.75	89.55	89.61	90.10	90.14	91.73	86.04	71.32	88.43	86.67
	XCLIP-P32	90.30	96.04	98.92	98.84	98.64	99.43	99.02	97.32	76.66	94.40	94.96
BLIP-large	CLIP-P16	85.25	91.50	93.74	93.42	95.28	93.02	95.20	93.33	69.24	86.43	89.64
	CLIP-P32	85.06	91.01	95.46	93.72	93.86	94.54	95.47	91.14	78.66	88.51	90.74

TABLE XIV : Ablation results of ATSS using various image captioning backbones and visual-textual encoders on the GenVideo dataset, measured by AUC (%).

Caption Model	Visual & Textual Encoder	MS	MPS	MV	HotShot	Show-1	Gen2	Crafter	Lavie	Sora	WS	mean
BLIP-base	CLIP-P16	82.86	87.70	89.45	89.52	90.71	90.18	90.43	86.07	77.45	86.62	87.10
	CLIP-P32	84.84	92.86	96.19	95.24	96.83	96.78	97.26	91.67	73.53	91.26	91.64
	XCLIP-P16	74.52	76.03	76.33	75.63	75.16	76.41	77.24	74.72	64.71	76.58	74.73
	XCLIP-P32	80.63	89.68	94.68	95.95	93.41	96.70	96.07	92.14	64.71	85.44	88.94
BLIP-large	CLIP-P16	76.27	80.40	81.83	81.75	84.13	81.44	83.76	83.10	68.63	77.63	79.89
	CLIP-P32	76.03	81.59	89.18	86.19	86.35	87.44	88.24	83.45	71.57	79.12	82.92

TABLE XV : Ablation results of ATSS using various image captioning backbones and visual-textual encoders on the GenVideo dataset, measured by ACC (%).

Panel of Figure 18 — Fig. 3 : t-SNE visualizations of 10 subsets on the GenVideo dataset: (a) Crafter, (b) Gen2, (c) HotShot, (d) Lavie, (e) ModelScope, (f) MoonValley, (g) MorphStudio, (h) Show-1, (i) Sora, and (j) WildScrape.

Fig. 4 : Visualization of Attention Density Maps. Each row displays the attention weights for the visual, textual, and cross-modal branches of a real video and ten AI-generated samples, which are randomly selected from MSR-VTT and each generator subset of GenVideo, respectively.

Explore similar work

CMTA: Leveraging Cross-Modal Temporal Artifacts for Generalizable AI-Generated Video Detection
May 1, 2026Hang Wang, Chao Shen, Chenhao Lin +3Cross-Modal Alignment Deepfake Detection
Xi’an Jiaotong University, Xi’an, China · The Hong Kong Polytechnic University, Hong Kong, China · Guangdong OPPO Mobile Communications Co., Ltd. +1
Retrieval-Driven Training-Free AI-Generated Video Attribution
Jul 31, 2026Renxi Cheng, Chaolei Han, Jie Gui +1Deepfake Detection
School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China · Purple Mountain Laboratories, Nanjing 210000, China · Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, China +2
TRACE: Trajectory Representation and Consistency Estimation for AI-Generated Video Detection
Sep 22, 2026Huangsen Cao, Hongkang chu, Siyao Yu +3Deepfake Detection
Zhejiang University · University of the Chinese Academy of Sciences · Nanjing University of Information Science and Technology +1

May 1, 2026J/K move · Enter open · S save

CMTA: Leveraging Cross-Modal Temporal Artifacts for Generalizable AI-Generated Video Detection

Hang Wang, Chao Shen, Chenhao Lin, Minghui Yang+2

Xi’an Jiaotong University, Xi’an, China · The Hong Kong Polytechnic University, Hong Kong, China · Guangdong OPPO Mobile Communications Co., Ltd.+1

Cross-Modal Alignment Deepfake Detection

Open paper AskPDF arXiv

The proliferation of advanced AI video synthesis techniques poses an unprecedented challenge to digital video authenticity. Existing AI-generated video (AIGV) detection methods primarily focus on uni-modal or spatiotemporal artifacts, but they overlook the rich cues within the visual-textual cross-modal space, especially the temporal stability of semantic alignment. In this work, we identify a distinctive fingerprint in AIGVs, termed cross-modal temporal artifact (CMTA). Unlike real videos that exhibit natural temporal fluctuations in cross-modal alignment due to semantic variations, AIGVs display unnaturally stable semantic trajectories governed by given input prompts. To bridge this gap, we propose the CMTA framework, a cross-modal detection approach that captures these unique temporal artifacts through joint cross-modal embedding and multi-grained temporal modeling. Specifically, CMTA leverages BLIP to generate frame-level image captions and utilizes CLIP to extract corresponding visual-textual representations. A coarse-grained temporal modeling branch is then designed to characterize temporal fluctuations in cross-modal alignment with a GRU. In parallel, a fine-grained branch is constructed to capture intricate inter-frame variations from integrated visual-textual features with a Transformer encoder. Extensive experiments on 40 subsets across four large-scale datasets, including GenVideo, EvalCrafter, VideoPhy, and VidProM, validate that our approach sets a new state-of-the-art while exhibiting superior cross-generator generalization. Code and models of CMTA will be released at https://github.com/hwang-cs-ime/CMTA

Abstract

Explore similar work

CMTA: Leveraging Cross-Modal Temporal Artifacts for Generalizable AI-Generated Video Detection

Retrieval-Driven Training-Free AI-Generated Video Attribution

TRACE: Trajectory Representation and Consistency Estimation for AI-Generated Video Detection