Organizations: Huazhong University of Science and Technology · Institute of Automation, Chinese Academy of Sciences · Jilin University · Vast Intelligence Lab
AI-generated videos (AIGVs) typically contain subtle temporal artifacts that arise from inter-frame inconsistencies rather than within individual frames. A detector that captures such artifacts should therefore benefit from video pretrained backbones over image only ones. In practice, however, video backbones with standard global readouts often fail to outperform strong image pretrained probes on AIGV benchmarks. We attribute this gap to excessive spatiotemporal aggregation in the readout. Video pretrained backbones tend to compress each frame into a single global descriptor. This compression suppresses local patch level temporal dynamics and discards inter patch relations, which are precisely the cues that AIGV detection most reliably depends on. Based on this, we propose Velocity Gated Patch Velocity Profiling (V-PVP), a lightweight readout that replaces only the aggregation layer with two parallel streams over the patch velocity field, adding only about 0.5M trainable parameters. V-PVP serves as a general plug-and-play module that consistently improves performance across diverse video backbones under both end-to-end fine-tuning and linear probing settings. Our method reaches 95.28 AUC on AIGVDBench while keeping the backbone fully frozen. The results show that simply replacing the aggregation layer reactivates the temporal potential of frozen video backbones, restoring their advantage on AIGV detection. Code is available at https://github.com/manic-cui/Rethinking-the-readout.
Figures & tables
Figure 1: The readout is the bottleneck for AIGV detection. Left: AIGV detection relies on patch level temporal cues. Middle: Each colored cell represents a patch token, with color depth indicating activation strength. Standard video backbones aggregate these tokens into a single global vector via GAP or CLS, which suppresses patch level variations. Right: This collapse weakens local details and erases structured inter patch relations, leaving the bottleneck in the readout rather than in the backbone representation.
Figure 2: The readout is the bottleneck. Real vs fake statistics on all 63k AIGVDBench test videos. (a) Loss 1. Spectral condition number of the patch velocity matrix Vt∈RP×D , defined as κspec=logσ1−logσ10 ; vertical lines mark the group means. Generated videos are systematically higher, with Cohen’s d≈0.64 . This signal is no longer measurable after aggregation. (b) Loss 2. Per channel velocity energies, normalized so that the real video mean equals 1 ; whiskers show ±1 SD. Pre pool, real and fake energies differ by 1.00 vs 0.80 with Cohen’s d≈0.69 . After pooling, the gap shrinks to 1.00 vs 0.87 with d≈0.41 .
Figure 3: Overall architecture of V-PVP. (a) The video backbone stays frozen and only the readout ( ≈0.5 M parameters) is trained: two streams over the patch velocity vt(p)=zt+1(p)−zt(p) are projected by Wz and Wr , concatenated into xt , and read by a temporal head into a real/fake score. (b) An MLP scores each patch from [zt(p),∥vt(p)∥2] , and the softmax weights αt(p) pool the patch tokens into z~t . (c) The element-wise ∣vt(p)∣ , averaged over patches, gives rt .
I2V
T2V
V2V
Method
Easy Animate
LTX
Pyramid Flow
SEINE
SVD
Video Crafter
Acc Video
Animate Diff
Cogvideo x1.5
Easy Animate
Hunyuan
IPOC
LTX
Open Sora
Pyramid Flow
Rep Video
Video Crafter
Wan 2.1
Cogvideo x1.5
LTX
AVG
AI-Generated Image Detection Models
NPR [ 11 ]
96.78
85.76
99.22
99.21
99.93
93.97
68.32
66.10
93.72
96.23
74.35
93.82
92.04
99.99
99.45
86.50
98.44
76.72
96.96
80.50
89.90
UnivFD [ 25 ]
80.18
82.77
86.83
94.29
96.39
80.01
82.74
93.29
94.77
90.77
83.27
97.49
93.30
99.72
97.23
90.36
98.03
84.16
93.47
81.56
90.03
Effort [ 13 ]
81.88
82.29
86.96
92.90
96.02
79.80
82.39
93.42
94.82
92.44
83.85
97.75
93.28
99.74
97.71
92.18
98.33
85.58
93.54
80.88
90.29
DDA [ 12 ]
78.28
90.54
88.79
99.57
99.99
99.76
86.02
92.43
89.82
75.77
87.34
97.58
92.15
98.85
88.78
82.53
99.74
89.70
89.93
87.08
90.73
Table 1: Per-generator AUC on the AIGVDBench open-source split (20 generators). Trained to convergence (Section 5.1 ), unlike the 5-epoch ablations in Table 3 . Best average in bold , second underlined .
Figure 4: Per-generator AUC on AIGVDBench of fully fine-tuned VideoMAE with the standard readout (gray) and frozen VideoMAE with V-PVP (green), with the gain of V-PVP on the right. ∗ marks the fake training generators.
Method
MuseV
SVD
CogVideo
Mora
AVG
AI-Generated Image Detection Models
NPR [ 11 ]
48.87
30.03
91.27
30.89
50.26
UnivFD [ 25 ]
65.79
55.37
97.40
96.46
78.76
Effort [ 13 ]
65.85
55.51
97.88
97.08
79.08
DDA [ 12 ]
91.72
87.40
96.92
95.12
92.79
MIRROR [ 26 ]
77.29
74.59
59.93
91.42
75.81
Table 2: Per-generator AUC on GenVidBench-143k . Best average in bold , second underlined .
Table 3: Ablation studies on V-PVP. Fixed 5-epoch ablation setting. Left: stream-level ablation. Right: alternative readouts on patch tokens or per-patch energy ( z : patch tokens; E : ∥vt(p)∥2 ).
Figure 5: AUC comparison across different backbones and training strategies on AIGVDBench
Adaptation
Trainable params
Standard readout
V-PVP
Δ
Frozen
∼ 0.5 M
89.82
95.28
+5.5
LoRA ( r=8 )
∼ 1 M
88.22
95.06
+6.8
Full fine-tune
∼ 87 M
88.34
97.21
+8.9
Table 4: V-PVP composes with backbone adaptation. mga AUC on AIGVDBench with VideoMAE under three adaptation schemes. V-PVP improves over the standard readout under every scheme; the frozen V-PVP configuration already exceeds full fine-tuning under the standard readout at a fraction of the parameter cost.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Backbone
Architecture
VideoMAE-Base
Pretraining
Kinetics-400
Frozen
Yes
Token tap
Last transformer block (layer 11), pre-final-norm
Spatial patch tokens ( P )
196
Appendix
Table 5: Default V-PVP training configuration. Values are shared across the reported V-PVP runs unless explicitly stated otherwise. The number of training epochs is not fixed for the main comparisons; training stops once the validation AUC fails to improve by more than 0.5 percentage points for three consecutive epochs.
Backbone
Adaptation
Trainable params
Std readout
V-PVP
Δ
I3D-R50
Frozen
∼ 1.0 M
83.62
86.53
+2.91
SlowFast-R50
Frozen
∼ 1.0 M
82.35
85.06
+2.71
Appendix
Table 6: V-PVP versus the standard readout on frozen CNN backbones, mga AUC on AIGVDBench.
ϕ
Violation
Per-channel d (mean ± std)
Relative gap
∣⋅∣ (V-PVP)
none
0.372±0.092
—
ReLU
first criterion
0.340±0.102
−9%
x2
third criterion
0.321±0.101
−14%
id ( vˉt )
no pre-summation non-linearity
0.153±0.111
−59%
Appendix
Table 7: Per-channel discriminability of four ϕ candidates. Relative gap is computed against ϕ=∣⋅∣ .
Figure 6: Velocity gated weights on (a) successes and (b) failures of V-PVP. Each scene pairs a real clip (blue) with an image-to-video fake generated from its first frame (orange): eight frames, the weights between consecutive frames (GAP =1 ), and their mean. Right: weight persistence r (correlation of the weights between consecutive frame pairs), probability of being generated (wrong decisions in bold), and mean patch velocity relative to real test clips.
Quantity
Value
ϕ(ez,er)
0.103±0.024
P(er=1) marginal
0.182
P(er=1∣ez=1) conditional
0.214
Appendix
Table 8: Sample-level error correlation between the two single-stream models. Mean across three random seeds with standard deviation.
Xi’an Jiaotong University, Xi’an, China · The Hong Kong Polytechnic University, Hong Kong, China · School of Engineering and Technology, University of Washington, Tacoma, WA 98402, USA