We introduce ACCIDENT, a benchmark dataset for traffic accident detection in CCTV footage, designed to evaluate models in supervised (IID and OOD) and zero-shot settings, reflecting both data-rich and data-scarce scenarios. The benchmark consists of a curated set of 2,027 real and 2,211 synthetic clips annotated with the accident time, spatial location, and high-level collision type. We define three core tasks: (i) temporal localization of the accident, (ii) its spatial localization, and (iii) collision type classification. Each task is evaluated using custom metrics that account for the uncertainty and ambiguity inherent in CCTV footage. In addition to the benchmark, we provide a diverse set of baselines, including heuristic, motion-aware, and vision-language approaches, and show that ACCIDENT is challenging. You can access the ACCIDENT at: https://accidentbench.github.io
Figures & tables
Type of annotations
Dataset
# clips
Object
Time
Location
Type
UCF-Crime [ 34 ]
150
∘
∘
∘
∘
Smart City [ 1 ]
927
∘
∘
∘
∙
CADP [ 31 ]
240 †
∙
∙
∘
∘
TAD [ 41 ]
294
∘
∙
∙
∙
TADS [ 10 ]
966 ‡
∘
∙
∙
∙
Table 1: Comparison of accident-related surveillance datasets. Existing datasets vary substantially in scale and annotation design, often providing only partial coverage of object, temporal, spatial, and collision-type labels. ACCIDENT unifies accident time, location, and type annotations across real ( R ) and synthetic ( S ) data. † Only a subset includes temporal labels. ‡ Classes not defined by collision geometry.
Figure 2: Diversity and challenge factors in the ACCIDENT dataset. (a) Video quality is biased towards low and medium, reflecting the typical quality for CCTV feeds. (b) Scene layouts are dominated by highways and intersections with multi-agent scenarios, critical for robust model evaluation. (c) Weather distributions include a range of rain and snow conditions, enabling stress testing under adverse environments. (d) Accident types are well distributed, avoiding over-concentration in a single class and ensuring balanced evaluation.
Figure 3: Synthetic accident scenarios. Examples are generated from static third-person viewpoints that emulate CCTV. Road geometry, lighting, and weather are varied to stress-test detection under diverse conditions, while accident timing and agent behavior is controlled. Top: raw footage. Bottom: annotations showing class-labeled 2D boxes, collision indicator, and semantic segmentation.
Data Split
Training
Test
Clips
Fraction
Clips
Fraction
In-distribution
507
25%
1,520
75%
Geo-aware
454
22.4%
1,573
77.6%
Zero-shot
–
0%
2,027
100%
Synthetic
2,211
100%
–
0%
Table 2: Benchmarks statistics. The IID and OOD splits are sampled to maintain a similar training/test ratio.
Figure 4: Zero-shot spatial localization with Molmo-7B . The top row shows representative successful localizations of the accident, while the bottom row shows typical failure cases, often caused by occlusion, poor lighting, low video quality, or ambiguous scenes.
Method
σt=0.5
σt=1
σt=2
Naive Baseline
0.107
0.190
0.295
Optical Flow (OF)
0.140
0.266
0.466
Object Size Dynamics (OSD)
0.136
0.259
0.452
OF + OSD
0.150
0.287
0.496
Qwen2.5-VL-7B
0.145
0.266
0.477
Molmo-7B
0.207
0.343
0.539
Table 3: Zero-shot temporal localization. Mean Gaussian similarity to impact time evaluated at three temporal tolerances ( σ={0.5,1,2} ). Human performance as mean ± std.
Method
Full image
Fixed crop
Naive Baseline
0.335
Zero-shot VLMs
Qwen2.5-7B
0.115
0.119
Molmo-7B
0.271
0.262
Sim → Real
SigLIP2
0.398
0.471
DINO-v2
0.440
0.363
Human Performance
0.9228 ±0.0263
Table 4: Collision-type classification ( C ) baselines. Top-1 accuracy across two visual contexts. Zero-shot VLMs are evaluated without task-specific tuning. Sim → real classification follows the standard linear probing setting using the subset of synthetic data. The Naive Baseline is based on the majority class; Human Performance reports mean ± std across annotators.
Oracle-Aided
End-to-End
Model
T
S
C
T
S
C
ACC S
Naive
0.190
0.250
0.335
0.190
0.250
0.335
0.245
Heuristics
0.287
0.339
–
0.287
0.273
–
–
Molmo-7B
0.343
0.596
0.271
0.343
0.488
0.293
0.358
Best from all
0.343
0.596
0.471
0.343
0.488
0.433
0.412
Human
0.979
0.995
0.923
–
–
–
–
Table 5: Baseline end-to-end performance on ACCIDENT. Temporal ( T ) and spatial localization ( S ), and collision-type classification ( C ). Oracle-aided evaluates each component in isolation using ground-truth inputs, whereas End-to-End chains predictions ( t^→p^→c^ ) and therefore exposes error propagation across the pipeline. Heuristics uses OSD+OF for T and the bounding-box midpoint for S . Best from all chains the strongest components from the preceding analyses; for C we use DINO-v2 for its robustness to temporal and spatial errors. It should therefore be interpreted as a modular upper bound rather than as a single model. The Naive Baseline is identical across regimes because it ignores visual content. Human Performance is shown for reference.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
A1
A2
A3
A4
A5
A6
mean ± std
T
0.978
0.982
0.986
0.979
0.979
0.973
0.979±0.004
S
0.993
0.996
0.997
0.995
0.996
0.993
0.995±0.001
C
0.949
0.870
0.915
0.931
0.928
0.944
0.923±0.026
Appendix
Table 6: Inter-annotator agreement accuracy. Per-task accuracies against the consensus. Annotators with > 700 clips only. Rows denote T = temporal , S = spatial , C = collision type . For temporal localization, σt=1 is used.
Figure 5: Collision-type annotation confusion. As expected, errors concentrate between semantical similar classes (e.g., t-bone vs. sideswipe ), with just handful of unrelated confusions.
Video surveillance uses computer vision and multimodal AI to detect, track, explain, and retrieve events in camera streams while balancing reliability, efficiency, and privacy.