A Modular Zero-Shot Pipeline for Accident Detection, Localization, and Classification in Traffic Surveillance Video

Amey Thakur; Sarvesh Talele

cs.CVApr 5, 2026

A Modular Zero-Shot Pipeline for Accident Detection, Localization, and Classification in Traffic Surveillance Video

Authors: Amey Thakur, Sarvesh Talele

Organizations: Independent Researcher, Toronto, Canada · Independent Researcher, Mumbai, India

Abstract

We describe a zero-shot pipeline developed for the ACCIDENT @ CVPR 2026 challenge. The challenge requires predicting when, where, and what type of traffic accident occurs in surveillance video, without labeled real-world training data. Our method separates the problem into three independent modules. The first module localizes the collision in time by running peak detection on z-score normalized frame-difference signals. The second module finds the impact location by computing the weighted centroid of cumulative dense optical flow magnitude maps using the Farneback algorithm. The third module classifies collision type by measuring cosine similarity between CLIP image embeddings of frames near the detected peak and text embeddings built from multi-prompt natural language descriptions of each collision category. No domain-specific fine-tuning is involved; the pipeline processes each video using only pre-trained model weights. Our implementation is publicly available as a Kaggle notebook.

Figures & tables

Figure 1: Chronological sampled frames from a synthetic CARLA traffic incident in the ACCIDENT @ CVPR 2026 dataset. Most clips record approximately 18 seconds of motion at 20 FPS resolution.

Figure 2: Temporal localization on a synthetic CARLA video. Top: mean absolute frame difference $d_{t}$ across all frames. Bottom: z-score anomaly series $z_{t}$ after rolling-mean smoothing. The dashed line marks the detection threshold.

Figure 3: Cumulative Farneback optical flow magnitude $M(u,v)$ on a synthetic CARLA video after 90th-percentile thresholding. The bright region corresponds to the collision area; diffuse background motion is suppressed.

Type	Example prompt
head-on	“two cars colliding head-on from opposite directions”
rear-end	“a car colliding into the back of another car”
sideswipe	“two vehicles scraping alongside each other”
single	“a single car crashing into a wall or obstacle”
t-bone	“a car hitting the side of another car at an intersection”

Table 1: Prompt templates for collision type classification. Five prompts per class are averaged into one text embedding.

Figure 4: Distribution of ground-truth accident times (seconds) across the 2,211 synthetic videos. Most incidents occur within the first 10 seconds of the clip.

Figure 5: Collision type frequency in the synthetic development split. The rear-end category has 12 times more samples than the single-vehicle category.

Figure 6: Ground-truth impact point distribution across 2,211 synthetic videos, colored by collision type. Points cluster near the frame center, with head-on and sideswipe events showing the widest spatial spread.

Component	Parameter	Value
Temporal	Smoothing window $w$	5
Temporal	Z-score threshold $\tau$	1.5
Spatial	Start frame	centered on $t^{*}$
	Context window $T$	30 frames
	Pyramid scale	0.5
	Pyramid levels	3

Table 2: Pipeline hyperparameters, held constant for all test videos.

Figure 7: Predicted collision type distribution on the 2,027 real test videos. Sideswipe and single dominate the predictions, while rear-end is rarely selected, inverting the synthetic training distribution.

Explore similar work

SynCrash: A Multi-Stage Pipeline for Zero-Shot Accident Detection and Localization in Traffic Surveillance Video
Aug 30, 2026Arkya Jyoti Bagchi, Ritul Jangir, Varun RaskarTraffic Accident Anticipation Anomaly Localization
Indian Institute of Technology Jodhpur, India
Multi-Stage VLM Pipeline for Zero-Shot Traffic Accident Understanding
May 28, 2026Fumiya Tatematsu, Fumihiko TakahashiVLMs for Autonomous Driving Temporal Video Understanding
GO Drive Inc.
Zero-Shot Traffic Accident Detection via a Coarse-to-Fine VLM-Tracking Pipeline
Aug 9, 2026Dipit Saha, Shah Mohammad Abdul Mannan, Mohammad Raihan Rashid +2Temporal Video Understanding
Bangladesh University of Engineering and Technology

Aug 30, 2026J/K move · Enter open · S save

SynCrash: A Multi-Stage Pipeline for Zero-Shot Accident Detection and Localization in Traffic Surveillance Video

Arkya Jyoti Bagchi, Ritul Jangir, Varun Raskar

Indian Institute of Technology Jodhpur, India

Traffic Accident Anticipation Anomaly Localization Video Anomaly Detection Temporal Action Detection Zero-Shot Anomaly Detection

Open paper AskPDF arXiv

We present SynCrash, a multi-stage pipeline for zero-shot accident detection, spatial localization, and collision-type classification in fixed-view CCTV surveillance video. Our approach addresses the ACCIDENT at CVPR 2026 Challenge, which requires predicting when an accident occurs, where in the frame the impact happens, and what type of collision it is, all without access to labeled real-world training data. The pipeline operates in three decoupled stages: (1) Temporal localization via a VideoMAEv2-giant backbone fine-tuned on CARLA-based synthetic clips with metadata-aware embeddings and dense sliding-window inference; (2) Spatial localization using YOLO for object detection combined with a physics-informed hybrid heuristic that leverages bounding-box overlap and trajectory-based reasoning to predict the impact point; and (3) Collision-type classification using a lightweight rule-based strategy derived from the number and configuration of detected vehicles. The key insight is that temporal understanding benefits from supervised fine-tuning on synthetic data, whereas spatial understanding is better served by pretrained object detectors and physics priors that transfer naturally across domains.

Abstract

Explore similar work

SynCrash: A Multi-Stage Pipeline for Zero-Shot Accident Detection and Localization in Traffic Surveillance Video

Multi-Stage VLM Pipeline for Zero-Shot Traffic Accident Understanding

Zero-Shot Traffic Accident Detection via a Coarse-to-Fine VLM-Tracking Pipeline