Beyond the Beep: Scalable Collision Anticipation and Real-Time Explainability with BADAS-2.0

Roni Goldshmidt; Hamish Scott; Lorenzo Niccolini; Hernan Matzner

cs.CVApr 7, 2026

Beyond the Beep: Scalable Collision Anticipation and Real-Time Explainability with BADAS-2.0

Authors: Roni Goldshmidt, Hamish Scott, Lorenzo Niccolini, Hernan Matzner

Organizations: Nexar AI

Abstract

We present BADAS-2.0, the second generation of our collision anticipation system, building on BADAS-1.0, which showed that fine-tuning V-JEPA2 on large-scale ego-centric dashcam data outperforms both academic baselines and production ADAS systems. BADAS-2.0 advances the state of the art along three axes. (i) Long-tail benchmark and accuracy: We introduce a 10-group long-tail benchmark targeting rare and safety-critical scenarios. To construct it, BADAS-1.0 is used as an active oracle to score millions of unlabeled drives and surface high-risk candidates for annotation. Combined with Nexar's Atlas platform for targeted data collection, this expands the dataset from 40k to 178,500 labeled videos (~2M clips), yielding consistent gains across all subgroups, with the largest improvements on the hardest long-tail cases. (ii) Knowledge distillation to edge: Domain-specific self-supervised pre-training on 2.25M unlabeled driving videos enables distillation into compact models, BADAS-2.0-Flash (86M) and BADAS-2.0-Flash-Lite (22M), achieving 7-12x speedup with near-parity accuracy, enabling real-time edge deployment. (iii) Explainability: BADAS-2.0 produces real-time object-centric attention heatmaps that localize the evidence behind predictions. BADAS-Reason extends this with a vision-language model that consumes the last frame and heatmap to generate driver actions and structured textual reasoning. Inference code and evaluation benchmarks are publicly available.

Figures & tables

	BADAS-Open	BADAS-1.0	BADAS-2.0
Labeled videos	1.5k	40k	178.5k
Training clips	15k	400k	$\sim$ 2M
Positives	0.8k	20k	24.1k
Negatives	0.8k	20k	119.5k
Hard-braking events	—	—	34.9k
Non-front collisions	—	—	4.5k

Table 1: BADAS dataset evolution. Clips are windowed training segments extracted from labeled videos.

	Kaggle (AP@TTA)			Kaggle
Model	@0.5s	@1.0s	@1.5s	mAP	FPR $\downarrow$	Params
BADAS-1.0 (ViT-L)	0.935	0.936	0.904	0.925	10.9%	300M
BADAS-2.0 (ViT-L)	0.943	0.957	0.921	0.940	4.6%	300M
BADAS-2.0-Flash (ViT-B)	0.945	0.962	0.915	0.941	9.7%	86M
BADAS-2.0-Flash-Lite (ViT-S)	0.946	0.947	0.907	0.933	12.2%	22M

Table 2: Main results. Nexar Kaggle competition (single window, mean AP over three lead-time thresholds). Best per column in bold ; second-best underlined .

\downarrow

lower is better for FPR.

Model	SSL	KD	AP	F1	FPR $\downarrow$
ViT-S (no SSL)	$\times$	$\times$	0.693	0.629	35.5%
ViT-S (SSL only)	$\checkmark$	$\times$	0.974	0.880	20.6%
BADAS-2.0-Flash-Lite	$\checkmark$	$\checkmark$	0.984	0.926	9.1%

Table 3: SSL pre-training ablation (ViT-S). Evaluated on the 10-group long-tail benchmark. Without SSL, random initialization produces near-random AP. SSL alone approaches production quality; SSL plus distillation closes the remaining gap.

Model	Arch	SSL	PGA	$\Delta$ rand
Random baseline	—	—	11.5%	—
BADAS-1.0	ViT-L	$\times$	49.8%	+38.3 pp
BADAS-2.0	ViT-L	$\times$	52.4%	+40.9 pp
BADAS-2.0-Flash-Lite	ViT-S	$\checkmark$	69.8%	+58.3 pp
BADAS-2.0-Flash	ViT-B	$\checkmark$	72.1%	+60.6 pp

Table 4: Heatmap localization (PGA). Random baseline: 11.5% (mean GT box coverage over 1,894 clips). SSL-initialized backbones achieve substantially higher localization than the ViT-L models, despite having 3–14

\times

fewer parameters.

	Loss	Perplexity	Act. Match
Baseline (Qwen3-VL-4B)	2.657	14.25	12.2%
BADAS-Reason (fine-tuned)	0.612	1.84	43.6%
Image format ablation (val loss):
Bounding-box crop (ours)	0.612	—	—
Heatmap overlay	0.628	—	—

Table 5: BADAS-Reason results on held-out test set (

n

=887). Ablation (bottom) compares image preparation strategies.

	BADAS-1.0				BADAS-2.0				BADAS-2.0-Flash				BADAS-2.0-Flash-Lite
Group	AUC	F1	EWR	TTA	AUC	F1	EWR	TTA	AUC	F1	EWR	TTA	AUC	F1	EWR	TTA
Animal	0.948	0.842	66.1%	0.63	0.964	0.938	78.9%	0.84	0.948	0.929	73.4%	0.86	0.924	0.881	67.0%	0.91
Pedestrian	0.991	0.929	94.9%	1.68	0.998	0.981	93.7%	1.32	0.996	0.957	94.9%	1.44	0.991	0.944	87.3%	1.57
Intersection	0.988	0.803	94.4%	1.96	1.000	0.973	96.3%	1.76	1.000	0.915	96.3%	1.88	0.998	0.931	92.6%	1.75
Pass/Overtake	0.974	0.853	96.7%	1.49	1.000	0.938	100%	1.47	1.000	0.938	100%	1.67	0.997	0.923	96.7%	1.76
Cyclist	0.986	0.896	93.3%	1.33	1.000	0.923	93.3%	1.16	0.987	0.879	93.3%	1.28	0.994	0.923	93.3%	1.22

Table 6: Long-tail benchmark results (10 groups, 888 clips, sliding window, threshold 0.75). Columns: AUC, F1, Early Warning Recall (EWR), Mean Time to Alert (MTTA, in seconds). Best AUC, F1, and EWR per group in bold ; ties also shown in bold.

	DAD		DoTA		DADA-2000
Model	AUC	AP	AUC	AP	AUC	AP
BADAS-1.0	0.99	0.94	0.72	0.95	0.87	0.90
BADAS-Open	0.87	0.66	0.70	0.94	0.77	0.87
BADAS-2.0	0.993	0.922	0.991	0.999	0.991	0.996
BADAS-2.0-Flash	0.987	0.849	0.985	0.998	0.990	0.995
BADAS-2.0-Flash-Lite	0.982	0.870	0.985	0.998	0.981	0.992

Table 7: External benchmark results (sliding window, ego-centric evaluation). DAD has only 13 ego-involved positives; a single missed detection has outsized AP impact. Best per column in bold .

Model	Params	A100	J. Thor	Speedup
BADAS-1.0	300M	2500 ms	—	—
BADAS-2.0	300M	34 ms	41 ms	74 $\times$
BADAS-2.0-Flash	86M	4.8 ms	12.5 ms	521 $\times$
BADAS-2.0-Flash-Lite	22M	2.8 ms	5.9 ms	893 $\times$

Table 8: Inference latency per prediction window. FP16 precision with ahead-of-time compilation (end-to-end,

+

0.4–0.7 ms preprocessing). BADAS-1.0 measured on A100 FP32 with full-video decode.

Traffic Accident Anticipation

Traffic accident anticipation predicts imminent collisions or rising crash risk early enough to support driver warnings and proactive road-safety decisions.

See all 22 papers

Collision-Aware Vision-Language Learning for End-to-End Driving with Multimodal Infraction Datasets
Mar 26, 2026Alex Koran, Dimitrios Sinodinos, Hadi Hojjati +3End-to-End Autonomous Driving Traffic Accident Anticipation
McGill University, Montréal, Canada · Mila - Québec AI Institute, Montréal, Canada · Nissan Motor Corporation, Yokohama, Japan
A VideoMAE-v2 Approach to Zero-Shot Traffic Accident Anticipation
Jun 8, 2026Siyuan Li, Xiaoyang Bi, Mengshi QiZero-Shot Learning Autonomous Driving
State Key Laboratory of Networking and Switching Technology · Beijing University of Posts and Telecommunications
ACCIDENT: A Benchmark Dataset for Vehicle Accident Detection from Traffic Surveillance Videos
Apr 10, 2026Lukas Picek, Michal Čermák, Marek Hanzl +1Video Surveillance
PiVa AI · MIT · University of West Bohemia in Pilsen +1

Mar 26, 2026J/K move · Enter open · S save

Collision-Aware Vision-Language Learning for End-to-End Driving with Multimodal Infraction Datasets

Alex Koran, Dimitrios Sinodinos, Hadi Hojjati, Takuya Nanri+2

McGill University, Montréal, Canada · Mila - Québec AI Institute, Montréal, Canada · Nissan Motor Corporation, Yokohama, Japan

End-to-End Autonomous Driving Traffic Accident Anticipation Multimodal Anomaly Detection VLMs for Autonomous Driving Autonomous Driving Datasets

Open paper AskPDF arXiv

High infraction rates remain the primary bottleneck for end-to-end (E2E) autonomous driving, as evidenced by the low driving scores on the CARLA Leaderboard. Despite collision-related infractions being the dominant failure mode in closed-loop evaluations, collision-aware representation learning has received limited attention. To address this gap, we first develop a Video-Language-Augmented Anomaly Detector (VLAAD), leveraging a Multiple Instance Learning (MIL) formulation to obtain stable, temporally localized collision signals for proactive prediction. To transition these capabilities into closed-loop simulations, we must overcome the limitations of existing simulator datasets, which lack multimodality and are frequently restricted to simple intersection scenarios. Therefore, we introduce CARLA-Collide, a large-scale multimodal dataset capturing realistic collision events across highly diverse road networks. Trained on this diverse simulator data, VLAAD serves as a collision-aware plug-in module that can be seamlessly integrated into existing E2E driving models. By integrating our module into a pretrained TransFuser++ agent, we demonstrate a 14.12% relative increase in driving score with minimal fine-tuning. Beyond closed-loop evaluation, we further assess the generalization capability of VLAAD in an open-loop setting using real-world driving data. To support this analysis, we introduce Real-Collide, a multimodal dataset of diverse dashcam videos paired with semantically rich annotations for collision detection and prediction. On this benchmark, despite containing only 0.6B parameters, VLAAD outperforms a multi-billion-parameter vision-language model, achieving a 23.3% improvement in AUC.

Abstract

Traffic Accident Anticipation

Explore similar work

Collision-Aware Vision-Language Learning for End-to-End Driving with Multimodal Infraction Datasets

A VideoMAE-v2 Approach to Zero-Shot Traffic Accident Anticipation

ACCIDENT: A Benchmark Dataset for Vehicle Accident Detection from Traffic Surveillance Videos