Perceived risk in driving evolves over time and may be supported by specific scene entities, yet supervision is typically limited to coarse video-level judgments. Learning when supporting evidence emerges and which entities support a risk predictor would ordinarily require costly temporal- and entity-level annotations. We introduce CoRE, a weakly supervised coarse-to-fine framework that learns fine-grained prediction support from coarse video supervision. CoRE first trains a video-level predictor and then freezes it. Structured interventions over candidate temporal regions or entity tracks measure how each candidate changes the coarse prediction, producing graded prediction-effect targets. These targets are distilled into a student that directly predicts temporal and entity support from the original video, without requiring interventions at inference. We evaluate this learning principle across three complementary settings: RISEE tests perceived-risk support from subjective clip-level judgments without temporal or entity-level risk annotations; DoTA provides independent temporal event annotations for evaluating weakly supervised traffic-anomaly localization; and UCF-Crime tests whether the same coarse-to-fine mechanism extends to a standard non-driving anomaly-detection benchmark. Across these settings, CoRE learns informative fine-grained support from coarse supervision, with strong temporal localization on DoTA and competitive performance on UCF-Crime. These results show that coarse video predictions can provide useful supervision for recovering the fine-grained evidence supporting them, without requiring corresponding fine-grained labels.
Figures & tables
Figure 1: CoRE decomposes coarse video-level perceived risk into fine-grained support, revealing how risk evolves over time, what visual evidence drives the prediction, and how strongly individual objects contribute.
Figure 2: Overview of CoRE. A coarse video predictor is trained from video-level supervision and frozen. Structured interventions over temporal and entity candidates produce prediction effects that are converted into graded support targets and distilled into a student. At inference, the student directly predicts the coarse response together with temporal and entity support in a single forward pass.
Method
MAE ↓
RMSE ↓
Spearman ρ↑
PairAcc ↑
Frame-average regression
0.640
0.788
0.581
0.721
Attention MIL [ 14 ]
0.577
0.709
0.653
0.750
Soft top- k MIL [ 34 ]
0.683
0.815
0.575
0.713
Temporal convolution
0.649
0.808
0.572
0.710
Temporal Transformer
0.659
0.801
0.552
0.703
Video Swin regression [ 19 ]
0.592
0.718
0.641
0.742
Table 1: Clip-level perceived-risk prediction on RISEE. Scenario-level five-fold evaluation using only human clip-level ratings. Best per metric in bold , second best underlined .
Method
Sel. Drop ↑
Gain/Rand. ↑
Effect ρ↑
Top Perc. ↑
Pos. Rate ↑
Uniform weighting
0.045
-0.049
0.000
0.538
0.726
Attention MIL [ 14 ]
0.032
-0.050
-0.167
0.482
0.620
Soft top- k MIL [ 34 ]
0.050
-0.038
-0.214
0.474
0.592
CoRE (ours)
0.320
0.245
0.542
0.888
0.989
Table 2: Temporal prediction support on RISEE. Held-out interventions measure prediction dependence; Effect ρ measures agreement with candidate effects, while Top Perc. and Pos. Rate summarize selection quality.
Table 3: Weakly supervised temporal anomaly localization on DoTA. All results use the same WS-DoTA split and evaluation pipeline. Methods within each block share the same visual feature bank. † denotes a protocol-matched adaptation based on released official code; ‡ denotes our reimplementation from the paper/supplement when executable official code was unavailable; § denotes our controlled baseline. Best in bold ; second best underlined .
Figure 3: Qualitative temporal localization on UCF-Crime. Each example shows video frames, the frame-level anomaly interval (pink), and the temporal scores produced by CoRE and MGFN [ 5 ] . Red borders identify displayed frames within the annotated interval.
Figure 4: Qualitative temporal localization on DoTA. Ground-truth intervals are compared with OE-CTST [ 23 ] , PE-MIL [ 4 ] , and CoRE on the same videos.
Figure 5: Temporal support on contrasting RISEE clips. CoRE rises with perceived risk and remains suppressed in the low-risk clip; t1 – t5 mark the displayed frames. Attn. MIL ; soft top- k ; CoRE .
Method
Venue
Feature
AUC (%) ↑
MIL-Rank [ 34 ]
CVPR’18
C3D RGB
75.41
MIST [ 8 ]
CVPR’21
I3D RGB
82.30
RTFM † [ 37 ]
ICCV’21
I3D RGB
84.30
NL-MIL [ 27 ]
WACV’23
I3D RGB
85.63
MGFN † [ 5 ]
AAAI’23
I3D RGB
82.79
PE-MIL [ 4 ]
CVPR’24
I3D RGB
86.83
Table 4: Weakly supervised anomaly detection on UCF-Crime. Frame-level AUC (%). We reproduce all baselines for which official code is publicly available ( † ); remaining methods are listed with their originally reported numbers. Best in bold , second best underlined .
Configuration
AUC ↑
AP ↑
F1 ↑
tIoU ↑
w/o effect supervision
0.419
0.264
0.139
0.329
w/o bag objective
0.754
0.518
0.309
0.335
w/o soft targets
0.453
0.283
0.133
0.326
w/o multi-scale candidates
0.504
0.314
0.040
0.098
Full CoRE
0.744
0.514
0.364
0.429
Table 5: Component ablation on DoTA. Each row removes one component of CoRE ; the full configuration is shown in the last row. Best per column in bold .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
RISEE [ 40 ]
DoTA [ 42 ]
UCF-Crime [ 34 ]
Training supervision
Perceived-risk score
Video-level normal/abnormal
Video-level normal/abnormal
Evaluation protocol
5-fold scenario CV
Released 1,140-video test split
Standard 290-video test split
Input representation
ResNet-50; object track features
ResNet-50 / CLIP ViT-B/32
Ten-crop I3D RGB
Support candidates
16 positions; tracked objects
Temporal windows {1,2,4}
32 segments; windows {1,3,5}
Default intervention
Local mean
Local mean
Video-mean replacement
Target form
Competitive support
Competitive support
Independent temporal support
Appendix
Table 6: Experimental setup for CoRE. Dense temporal or object labels are not used during training.
Support
Train → Eval
Sel. Drop ↑
Gain vs. Rand. ↑
Temporal
Mean → Mean
0.0826
0.0558
Temporal
Blur → Mean
0.0810
0.0542
Temporal
Mean → Blur
0.0706
0.0418
Temporal
Blur → Blur
0.0705
0.0417
Object
Mean → Mean
0.0615
0.0284
Object
Blur → Mean
0.0588
0.0257
Appendix
Table 7: Cross-operator robustness of CoRE support learning on RISEE. We report the selection metrics shared by temporal and object support.
Support
Effect ρ↑
Agree@1 ↑
Top-3 Jaccard ↑
Temporal
0.7344
0.6089
0.6765
Object
0.7356
0.8333
0.8512
Appendix
Table 8: Agreement between local-mean and blur intervention effects.
Teacher
MAE ↓
Spearman ↑
Sel. Drop ↑
Gain vs. Rand. ↑
Effect ρ↑
Mean Pool
0.6139
0.6966
0.8519
0.5575
0.7956
Attention-MIL
0.5389
0.7241
2.4585
1.7356
0.6028
Canonical
0.5236
0.7343
2.5256
1.6813
0.7417
Appendix
Table 9: Robustness of CoRE to the choice of coarse risk teacher on RISEE. Only the teacher used to construct intervention effects is changed.
Support
Sel. Drop ↑
Gain vs. Rand. ↑
Effect ρ↑
NDCG@3 ↑
Top Perc. ↑
Attention-MIL attention
2.3295
1.6066
0.5683
0.8211
0.8823
CoRE distilled support
2.4585
1.7356
0.6028
0.8549
0.9082
Appendix
Table 10: Direct Attention-MIL attention versus CoRE support distilled from intervention effects of the same Attention-MIL teacher.
Component
Mean (ms) ↓
p95 (ms) ↓
Peak GPU (MB) ↓
Coarse teacher head
2.0679
2.0836
82.87
Temporal student head
2.0676
2.0833
82.87
Object student head
0.5510
0.5663
175.92
ResNet-50, 16 frames
10.8651
10.8832
260.82
Offline temporal target construction
14.2064
14.2444
88.10
Naive intervention explanation
14.2172
14.2654
88.10
Appendix
Table 11: Runtime on an NVIDIA H100 NVL. Intervention-based target construction is performed offline; CoRE uses direct student inference at test time.
Achieving zero-collision mobility remains a key objective for intelligent vehicle systems, which requires understanding driver risk perception-a complex cognitive process shaped by voluntary response of the driver to external stimuli and the attentiveness of surrounding road users towards the ego-vehicle. To support progress in this area, we introduce RAID (Risk Assessment In Driving scenes)-a large-scale dataset specifically curated for research on driver risk perception and contextual risk assessment. RAID comprises 4,691 annotated video clips, covering diverse traffic scenarios with labels for driver's intended maneuver, road topology, risk situations (e.g., crossing pedestrians), driver responses, and pedestrian attentiveness. Leveraging RAID, we propose a weakly supervised risk object identification framework that models the relationship between driver's intended maneuver and responses to identify potential risk sources. Additionally, we analyze the role of pedestrian attention in estimating risk and demonstrate the value of the proposed dataset. Experimental evaluations demonstrate that our method achieves 20.6% and 23.1% performance gains over prior state-of-the-art approaches on the RAID and HDDS datasets, respectively.
Nakul Agarwal, Yi-Ting Chen, Behzad Dariush
Honda Research Institute USA, Inc. · National Yang Ming Chiao Tung University.
With the rapid growth of video centered social media, the ability to anticipate risky events from visual data is a promising direction for ensuring public safety and preventing real world accidents. Prior work has extensively studied supervised video risk assessment across domains such as driving, protests, and natural disasters. However, many existing datasets provide models with access to the full video sequence, including the accident itself, which substantially reduces the difficulty of the task. To better reflect real world conditions, we introduce a new video understanding benchmark RiskCueBench in which videos are carefully annotated to identify a risk signal clip, defined as the earliest moment that indicates a potential safety concern. Experimental results reveal a significant gap in current systems ability to interpret evolving situations and anticipate future risky events from early visual signals, highlighting important challenges for deploying video risk prediction models in practice.
Sha Luo, Yogesh Prabhu, Timothy Ossowski +2
University of Wisconsin–Madison, Madison, WI, USA · University of California San Diego, San Diego, CA, USA
Grounding traffic accidents in real CCTV footage is a rare-event problem where training on labeled accident video is often prohibited, yet accurate joint localization in time, space, and collision type is required. We present a no-fine-tuning pipeline that elicits this joint output from frozen vision-language models through two ideas. First, a coarse-to-fine two-pass decomposition: a full-video pass at 1 fps produces a coarse (t, x, y, c) tuple, then a second pass at 5 fps within a +/- 3 s window refines time and location, with two deterministic confidence gates that revert to the coarse estimate on boundary hedges or edge-clamped coordinates. Second, a specialist role assignment: Qwen3-VL-Plus handles grounding, Gemini 3.1 Flash-Lite handles typing on a centered video clip. On the ACCIDENT@CVPR 2026 benchmark (2,027 real CCTV videos) we reach ACC^S = 0.539 (95% CI [0.525, 0.553]): +0.127 over the benchmark paper's best-of-baselines oracle (0.412), +0.143 over the strongest single-VLM baseline (Molmo-7B, 0.396), and +0.250 over the naive baseline (0.289). The VLM path uses up to three API calls per video (17% fall back to physics on API failures); the full run costs ~$20.