EG-ARSA: An Expert-Grounded Open Model for Visual Road Safety Auditing in Low-Resource Settings

Md Thamed Bin Zaman Chowdhury; Moazzem Hossain

cs.CVAug 24, 2026

EG-ARSA: An Expert-Grounded Open Model for Visual Road Safety Auditing in Low-Resource Settings

Authors: Md Thamed Bin Zaman Chowdhury, Moazzem Hossain

Organizations: Department of Civil Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh

Abstract

Road traffic injuries remain a major challenge in low- and middle-income countries, where proactive road safety auditing is limited by incomplete crash records, shortages of qualified auditors, and the high cost of large-scale field inspections. To address this problem, we propose Expert-Grounded Distillation (EGD), a novel artificial intelligence framework that transfers institutional road safety expertise into a compact vision-language model for scalable visual road safety auditing. The key innovation is a quantified expert-grounding stage in which the teacher vision-language model is calibrated against authoritative field audits. Large-scale annotation is permitted only after the teacher reaches substantial agreement with expert risk assessments (Cohen's kappa = 0.74). The calibrated teacher then generates structured supervision that is distilled into an 8-billion-parameter student vision-language model using Low-Rank Adaptation and a single leakage-free prompt. We also introduce Bangladesh Road Safety Audit (BD-ARSA), the first open, expert-grounded Bangladeshi visual road safety audit dataset containing 21,947 image-audit records with near-national coverage, and Expert-Grounded Road Safety Auditor (EG-ARSA), the first vision-language model developed specifically for this task. Experimental results show that grounded fine-tuning substantially improves ordinal risk assessment over the zero-shot baseline, while blind expert evaluation demonstrates that the compact student outperforms both its 31 billion-parameter teacher and Gemini-2.5-Flash. These findings demonstrate that EGD provides an effective and scalable engineering solution for proactive road safety auditing in resource-constrained environments.

Figures & tables

Category	Definition
vision_obstruction	Sight distance blocked by trees, walls, vegetation, shops, or structures at curves and junctions.
pedestrian_facilities	Footpath, crossing, or walkway absent or broken where pedestrian movement is high.
shoulder_condition	Paved or soft shoulder absent, narrow, or ineffective; no usable verge.
roadside_severity	Poles, trees, or solid structures within the clear zone ( $\sim$ 3 ft of the pavement edge).
road_markings	Centreline, edge line, lane marking, or crossing absent, faded, or poorly applied.
intersection	Junction geometry problems: staggered or skewed legs, poor flaring, uncontrolled crossings.

Table 1: The 12 LGED visual road-safety hazard categories, with one-line definitions.

Tier	Records	Label provenance
expert_gold	343	Single-hazard crops with auditor captions read directly from report 1 (72 audited locations); no teacher prompt.
expert_silver	707	Road-scene photos from report 2, image-aligned by us to the expert hazard findings (alignment manually verified) and curated into full structured audits under human review.
streetview	20,897	Sampled road-scene imagery; full teacher-VLM audits with no expert findings, validated against the 12-category taxonomy and risk-boundary post-processed.
Total	21,947

Table 2: The three provenance tiers of BD-ARSA: record counts and label provenance.

By source tier
	Train	Val	Test	Total
expert_gold	107	78	158	343
expert_silver	507	96	104	707
streetview	15,468	2,244	3,185	20,897
All sources	16,082	2,418	3,447	21,947
By overall-risk level

Table 3: Location-disjoint splits by source tier and the overall-risk distribution per split.

Figure 1: Three-tier data provenance of BD-ARSA. Expert field audits from the ARI–BUET/LGED programme yield the gold (individually captioned hazard crops) and silver (road scenes we aligned to the expert findings and curated under human review) tiers; the street-view tier is teacher-audited road-scene imagery whose generation prompt was calibrated on the expert audits, with no per-image expert findings.

Figure 2: Nationwide street-view coverage. The 155 sampled road corridors span 63 districts across all 8 administrative divisions of Bangladesh. The expert-grounded gold and silver tiers are drawn from the 18 LGED-audited districts within this footprint.

Figure 3: A sample of BD-ARSA imagery, mixing gold single-hazard crops with street-view and silver road scenes across hazard categories and risk levels.

Property	Value
Records (image–audit)	21,947 (343 expert-gold / 707 expert-silver / 20,897 street-view)
Hazard instances (total)	92,347
Mean hazards per street-view scene	4.22 (range 3–9)
Hazard categories	12 (LGED schema)
Category entropy (normalised)	0.79 (of 1.0)
Category Gini coefficient (concentration)	0.53

Table 4: BD-ARSA corpus summary statistics (released dataset; metadata computed over all 21,947 records).

Figure 4: Distributional structure of BD-ARSA. (a) Hazard-category prevalence (log scale); (b) category composition within each provenance tier; (c) severity composition per category (Cramér’s $V=0.68$ ); (d) number of hazards per street-view scene (mean 4.2).

Figure 5: Hazard-category co-occurrence within records (Jaccard index). The four common categories form a tightly coupled cluster (up to 0.84); the rare categories are largely independent.

Figure 6: Hazard prevalence as a base-rate prior. (a) Lorenz curve of the 12-category instance distribution (Gini 0.53). (b) Per-category prevalence coloured by visual inferability; the two partly-non-visual categories ( drainage , skid_resistance ) — the model’s hardest set — sit at mid-range frequency, decoupling category difficulty from class scarcity.

Figure 7: The Expert-Grounded Distillation pipeline. Left: grounded data generation across the three tiers, curated into BD-ARSA (summarised here; full provenance in Fig. 1 ). Middle: the student architecture — a frozen vision encoder and projector feed image tokens, alongside text tokens, to the Qwen3-VL-8B-Instruct backbone, which is adapted with LoRA on the attention projections and trained with the multi-task objective and train-only logit adjustment. Right: single-image, leakage-free inference.

Figure 8: The student prompt (left) and the structured JSON audit target (right), with a worked street-view example. The prompt instructs the model to audit based only on what is visible in the image; the target lists hazards with an LGED category and severity, an ordinal overall risk, and a recommendation.

Component	Setting
Base model	Qwen3-VL-8B-Instruct (vision encoder frozen)
Adaptation	LoRA, $r=16$ , $\alpha=32$ , dropout 0.05, on $q,k,v,o\_proj$
Precision	bf16 + gradient checkpointing
Image resolution	1024 px (native max dimension)
Optimiser schedule	LR $1\times 10^{-4}$ , cosine, 3% warmup
Effective batch	16 (micro 2 $\times$ accumulation 8)

Table 5: Training configuration.

Figure 9: Training dynamics over the full run: per-task losses (hazard, risk, recommendation) and validation QWK against training steps, with the selected checkpoint marked. All task losses decrease and validation QWK rises across the two epochs.

Model / setting	QWK	Linear $\kappa$	Macro-F1	Accuracy
Zero-shot base (Qwen3-VL-8B)	0.0772 [0.0440, 0.1082]	0.0831	0.3585	0.5440
Fine-tuned (raw)	0.4815 [0.4538, 0.5097]	0.4562	0.4933	0.7171

Table 6: Overall-risk prediction on the test set (

n=3{,}447

): the fine-tuned student versus the zero-shot base. Brackets give bootstrap 95% confidence intervals.

Risk class	Precision	Recall	F1	Support
Low	0.667	0.020	0.038	102
Medium	0.632	0.714	0.671	1,381
High	0.787	0.756	0.771	1,964

Table 7: Per-class precision/recall/F1 for the fine-tuned student (raw operating point) on the test set.

Figure 10: Risk QWK and exact-risk accuracy for the zero-shot Qwen3-VL-8B-Instruct base versus the fine-tuned student, visualising the

+0.40

QWK gain from Expert-Grounded fine-tuning.

Figure 11: Overall-risk confusion matrices for the fine-tuned student: raw operating point (left) and $\delta$ -adjusted (right). Errors are predominantly between adjacent risk levels; the $\delta$ -adjusted matrix recovers Low recall at a small ordinal cost.

	Risk accuracy			Silver hazard R/P
Model	All	Gold	Silver	(recall / precision)
EG-ARSA (8B student)	0.737	0.753	0.712	0.65 / 0.85
Gemini-2.5-Flash	0.592	0.544	0.663	0.79 / 0.77
Teacher (31B, leakage-free)	0.359	0.266	0.500	0.72 / 0.86

Table 8: Multi-model comparison on the 262 expert-grounded entries, under identical single-image, leakage-free prompts. Risk accuracy is reported overall and split by source; silver hazard-category recall/precision is also shown.

Blind three-model comparison ( $n=80$ )
Model	Overall (1–5)	Risk-correct
EG-ARSA (8B student)	3.94	0.81 [0.73, 0.90]
Gemini-2.5-Flash	3.65	0.58
Teacher (31B, leakage-free)	3.46	0.42
EG-ARSA full expert-grounded evaluation ( $n=262$ )
Issue correctness	4.02

Table 9: Blind human evaluation. Top: the 80-item blind three-model comparison (overall quality on a 1–5 scale and binary risk-correct). Middle: the student’s full expert-grounded evaluation (

n=262

). Bottom: the cross-validation contrasting blind-human risk-correct with independent automated risk accuracy.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Comparison	Value
Short labels — exact match (top-1 vs. top-1)	40.4%
Short labels — top-2 inclusion	60.6%
Short labels — Cohen’s kappa	0.331
Finding texts — mean Jaccard (multi-label)	0.373
Finding texts — perfect agreement	12.3%
Finding texts — low agreement (Jaccard $<$ 0.5)	61.5%

Appendix

Table 10: Inter-classifier agreement between the two zero-shot label generators (Gemini-2.5-Flash vs. BART-large-MNLI).

Panel of Figure 12 — Figure 12: Word clouds of the expert corpus: (top) the detailed per-location findings layer, and (bottom) the independently-written auditor summary layer.

Category	Findings-layer hits	Summary-layer hits
vision_obstruction	1822	72
roadside_severity	1300	141
intersection	1212	176
shoulder_condition	1037	113
embankment_safety	915	154
speed_management	890	61

Appendix

Table 11: Category keyword signal: keyword hits per hazard category in the detailed-findings layer and in the auditor-summary layer. Both independent layers cover all 12 categories.

Explore similar work

Jun 29, 2026

From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA

High benchmark accuracy does not guarantee genuine use of visual evidence. We study this problem in traffic accident Video Question Answering (VideoQA), where correct answers should depend on scene-specific visual evidence but may instead be inferred from textual shortcuts. Through an audit of four public benchmarks, we find that several recent open-weight Vision-Language Models (VLMs) perform competitively, and sometimes better, without video input. On the MM-AU benchmark, removing video consistently improves accuracy, and adding more frames further degrades performance. To quantify visual dependence, we introduce two dataset-level diagnostics: Blind Gap, measuring above-chance text-only performance, and Visual Gain, measuring the marginal benefit of adding video. We further propose an instance-level Shortcut Score that combines text-only confidence with visual necessity signals, enabling continuous, training-free filtering of shortcut-prone questions. The resulting subsets reduce shortcut bias and improve visual grounding. Our findings reveal large differences in grounding quality across benchmarks and show that visually grounded evaluation, not just high accuracy, is essential in safety-critical VideoQA.

Sena Korkut, María Alejandra Bravo Sarmiento, Sanghwan Kim +1

Technical University of Munich, Munich, Germany · Helmholtz Zentrum München, Munich, Germany · Munich Center for Machine Learning, Munich, Germany

VLM Evaluation Vision-Language Models Video QA Visual Grounding

Ask PDF

Apr 9, 2026

CrashSight: A Phase-Aware, Infrastructure-Centric Video Benchmark for Traffic Crash Scene Understanding and Reasoning

Cooperative autonomous driving requires traffic scene understanding from both vehicle and infrastructure perspectives. While vision-language models (VLMs) show strong general reasoning capabilities, their performance in safety-critical traffic scenarios remains insufficiently evaluated due to the ego-vehicle focus of existing benchmarks. To bridge this gap, we present CrashSight, a large-scale vision-language benchmark for roadway crash understanding using real-world roadside camera data. The dataset comprises 250 crash videos, annotated with 13K multiple-choice question-answer pairs organized under a two-tier taxonomy. Tier 1 evaluates the visual grounding of scene context and involved parties, while Tier 2 probes higher-level reasoning, including crash mechanics, causal attribution, temporal progression, and post-crash outcomes. We benchmark 8 state-of-the-art VLMs and show that, despite strong scene description capabilities, current models struggle with temporal and causal reasoning in safety-critical scenarios. We provide a detailed analysis of failure scenarios and discuss directions for improving VLM crash understanding. The benchmark provides a standardized evaluation framework for infrastructure-assisted perception in cooperative autonomous driving. The CrashSight benchmark, including the full dataset and code, is accessible at https://mcgrche.github.io/crashsight.

Rui Gan, Junyi Ma, Pei Li +4

University of Wisconsin–Madison · University of Wyoming · Columbia University

Autonomous Driving Benchmarks Autonomous Driving Safety Evaluation Autonomous Driving Temporal Video Understanding

Ask PDF

May 19, 2026

WildRoadBench: A Wild Aerial Road-Damage Grounding Benchmark for Vision-Language Models and Autonomous Agents

We introduce WildRoadBench, a wild aerial road-damage grounding benchmark that couples direct visual grounding by vision-language models with autonomous research-and-engineering by LLM-driven agents on a single professionally annotated UAV corpus. The same image set and the same per-class AP_50 metric are evaluated under two protocols. The VLM Track measures whether a fixed VLM can localise domain-specific damage from one image and one short prompt under a unified prompting, decoding and parsing pipeline. The Agent Track measures whether an autonomous agent, given only a written task brief, a small exploratory slice and a fixed interaction budget, can search the public web, adapt pretrained components, write training and inference code, and submit predictions through a scalar-feedback oracle on a hidden holdout. We benchmark a broad pool of closed-source frontier models and open-source VLMs together with several frontier LLM-driven agents. Both routes remain far from reliable performance in this wild setting: closed-source frontier models lead the VLM leaderboard but still leave more than half of the metric on the table; open-source grounders plateau well below them, and newer generations or reasoning-style variants do not consistently improve grounding; small targets collapse for every open-source model; agents lag the strongest VLM despite richer affordances, and several fail to land a valid submission within the budget. We release the code and data at https://anonymous.4open.science/r/wildroadbench-0607 to support reproducible follow-up research.

Bingnan Liu, Chenhang Cui, Rui Huang +7

University of Electronic Science and Technology of China · National University of Singapore · De Artificial Intelligence Lab +2

VLM Evaluation AI Agent Benchmarks Vision-Language Grounding

Ask PDF