AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models

Zijin Zhou; Songan Zhang

cs.CLApr 11, 2026

AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models

Authors: Zijin Zhou, Songan Zhang

Organizations: Global Institute of Future Technology, Shanghai Jiao Tong University

Abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable progress in Traffic Accident Detection (TAD) and Traffic Accident Understanding (TAU). However, existing studies mainly focus on describing and interpreting accident videos, leaving room for deeper causal reasoning and integration of legal knowledge. Traffic Accident Responsibility Allocation (TARA) is a more challenging task that requires multi-step reasoning grounded in traffic regulations. To address this, we introduce AITP (Artificial Intelligence Traffic Police), a multimodal large language model for responsibility reasoning and allocation. AITP enhances reasoning via a Multimodal Chain-of-Thought (MCoT) mechanism and integrates legal knowledge through Retrieval-Augmented Generation (RAG). We further present DecaTARA, a decathlon-style benchmark unifying ten interrelated traffic accident reasoning tasks with 67,941 annotated videos and 195,821 question-answer pairs. Extensive experiments show that AITP achieves state-of-the-art performance across responsibility allocation, TAD, and TAU tasks, establishing a new paradigm for reasoning-driven multimodal traffic analysis.

Figures & tables

Figure 1: One example to illustrate the limitations of general MLLM in TARA. In the scenario pedestrian was hit while crossing the road. Qwen3-VL fail to detect this issue.

Figure 2: DecaTARA comprises ten tasks: Tasks 1-4 focus on traffic accident detection, Tasks 5-7 are about traffic accident understanding, Tasks 8-9 cover vehicle behavior and cause reasoning in non-accident scenarios, and Task 10 concerns traffic accident responsibility allocation.

Source	Videos	Test Videos	Q/A Pairs
AV-TAU	29,865	3,000	89,595
MM-AU	11,730	1,254	46,920
BDDX	26,526	2,000	53,052
TARA Dataset	–	–	6,254
DecaTARA	67,941	6,254	195,821

Table 1: Composition of the DecaTARA benchmark. DecaTARA adds responsibility labels to the test splits of existing datasets (no new videos are added).

Figure 3: Overview of the TARA inference pipeline. The model first performs Multimodal Chain-of-Thought (MCoT) reasoning, sequentially completing Traffic Accident Detection (TAD) and Traffic Accident Understanding (TAU). In time location, the model predicts the accident frame index and extracts the corresponding video frames for evidence grounding. During the final TARA stage, the model retrieves relevant legal articles through the RAG module.

Task	Metric	General MLLMs				Traffic-specific
		Gemma-3n-E4B(8B)	Qwen3-VL (8B)	Kimi-VL-A3B(16B)	InternVL3.5(8B)	AITP (8B)
	Accident	0.0755	0.1895	0.0637	0.3377	0.7218
TARA	Non-Accident	0.9849	0.9937	0.9727	0.9795	0.8117
	Average	0.4303	0.4956	0.4183	0.5881	0.7569
	BLEU	0.0301	0.0301	0.0219	0.0354	0.0395
	ROUGE-L	0.1727	0.1732	0.1433	0.1618	0.2027

Table 2: Comparison of General MLLMs and Traffic-specific MLLMs across all AITP tasks. Bold values indicate the best results.

Model	Traffic regulations recall
Qwen3-VL	A. Pedestrian takes main responsibility. Pedestrians should pay attention to their surroundings.
AITP with RAG	A. Pedestrian takes main responsibility. Traffic regulations: According to Article 62, pedestrians shall use pedestrian crossings or crosswalks when crossing intersections or roads; when crossing intersections without traffic lights or pedestrian crossings, or when crossing roads in sections without crosswalks, they shall proceed only after confirming it is safe to do so…

Table 3: Qualitative evaluation of the AITP framework.

Training Strategy
Method	Accident	Non-Accident	Average
Qwen3-VL (AITP_stage0)	0.1895	0.9937	0.4956
AITP_stage1	0.3558	0.9380	0.5830
AITP_stage2	0.5206	0.8634	0.6568
AITP_mixtrained	0.6086	0.8019	0.6841
AITP w/o CoT (AITP_stage3)	0.7218	0.7771	0.7434

Table 4: Ablation results on progressive training and inference strategies, including the effects of DecaTARA fine-tuning, MCoT, and RAG.

Explore similar work

TrafficRAG: A Multimodal RAG Framework for Traffic Accident Liability Determination
Jun 1, 2026Xu Li, Zedong Fu, Xinyi Li +1Multimodal RAG Intelligent Transportation Systems
Southwest Petroleum University, China · Sichuan Police College, China
Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models
Jul 3, 2026Ryosei Tamura, Andrew ShinMultimodal Large Language Models Egocentric Video Understanding
Keio University
UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations
Aug 13, 2026Peng Li, Qianqian Xu, Shilong Bao +2Pedestrian Intention Prediction Intelligent Transportation Systems
School of Computer Science and Technology, University of Chinese Academy of Sciences (UCAS), Beijing, China · State Key Laboratory of AI Safety, Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), Beijing, China · Beijing Academy of Artificial Intelligence (BAAI), Beijing, China +1

Jun 1, 2026J/K move · Enter open · S save

TrafficRAG: A Multimodal RAG Framework for Traffic Accident Liability Determination

Xu Li, Zedong Fu, Xinyi Li, Xun Han

Southwest Petroleum University, China · Sichuan Police College, China

Multimodal RAG Intelligent Transportation Systems Legal QA Legal Reasoning

Open paper AskPDF arXiv

Traffic accident liability analysis is a critical yet challenging task in intelligent transportation and legal assistance. Existing methods often suffer from low efficiency, subjective judgment, and inconsistent analysis results. Meanwhile, large language models are constrained by noisy video inputs and insufficient legal domain knowledge. To address these issues, this work presents TrafficRAG, a multimodal retrieval-augmented framework for automated traffic accident analysis and report generation. Specifically, the proposed framework first adopts a vision-language model to produce structured textual descriptions of accident scenarios, which serve as accurate retrieval queries. Based on these textual queries, a hybrid retrieval strategy integrating BM25 sparse retrieval and dense embedding retrieval is employed to fetch relevant traffic regulations and similar historical cases. Finally, the large language model incorporates retrieved legal knowledge and multimodal accident evidence for comprehensive reasoning, and generates standardized, legally grounded liability analysis reports. Extensive experiments show that TrafficRAG consistently outperforms baseline methods, achieving 77.32% Legal Norm Adaptation Accuracy, 81.71% Factual Faithfulness, and a Liability Ratio MAE of 5.48%. The results validate that integrating multimodal factual evidence with legal clauses via retrieval augmentation can effectively improve the reliability and accuracy of traffic accident liability determination.

Abstract

Explore similar work

TrafficRAG: A Multimodal RAG Framework for Traffic Accident Liability Determination

Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models

UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations