ExpressMind: A Multimodal Pretrained Large Language Model for Expressway Operation

Zihe Wang; Yihuan Wang; Haiyang Yu. Zhiyong Cui; Xiaojian Liao; Chengcheng Wang; Yonglin Tian; Yongxin Tong

cs.AIMar 17, 2026

ExpressMind: A Multimodal Pretrained Large Language Model for Expressway Operation

Authors: Zihe Wang, Yihuan Wang, Haiyang Yu. Zhiyong Cui, Xiaojian Liao, Chengcheng Wang, Yonglin Tian, Yongxin Tong

Organizations: Beihang University Beijing, China · Shandong Hi-speed Group Co., Ltd Jinan, China · Institute of automation, Chinese Academy of Sciences Beijing, China

Abstract

The current expressway operation relies on rule-based and isolated models, which limits the ability to jointly analyze knowledge across different systems. Meanwhile, Large Language Models (LLMs) are increasingly applied in intelligent transportation, advancing traffic models from algorithmic to cognitive intelligence. However, general LLMs are unable to effectively understand the regulations and causal relationships of events in unconventional scenarios in the expressway field. Therefore, this paper constructs a pre-trained multimodal large language model (MLLM) for expressways, ExpressMind, which serves as the cognitive core for intelligent expressway operations. This paper constructs the industry's first full-stack expressway dataset, encompassing traffic knowledge texts, emergency reasoning chains, and annotated video events to overcome data scarcity. This paper proposes a dual-layer LLM pre-training paradigm based on self-supervised training and unsupervised learning. Additionally, this study introduces a Graph-Augmented RAG framework to dynamically index the expressway knowledge base. To enhance reasoning for expressway incident response strategies, we develop a RL-aligned Chain-of-Thought (RL-CoT) mechanism that enforces consistency between model reasoning and expert problem-solving heuristics for incident handling. Finally, ExpressMind integrates a cross-modal encoder to align the dynamic feature sequences under the visual and textual channels, enabling it to understand traffic scenes in both video and image modalities. Extensive experiments on our newly released multi-modal expressway benchmark demonstrate that ExpressMind comprehensively outperforms existing baselines in event detection, safety response generation, and complex traffic analysis. The code and data are available at: https://wanderhee.github.io/ExpressMind/.

Figures & tables

Figure 2. The Overall Framework of ExpressMind. The architectural diagram of ExpressMind showing three key components: Adaptive Visual Encoding, RL-Enhanced CoT Reasoning, and Domain Knowledge Integration.

Figure 3. Task-oriented Domain Data Profiling.

Figure 4. Schematic of the RL-based Reasoning Enhancement.

Figure 5. Multimodal Encoding Framework.

Expressway Laws & Regulations QA
Models	MCQ [%]			True/False [%]			Fill-in-the-Blank [%]		Short Answer [%]
Models	Acc	Rec	F1	Acc	Rec	F1	Emb	F1	GPT-Score	F1
Qwen-32B	97.9	96.5	97.2	96.1	95.8	96.4	93.7	85.3	75.6	82.1
Baichuan-32B	91.3	88.5	89.9	89.5	88.2	89.4	88.8	78.2	69.3	74.5
DeepSeek-R1-Distill-Qwen-32B	96.1	96.3	97.4	96.5	96.0	96.1	92.4	84.1	76.6	82.4
DeepSeek-R1-Distill-Llama-70B	97.0	96.2	96.5	97.0	95.8	96.4	95.9	88.7	85.9	87.8

Table 1. Pre-training results on QA tasks.

Figure 6. Performance of ExpressMind with RL Alignment.

Figure 7. The Reasoning Time of the RL Alignment.

Figure 8. An Example of Incident Response Strategy Generation.

Figure 9. The Ablation Experiment of RAG.

Panel of Figure 10 — Figure 10. Results of Traffic Incident Detection.

Panel of Figure 11 — Figure 11. Results of Traffic Incident Detection.

Model	LLM	BLEU-4	ROUGE-L	CIDEr	BERTScore
MoTIF	LLaMA2-7B	82.59	87.45	69.95	88.43
VideoLLaMA3	Qwen 2.5-7B	79.63	85.32	67.42	85.33
MiniCPM-V 4.5	LLaMA 3-8B	81.98	87.13	69.76	88.04
InternVL3.5	Qwen 3-38B	84.53	88.94	71.67	88.97
Qwen3-VL	Qwen 3-32B	84.85	89.30	72.96	89.18
ExpressMind-VL	ExpressMind-14B	85.24	89.25	73.36	89.28

Table 2. Automated evaluation of quantitative results.

Panel of Figure 12 — Figure 12. Ablation Results of VPA.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Pre-training Hyperparameters
Parameter Name	Value
Model Architecture	Qwen-14B
DeepSpeed Strategy	ZeRO Stage 3
Precision	bfloat16
Max Sequence Length	8192
Optimizer	AdamW

Appendix

Table 3. Hyperparameter Settings for Pre-training and GRPO Algorithm

Figure 13. The System Prompt of GPT-Score.

Figure 14. The System Prompt of SFT.

Stage Index ( $k$ )	Stage Name	Control Token ( $S_{k}$ )
1	Perception	[Incident Description]
2	Analysis	[Causal Inference]
3	Decision	[Response Strategy Formulation]
4	Reflection	[Strategy Evaluation]

Appendix

Table 4. Definition of Reasoning Stage Delimiters

Stage	Representative Keywords (Translated)
$S_{1}$ : Perception	Multi-vehicle pileup, Hazardous chemical leakage, Occupying emergency lane, Visibility range, Traffic volume saturation, Fire spreading
$S_{2}$ : Analysis	Secondary accident risk, Chain reaction, Brake failure, Fatigue driving, Lane capacity reduction, Danger radius
$S_{3}$ : Decision	Remote diversion, Upstream interception, Green wave control, Air-ground coordination, Break-bulk transport, Gating control
$S_{4}$ : Evaluation	Residual congestion, Rescue efficiency, Public sentiment monitoring, Secondary damage assessment

Appendix

Table 5. Examples of Expert Vocabulary

\mathcal{V}_{k}

for Each Stage

Figure 15. Example of Textual Knowledge QA.

Figure 16. Example of the Incident CoT Structured Data.

Figure 17. Visualization of the Traffic Incident VQA Data.

Figure 18. Application of EpxressMind-VL.

Intelligent Transportation Systems

Intelligent Transportation Systems apply sensing, communication, AI, and optimization to make transport networks safer and more efficient.

See all 299 papers

TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation
Apr 29, 2026Han Gong, Zhen Zhou, Yunyang Shi +4LLM Evaluation Intelligent Transportation Systems
School of Transportation Southeast University Nanjing, Jiangsu 211189 · School of Artificial Intelligence and Computer Science Jiangnan University Wuxi, Jiangsu 214122 · Department of Civil and Environmental Engineering Hong Kong Polytechnic University Kowloon, Hong Kong
AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models
Apr 11, 2026Zijin Zhou, Songan ZhangMultimodal Large Language Models
Global Institute of Future Technology, Shanghai Jiao Tong University
OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning
Jun 14, 2026Maonan Wang, Zhengyan Huang, Kemou Jiang +13Autonomous Driving Datasets Spatiotemporal Reasoning
The Chinese University of Hong Kong, Shenzhen · Shanghai AI Lab · Beihang University +3

Apr 29, 2026J/K move · Enter open · S save

TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation

Han Gong, Zhen Zhou, Yunyang Shi, Yan Tan+3

School of Transportation Southeast University Nanjing, Jiangsu 211189 · School of Artificial Intelligence and Computer Science Jiangnan University Wuxi, Jiangsu 214122 · Department of Civil and Environmental Engineering Hong Kong Polytechnic University Kowloon, Hong Kong

LLM Evaluation Intelligent Transportation Systems Multimodal Model Evaluation Point Cloud Understanding

Open paper AskPDF arXiv

Large language models (LLMs) and multimodal large models (MLLMs) are increasingly used for transportation tasks such as regulation question answering, traffic management support, engineering review, and autonomous-driving scene reasoning. Yet transportation workflows are rule-intensive, computation-intensive, safety-critical, and inherently multimodal. Existing general benchmarks provide limited evidence of whether a model can apply regulations correctly, perform verifiable engineering calculations, or interpret traffic scenes reliably, while the small number of public transportation benchmarks remain narrow in scope and rarely support fine-grained diagnosis across text, images, and point-cloud data. To address this gap, we present TRIP-Evaluate, an open multimodal benchmark for large models in transportation. The benchmark organizes 837 items using a role-task-knowledge taxonomy that covers vehicle, traffic-management, traveler, and planning-and-design functions. Each item is annotated with capability, modality, and difficulty labels, enabling diagnosis from overall accuracy down to specific failure modes. The current release includes 596 text items, 198 image items, and 43 point-cloud items. TRIP-Evaluate also standardizes item construction, quality control, prompting, decoding, and scoring to improve cross-model comparability. Results on a diverse panel of models show that text-based performance is improving, but substantial weaknesses remain in multi-step engineering calculation, rule-constrained reasoning, multimodal scene understanding, and point-cloud understanding. Overall, TRIP-Evaluate provides a reproducible, diagnosable, and engineering-aligned evaluation baseline for model selection, regression testing, and safer deployment in transportation applications.

Abstract

Intelligent Transportation Systems

Explore similar work

TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation

AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models

OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning