CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving
Authors: Zhe Huang, Zhaoxin Fan, Shuo Wang, Wenjun Wu, Xuan Zhao, Min Liu
Organizations: School of future Transportation, Chang’an University, Xi’an, China · Hangzhou International Innovation Institute, Beihang University, Beijing, China · School of information, Renmin University of China, Beijing, China · School of Automobile, Chang’an University, Xi’an, China · School of Artificial Intelligence and Robotics, Hunan University, and the National Engineering Research Center for Robot Visual Perception and Control Technology, Changsha, China
Multi-vehicle cooperative autonomous driving enhances the safety and reliability of autonomous driving systems through information sharing among connected vehicles, demonstrating significant potential for improving traffic safety. LLM-based approaches leverage strong reasoning capabilities of LLMs to enable effective inter-vehicle negotiation and improve cooperative driving performance. However, driving decisions in complex traffic scenarios are inherently multi-solution in nature. As a result, existing negotiation-based methods often converge prematurely to suboptimal solutions, hindering consensus formation and limiting the practical deployment of cooperative autonomous driving systems. To address this challenge, we propose CoLMIN, the LLM-based multi-decision path negotiation framework for cooperative autonomous driving, achieving stable decision consensus through multi-decision path negotiation and reflective reasoning. To achieve stable and high-quality consensus in cooperative autonomous driving, CoLMIN consists of three key components: (i) an LLM-based Multi-Intent Negotiation module (LMin), which adopts a Negotiator-Evaluator paradigm and generates multiple candidate driving intentions for joint evaluation; (ii) an Evaluation-based Shallow Reflection Module (ESRM), which analyzes negotiation outcomes and provides feedback to guide subsequent negotiations, thereby accelerating consensus formation; and (iii) an LLM-based Deep Reflection Module (LDRM), which performs long-term reflection over negotiation histories to mitigate cognitive fixation and prevent the system from converging to suboptimal solutions. Experimental results in the CARLA simulation environment demonstrate that CoLMIN significantly outperforms existing methods in challenging interactive driving scenarios.
Figures & tables
Fig. 1: Consensus quality under different negotiation paradigms. CoLMIN adopts a multi-decision path based negotiation paradigm to generate and evaluate negotiation outcomes, achieving optimal and stable consensus.
Fig. 2: Overall architecture of CoLMIN. The framework consists of the perception module and three negotiation-related modules. The perception module converts raw sensor data into environmental information for intention detection. When intention conflicts are detected, LMin performs multi-vehicle negotiation. ESRM provides short-term refinement to accelerate consensus, while LDRM uses long-term reflection to mitigate overconfident or repetitive decisions.
Fig. 3: The detailed reasoning workflows of the ESRM Module and the LDRM Module.
Without Traffic Participants
Total
IC
LM
LC
Method
DS↑
RC↑
IS↑
SR↑
DS↑
RC↑
IS↑
SR↑
DS↑
RC↑
IS↑
SR↑
DS↑
RC↑
IS↑
SR↑
VAD (CVPR2021) [ 51 ]
25.37
75.00
0.33
0.02
15.49
54.72
0.29
0.00
37.24
92.85
0.40
0.05
17.93
76.00
0.26
0.00
UniAD (CVPR2023) [ 52 ]
35.17
88.30
0.38
0.11
37.24
91.63
0.41
0.11
42.50
84.41
0.47
0.15
12.19
90.57
0.12
0.00
TCP (NeurIPS2022) [ 20 ]
73.68
90.54
0.82
0.50
77.64
82.83
0.94
0.50
82.18
95.18
0.86
0.70
43.52
96.30
0.45
0.00
LMDriver (CVPR2024) [ 3 ]
48.83
58.02
0.85
0.20
44.72
57.94
0.79
0.17
60.88
69.43
0.86
0.30
27.96
29.70
0.95
0.00
TABLE I: Driving performance in V2Xverse simulator. Red denotes the best and black the second-best or the same results. CoLMIN performs excellently across all scenarios, especially in driving score.
With Traffic Participants
Method
Total
IC
LM
LC
DS↑
RC↑
IS↑
SR↑
DS↑
RC↑
IS↑
SR↑
DS↑
RC↑
IS↑
SR↑
DS↑
RC↑
IS↑
SR↑
VAD (CVPR2021) [ 53 ]
25.18
75.66
0.31
0.02
22.13
61.31
0.32
0.00
35.10
88.23
0.39
0.05
7.24
76.56
0.09
0.00
UniAD (CVPR2023) [ 52 ]
37.13
88.71
0.41
0.11
37.45
83.52
0.44
0.06
48.29
91.33
0.52
0.20
8.48
93.82
0.09
0.00
TCP (NeurIPS2022) [ 20 ]
74.18
91.21
0.82
0.48
76.26
84.62
0.91
0.44
86.59
95.00
0.91
0.65
38.50
96.56
0.40
0.13
LMDriver (CVPR2024) [ 3 ]
49.95
61.61
0.84
0.13
47.65
59.34
0.81
0.00
54.12
67.10
0.85
0.20
44.69
53.02
0.87
0.25
TABLE II: Driving performance in V2Xverse simulator with traffic participants. CoLMIN consistently achieves the best performance, demonstrating strong robustness.
Single
LMin
ESRM
LDRM
DS
RC
IS
SR
47.64
96.43
0.485
0.13
✓
70.22
87.03
0.77
0.54
✓
✓
77.29
95.22
0.78
0.65
✓
✓
✓
90.02
94.72
0.90
0.78
✓
✓
✓
✓
90.12
94.36
0.92
0.80
TABLE III: Driving performance on the V2Xverse simulation platform. Red denotes the best and black the second-best results.
Fig. 4: Comparison of driving performance with vs. without latency.
Fig. 5: Visualization of CoLMIN in the Cross Change scenario. Four vehicles negotiate based on their individual states and execute a feasible joint action plan. Context slightly paraphrased for clarity. CoLMIN derives actions that are consistent with the current traffic context while ensuring collision-free coordination.
Fig. 6: Comparison of Negotiation Convergence Under Single-Decision path based and Multi-Decision path based Paradigms.
Variant
LMin
ESRM
LDRM
Driving Score (LC)
CoLMDriver-Qwen2.5-3B
–
–
–
59.21
CoLMDriver-Qwen2.5-7B
–
–
–
59.15
CoLMIN w/o LDRM
3B
3B
–
62.30
CoLMIN w/ LDRM-Qwen2.5-3B
3B
3B
3B
66.00
CoLMIN w/ LDRM-Qwen2.5-7B
3B
3B
7B
67.20
TABLE IV: Backbone controlled ablation study.
Inference Time (s)
Total↓
IC↓
LM↓
LC↓
Colmdriver [ 25 ]
31.74
32.93
26.80
41.44
CoLMIN
29.67
31.26
24.97
37.83
TABLE V: Average total inference time in the closed-loop setting.
Failure Type
Driving Score
Route Score
Penalty
Main Infraction
Collision failure
60.0
100.0
0.6
Vehicle collision
Timeout failure
3.988
3.988
1.0
Route timeout
Partial timeout
66.338
66.338
1.0
Route timeout
Timeout+stop
53.959
67.449
0.8
Timeout+stop
TABLE VI: Representative failure cases from closed-loop evaluation.
Fig. 7: Visualization experiment of failure cases.
Candidate Intents
DS
RC
IS
SR
Inference Time (s)
1
52.86
79.89
0.63
0.2
37.1
2
66.87
79.67
0.87
0.3
30.4
3
80.07
86.64
0.80
0.8
34.2
4
80.25
83.15
0.81
0.8
33.0
TABLE VII: Effect of number of candidate intents on performance and efficiency.
Method
Overconf. ↓
Rounds ↓
Conv. Rate ↑
Nego. Time
Tokens ↓
Baseline
2.08
2.52
33.90%
8.06s
4487
Ours
1.37
1.90
86.25%
9.51s
5053
TABLE VIII: Process-level analysis of negotiation convergence and efficiency, averaged over the evaluated negotiation samples. “Overconf.”, “Conv.”, and “Nego.” denote overconfidence, convergence, and negotiation, respectively.
Criterion
LMin Score
Objective Ref. Score
Abs. Gap ↓
Safety
0.928
0.959
0.031
Efficiency
0.198
0.155
0.043
Consensus
0.851
0.910
0.059
TABLE IX: Validation of the LLM-based evaluator on the final negotiation plans. LMin Score denotes the average score produced by the evaluator; Objective Ref. Score denotes the actual rollout score; Abs. Gap denotes their absolute difference.
Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.
L. Parada, K. Yu, and P. Angeloudis are with the Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, UK