General-purpose LLMs as Models of Human Driver Behavior: The Case of Simplified Merging

Samir H. A. Mohammad; Wouter Mooi; Arkady Zgonnikov

cs.AIMar 11, 2026

General-purpose LLMs as Models of Human Driver Behavior: The Case of Simplified Merging

Authors: Samir H. A. Mohammad, Wouter Mooi, Arkady Zgonnikov

Organizations: Department of Transport and Planning, Delft University of Technology · Department of Cognitive Robotics, Delft University of Technology, The Netherlands

Abstract

Human behavior models are essential as behavior references and for simulating human agents in virtual safety assessment of automated vehicles (AVs), yet current models face a trade-off between interpretability and flexibility. General-purpose large language models (LLMs) offer a promising alternative: a single model potentially deployable without parameter fitting across diverse scenarios. However, what LLMs can and cannot capture about human driving behavior remains poorly understood. We address this gap by embedding two general-purpose LLMs (OpenAI o3 and Google Gemini 2.5 Pro) as standalone, closed-loop driver agents in a simplified one-dimensional merging scenario and comparing their behavior against human data using quantitative and qualitative analyses. Both models reproduce human-like intermittent operational control and tactical dependencies on spatial cues. However, neither consistently captures the human response to dynamic velocity cues, and safety performance diverges sharply between models. A systematic prompt ablation study reveals that prompt components act as model-specific inductive biases that do not transfer across LLMs. These findings suggest that general-purpose LLMs could potentially serve as standalone, ready-to-use human behavior models in AV evaluation pipelines, but future research is needed to better understand their failure modes and ensure their validity as models of human driving behavior.

Figures & tables

Fig. 1: (A) The one-dimensional (1-D) merging task in the same simulated environment from [ 7 ] . Two vehicles emerge from tunnels on the branches of an inverse-Y track and approach a common merge point with no assigned right of way. Initial speeds ( $v_{L}$ , $v_{R}$ ) and projected headway at the merge are varied across 11 kinematic conditions, matching the human dataset of [ 19 ] . (B) Closed-loop LLM driver-agent framework. At each prompt time, the simulation state (panel A) is formatted into a structured prompt (components C0–C5, Table II ) and passed to a general-purpose LLM, which returns a tactical decision, an acceleration plan, and a natural-language rationale. The plan is executed under a receding horizon, after which the system re-queries the LLM with updated state. Both vehicles are controlled in parallel by independent LLM instances with mirrored observations.

Plan horizon $T$	$4.0$ s	Memory window $T_{m}$	$2.0$ s
Prompt rate $f_{p}$	$1$ Hz	Temperature $\gamma$	$1.0$ [-]
Step $\Delta t$	$0.2$ s	Vehicle length $l_{veh}$	4.5 m

TABLE I: Model and simulation parameters.

ID	Description (baseline)
C0	Observations and task context (required): road type (merge vs. straight), vehicle length, relative distance/headway, ego and other velocities, and distance-to-merge when applicable.
C1	History ( $T_{m}$ ): past ego accelerations and past observed other-vehicle velocities.
C2	Previous acceleration plan : acceleration plan returned at the previous prompt, included to encourage plan continuity under re-planning.
C3	Behavioral constraints : collision avoidance, human-likeness, conservative distance keeping, and periodic reassessment.
C4	Tactical plan : explicit merge-first vs. yield choice before generating the continuous control plan (merging scenarios only).
C5	Output schema (required): optional M/Y token, 2–3 sentence rationale, and a length- $N$ normalized acceleration array $\hat{\mathbf{u}}\in[-1,1]^{N}$ returned in a Python code block (array only).

TABLE II: Prompt components (C) and single-factor ablations (A).

Fig. 2: Baseline behavioral comparison between humans and LLM agents (collisions excluded). (A–B) Left-vehicle velocity time series after exiting the tunnel ( $t=0$ ), pooled across kinematic conditions, separated by whether the left vehicle merges first (A) or second (B) , illustrating piecewise-linear velocity profiles characteristic of intermittent acceleration control. (C) Probability that the left vehicle merges first as a function of projected headway advantage, shown separately for each initial relative-velocity condition $\Delta v\in\{-0.8,0.0,+0.8\}$ m/s. (D) Mean gap at the merge point versus projected headway advantage, per $\Delta v$ condition. (E) Mean per-vehicle RMSE of velocity deviation from the initial velocity versus projected headway advantage, per $\Delta v$ condition (each trial contributes two RMSE values: left and right vehicle). Error bars indicate 95% confidence intervals ( C : Wilson intervals for proportions; D–E : t-based intervals for means).

	b	SE	Z	p
Intercept - H	-0.30	0.135	-2.26	0.024
Intercept - o3	0.44	0.340	1.29	0.198
Intercept - G	0.08	0.301	0.26	0.792
Projected headway - H	1.11	0.075	14.89	$<.001$
Projected headway - o3	1.11	0.243	4.56	$<.001$
Projected headway - G	0.50	0.125	3.97	$<.001$

TABLE III: Logistic regression models of which driver merged first. Human (H): n=962; o3: n=109; Gemini 2.5 Pro (G): n=82.

	Est.	SE	t	p
Intercept - H	3.54	0.146	24.30	$<.001$
Intercept - o3	8.14	0.483	16.86	$<.001$
Intercept - G	2.73	0.335	8.16	$<.001$
Abs. projected headway - H	0.17	0.034	5.00	$<.001$
Abs. projected headway - o3	0.46	0.114	4.03	$<.001$
Abs. projected headway - G	0.40	0.082	4.95	$<.001$

TABLE IV: Linear regression models of the gap that drivers keep between the vehicles at the merge point. Human (H): n=962; o3: n=109; Gemini 2.5 Pro (G): n=82.

Indicator	H	o3	Gemini 2.5 Pro
Qualitative
The probability of merging first increases with projected headway advantage.	$\checkmark$	$\checkmark$	$\checkmark$
The probability of merging first decreases with relative velocity advantage.	$\checkmark$	X	X
The safety gap size is moderately positively affected by projected headway.	$\checkmark$	$\checkmark$	$\checkmark$
The safety gap size is not affected by initial relative velocity difference.	$\checkmark$	$\checkmark$	$\checkmark$
Drivers resolve the merging conflict with intermittent, piecewise-linear velocity profiles.	$\checkmark$	$\checkmark$	$\checkmark$

TABLE V: Selected qualitative and quantitative indicators of human (H) behavior during merging conflicts.

	o3						Gemini 2.5 Pro
Setting	I1	I2	I3	I4	I5	S	I1	I2	I3	I4	I5	S
Baseline	✓	X	✓	✓	✓	4/5	✓	X	✓	✓	✓	4/5
A1 (-History)	✓	X	✓	✓	✓	4/5	✓	X	X	✓	✓	3/5
A2 (-Prev. acc. plan)	✓	✓	✓	✓	✓	5/5	✓	X	X	✓	X	2/5
A3 (-Safety)	✓	X	✓	✓	✓	4/5	✓	X	✓	✓	✓	4/5
A4 (-Human)	✓	X	✓	✓	✓	4/5	✓	X	✓	✓	✓	4/5

TABLE VI: Alignment of model behavior with qualitative indicators (I) of human behavior across seven prompt ablations. I1: headway

\uparrow

merge-first; I2:

\Delta v\uparrow

merge-first

\downarrow

; I3: headway

\uparrow

gap; I4: gap

\perp\,\Delta v

; I5: piecewise-constant control. S: total score (out of 5).

	o3				Gemini 2.5 Pro
Setting	Coll. (%)	Gap (m)	RMSE	Joint (%)	Coll. (%)	Gap (m)	RMSE	Joint (%)
Baseline	0.00	9.28	1.34	94.50	25.45	3.84	0.70	42.68
A1 (-History)	2.73	9.29	1.30	95.33	38.18	4.82	0.85	76.47
A2 (-Prev. acc. plan)	2.73	10.63	1.50	100.00	16.36	5.14	0.86	47.83
A3 (-Safety)	3.64	9.06	1.24	98.11	30.91	3.68	0.63	47.37
A4 (-Human)	1.82	9.47	1.32	98.15	14.55	4.39	0.76	41.49

TABLE VII: Quantitative performance indicators of the models across prompt ablations. Human reference values: collision=2.83%, gap=3.85 m, RMSE=0.66 m/s, joint rate=53.0%.

Explore similar work

May 19, 2026

Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving

Large language models (LLMs) are promising for autonomous driving, but semantics-only decision policies can yield physically unsafe behavior in dynamic traffic. Existing methods either perform online language reasoning without explicit dynamics verification or use world models mainly in offline pipelines, leaving a gap between semantic intent and physical feasibility at decision time. We propose Reason--Imagine--Act (RIA), a closed-loop framework that couples an LLM reasoner with an action-conditioned world model for online safety verification. At each step, the LLM proposes an action template and candidate sub-actions, the world model performs short-horizon rollouts, and a safety scorer selects the safest executable action with feedback to the next reasoning step. Under a unified CARLA point-goal protocol (1000 episodes), RIA achieves 80.05% route completion, 51.10% arrival rate, and 0.20% collision rate. Under the same closed-loop interface, RIA consistently outperforms training-free baselines, including CARLA TM and MADA, on core closed-loop metrics. For reproducibility, code is available at https://github.com/pku-smart-city/source_code/tree/main/RIA.

Zhengqi Sun, Yiwen Sun, Boxuan Liu +3

Department of Information Management, Peking University, Beijing 100871, China · School of Intelligence Science and Technology, Peking University, Beijing 100871, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing 100080, China +3

Language Model-Based Control World Models World Models for Autonomous Driving Autonomous Driving Planning Autonomous Driving Simulation

Ask PDF

Aug 31, 2026

LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark

Public trust in Autonomous Vehicles (AVs) may depend not only on technical success but also on the fairness of their decision making. While a recent trend in AV research involves using general purpose "common sense" models to guide AV decision making, the degree to which these inherit human biases in driving is still understudied. Given that psychology studies have shown human driver biases exist, such as lower pedestrian-yielding rates to Black pedestrians in the US, we argue that analyses of model bias should also be part of AV evaluation. Concretely, in this paper we propose two new bias testing methodologies for Large Language Models (LLMs) and Visual-Language Models (VLMs)-"All Else Being Equal" tests and "Self-Consistency" tests-in order to assess bias in pedestrian-yielding decisions. Our findings show that both LLMs and VLMs make yielding decisions which are influenced by pedestrian gender, ethnicity, religion, disability, age, skin tone and socio-economic status. While the type and degree of bias is different from model to model, we highlight common patterns-and raise questions about the "common sense" model paradigm, particularly the need to either revise the paradigm or address issues of downstream bias.

Irem Yoldas, Martim Brandão, Jie Zhang +1

King’s College London, UK

Autonomous Driving Benchmarks Autonomous Driving Safety Evaluation Language Model Bias Evaluation Demographic Bias in Language Models

Ask PDF

Jun 18, 2026

Robusto-2: Benchmarking Humans & VLMs for Autonomous Driving in Lima & New York City

As Self-Driving Cars continue to expand internationally and use multi-modal systems such as VLMs as a cognitive backbone for their Action models; how well will these systems generalize in new settings, in particular out-of-distribution (OOD) edge-case scenarios in new geographies? In this paper, we study this open question by providing a full factorial analysis with human drivers of Lima, human drivers from New York City, and VLMs and showing them dashcam footage collected from Lima and New York City -- prompting them with a variety of questions under a Visual Question Answering (VQA) paradigm. In particular, we pick these two cities as they are highly challenging driving locations where no Self-Driving Car company currently operates in, and ask questions that span 4 categories: Factual, Ratings, Counterfactual and Reasoning. We find that Humans and VLMs diverge in their responses -- though this is modulated by the type of questions asked, and that Humans answer similarly independent of where they are from (Lima/NYC). To our surprise, we did not find a strong difference in terms of answers (Humans or VLMs) that was modulated by geography, likely due to their high out-of-distribution nature. Our dataset is available at: https://huggingface.co/datasets/Artificio/robusto-2

Adrian Cespedes, Marcelo Chincha, Dunant Cusipuma +3

Artificio Lima, Peru

VLM Evaluation Autonomous Driving Datasets Autonomous Driving Benchmarks VLM Robustness VLMs for Autonomous Driving Autonomous Driving

Ask PDF