VehicleArena: A Realistic Urban Environment for Multi-Agent Driving

Jie Yang; Jiajun Chen; Jiazheng Zhou; Mianqiu Huang; Yining Zheng; Yuxin Wang; Xipeng Qiu

cs.MASep 28, 2026

VehicleArena: A Realistic Urban Environment for Multi-Agent Driving

Authors: Jie Yang, Jiajun Chen, Jiazheng Zhou, Mianqiu Huang, Yining Zheng, Yuxin Wang, Xipeng Qiu

Organizations: Fudan University · Shanghai Innovation Institute

Abstract

Real-world embodied agents often pursue independent objectives within a shared physical environment, where their actions can alter the conditions faced by others. Existing benchmarks, however, typically assume shared goals or explicitly prescribed interaction protocols, leaving such emergent physical coupling underexplored. We introduce VehicleArena, a 3D urban-driving benchmark for studying independently operating agents in a dynamic shared world. In VehicleArena, LLM-controlled agents must fulfill evolving passenger requests while navigating complex traffic, and each agent's driving decisions can reshape traffic flow, delays, risks, and subsequent observations for surrounding agents. The benchmark provides 112 evaluation tasks spanning single-agent and multi-agent driving. Across nine evaluated models, the highest arrival rates reach only 65.0% on single-agent tasks and 65.6% on multi-agent tasks, while strong passenger-request or cabin scores do not reliably translate into successful trip completion. Moreover, in matched multi-agent runs, every tested focal policy reduces the arrival rate of surrounding vehicles relative to the simulator's native traffic controller, revealing measurable externalities beyond the focal vehicle itself.

Figures & tables

Figure 1 : VehicleArena architecture. Left: the shared 3D world and observation examples. Right: agent roles, support interfaces, and configurable cabin capabilities. Bottom: PA asks DA to play music after leaving an intersection and registers a verification trigger; DA sets a wake-up time to carry out the request, while Judge checks are triggered independently of DA wakes. The timeline is illustrative and not to scale. The main image is a native Web3D capture from a SUMO-only reference rollout; the smaller images are separate observation examples.

Figure 2 : A recorded episode illustrating interleaved driving and passenger service. The upper panels show three DA wakes and a passenger request, together with the relevant observations and actions. Following a rear-traffic warning, DA raises its target speed to 35 km/h; it later handles a cabin request without replacing that motion target. The lower road shows the corresponding speeds realized by SUMO, which advances at 0.1-second steps between wakes. Time gaps are compressed and intervening wakes are omitted.

	Basic (80 tasks)				Multi-Agent (32 tasks)
Model	Arr. (%)	Drive	Req.	Cabin	Arr. (%)	Drive	Req.	Cabin
DeepSeek-V4.1-Flash	62.5	85.0	88.5	76.6	56.3	81.7	82.1	73.9
MIMO-V2.6-Pro	40.0	79.5	90.0	57.5	62.5	80.0	88.0	66.7
Kimi-K3	51.3	78.9	84.9	41.5	40.6	71.4	76.4	49.2
GLM-5.3-Flash	45.0	78.6	85.7	55.6	62.5	80.9	85.7	63.1
Qwen3.8-Max	65.0	83.1	86.7	72.0	65.6	82.4	88.2	65.4

Table 1 : Main evaluation by task group. Basic has 80 tasks per model and Multi-Agent has 32. The best value in each metric column is bold; the second-best value is underlined, including ties.

Figure 3 : Planning activity versus completion-weighted outcome for Basic (left) and Multi-Agent (right): ego DA wakes per task (top) and tool calls per wake (bottom). Each marker denotes a model; Appendix A.7 defines the outcome index.

Model	Speed (km/h)	Collisions
DeepSeek-V4.1-Flash	19.0	4
MIMO-V2.6-Pro	16.8	0
Kimi-K3	18.6	4
GLM-5.3-Flash	18.5	3
Qwen3.8-Max	19.2	3
Qwen3.8-27B	16.9	5

Table 2 : Ego vehicle speed and collisions across all 112 tasks. Mean speed includes stopped time; collisions count ego-involved events.

Figure 4 : Ego DA skill and tool choices pooled over Basic and Multi-Agent tasks. Both heatmaps share the model order shown on the left. Left: fraction of tasks with a successful load of each skill. Right: share of non- finish tool calls in the six most common tool families; “Other” contains the remainder. Each panel has its own percentage color scale.

Figure 5 : Surrounding-traffic burden relative to matched SUMO runs in Basic (left) and Multi-Agent (right). Axes show added delay and collision/non-arrival rate excess; reductions appear as zero. Appendix A.6 gives definitions and scales.

	Basic (80 tasks)			Multi-Agent (32 tasks)
Model	Arr. (%)	Drive	Cabin	Arr. (%)	Drive	Cabin
Kimi-K3	61.3 ( $+10.0$ )	74.6 ( $-4.3$ )	71.6 ( $+30.1$ )	56.3 ( $+15.6$ )	77.2 ( $+5.8$ )	81.6 ( $+32.4$ )
GLM-5.3-Flash	62.5 ( $+17.5$ )	79.1 ( $+0.5$ )	75.6 ( $+20.0$ )	65.6 ( $+3.1$ )	78.8 ( $-2.1$ )	83.3 ( $+20.2$ )

Table 3 : Ego PA ablation using the main-table aggregation. Each scored cell reports the PA-off value and its change from PA on in parentheses; arrival changes are percentage points.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Role	Model	Temperature	Thinking	Effort	Context budget	Output limit
Ego DA	DeepSeek-V4.1-Flash	0.7	Enabled	max	1000000	32k
	MiMo-v2.6-Pro	1	Enabled	N/A a	1000000	32k
	Kimi-K3	0.7	Enabled	max	1000000	32k
	GLM-5.3-Flash	0.7	Enabled	max	1000000	32k
	Qwen3.8-Max	0.7	Enabled	xhigh	1000000	32k
	Qwen3.8-27B	0.7	Enabled	xhigh	1000000	32k

Appendix

Table 4 : Recorded inference settings for the main comparison. Context budgets are in tokens; 32k denotes 32,768 output tokens per call.

Statistic	Train	Test Basic	Test MultiLLM	Overall
Tasks	100	80	32	212
Maps	57	48	13	89
LLM vehicles	1.0	1.0	4.0	1.5
NPC vehicles	13.1 (8–24)	13.2 (8–24)	10.3 (8–13)	12.7 (8–24)
Pedestrians	0.5 (0–8)	0.5 (0–8)	0.0	0.4 (0–8)
Ego route (m)	373.8 (22.0–2620.7)	402.4 (63.4–4867.2)	199.6 (75.8–427.9)	358.3 (22.0–4867.2)

Appendix

Table 5 : Task-suite statistics. Tasks and maps are counts; other entries are per-task means, with ranges in parentheses.

Wake source	Default rule
Shared passenger-visible events	Simulation start, acoustic cue, weather change, and day–night change.
Hard braking edge	Acceleration at most $-3\,\mathrm{m/s^{2}}$ for 0.3 seconds; detection is rearmed only after acceleration exceeds $-1.5\,\mathrm{m/s^{2}}$ .
Prolonged-stop edge	Speed below $0.5\,\mathrm{km/h}$ for 15 seconds; detection is rearmed after speed exceeds $2\,\mathrm{km/h}$ .
Independent random wake	A delay sampled from $U(15,45)$ simulation seconds, unaffected by the Driving-Agent heartbeat.
Coalescing	A five-second cooldown merges repeated events by category. A PA may also finish without issuing a request; its update still wakes the Driving Agent once at that boundary.

Appendix

Table 6 : Default PA scheduling. The random stream is independently seeded per vehicle and scenario.

Family	Available predicates
Time	A bounded delay after request creation.
Route and map	Approaching, entering, or exiting the next intersection; distance to destination below a bounded threshold.
Vehicle state	Vehicle stopped, vehicle resumed moving, or a speed threshold held for a bounded duration.
Environment	A scheduled weather transition to an exposed condition, or a scheduled transition to darkness.

Appendix

Table 7 : Declarative Judge-trigger catalog. Only currently feasible, parameter-bounded members of this catalog are shown to the PA.

State	Meaning
Created / waiting	The frozen request exists; a deferred request is waiting for a feasible selected predicate.
Activated / checking	The predicate occurred and the Judge gathers the bounded time-window evidence.
Completed	Evidence establishes fulfillment within the request validity period; one-shot requests may close early once all core criteria are established.
Uncompleted / unverified	The check budget ends with unmet criteria or insufficient evidence, respectively.
Superseded / episode ended	A later PA request replaces the request, or the physical episode ends before its pending lifecycle is resolved.
NA / infrastructure failure	An invalid request or insufficient evidence is unscored (NA); operational failures are tracked separately.

Appendix

Table 8 : Request lifecycle. Terminal states are reported as coverage rather than silently folded into the request score.

Metric	Unit and denominator	Definition and companion report
Trip completion $A$	Percentage of tasks	Ego arrival by $T_{\mathrm{limit}}$ without collision or route failure. Peer arrivals are reported separately, not required for ego success.
Driving quality $D$	Score in $[0,100]$ per evaluated vehicle	Starts at 100 with deductions for driving violations; an at-fault collision or red-light violation sets the score to zero.
Passenger fulfillment $R$	Score in $[0,100]$ over valid scored requests	A–F Judge grades mapped to 100, 80, 60, 40, 20, and 0. Request-level and scenario-level means use explicit denominators.
Cabin compliance $C$	Score in $[0,100]$ over checked fields	Field-weighted accuracy pooled across ego checkpoints, including sentinel checks. Rules requiring unavailable equipment are excluded.
Externality $E$	Counts and vehicle-seconds per valid pair	Changes in NPC collisions and non-arrivals, plus positive increases in queueing time and trip-completion time relative to SUMO.
Efficiency $K$	Input and output tokens per task	Total model-token use, reported separately for input and output.

Appendix

Table 9 : VehicleArena metrics. Metrics are reported separately rather than combined into one scalar.

Grade	Score	Evidence-based interpretation
A	100	All requirements are resolved correctly within the expected response time.
B	80	All requirements are resolved, but after the expected response time and before expiry.
C	60	All core requirements are resolved, with minor secondary omissions or missing secondary evidence.
D	40	Useful partial fulfillment of core requirements, or failure to maintain a sustained requirement.
E	20	Relevant physical action is observed, but it produces no useful requested outcome.
F	0	No relevant action, an unrelated action, or an outcome contrary to the request.

Appendix

Table 10 : Passenger-request grading rubric. NA is an unscored status, not a seventh grade.

Successful skill loads (tasks)
Model	Day/night	Weather	Driving	Safety
DeepSeek-V4.1-Flash	51	41	62	5
MIMO-V2.6-Pro	37	35	7	0
Kimi-K3	14	23	2	0
GLM-5.3-Flash	30	30	4	0
Qwen3.8-Max	46	48	0	1

Appendix

Table 11 : Per-model skill loads, tool calls, and ego DA token counts.

Figure 6 : Inference cost and task outcome by model (Basic: left; Multi-Agent: right). Bars show mean ego DA tokens per task (left axis, millions); the line shows the completion-weighted index (right axis, points).

Scene at request time	Passenger request
Urban lead-vehicle following in Beijing, in cloudy weather. At 5.0 s, a hard-braking event prompts a request for more separation from the car ahead.	After that hard brake, slow to about 25 km/h so we keep more distance from the car ahead.
The same following episode, after rain begins. At 20.4 s, the passenger updates rain-related device settings and plans a quiet final approach.	Once we are within about 130 m of the destination, slow to about 20 km/h for the final approach and turn the music off.
Platoon-following traffic in Shanghai. At 14.1 s, fog appears as afternoon changes to dusk, reducing visibility.	It is foggy and we are approaching the destination; slow to about 20 km/h.
The same foggy episode at dusk. At 19.1 s, a later request specifies a further slowdown for the final approach and preparation for arrival.	When navigation shows less than 70 m to the destination, slow to about 10 km/h and pause the music to prepare for arrival.
An urban crosswalk route in Shanghai at dusk. Rain begins at 15.1 s; the passenger requests wipers, fog lights, and a cautious final approach.	Once we are within 50 m of the destination, slow to below 15 km/h, then stop smoothly after arrival.
An urban road with oncoming traffic in Sydney, in the afternoon. At 10.0 s, a passenger who had been reading asks for quieter audio and prepares to pack up.	Within 100 m of the destination, approach gently at under 10 km/h and turn off the passenger reading light so I can pack up.

Appendix

Table 12 : Scene context and low-speed passenger requests in the PA ablation.

Violation	Deduction
At-fault collision or red-light violation	Score becomes 0
Sustained speeding, at least 5 seconds	20
Short speeding episode, 1 to less than 5 seconds	5
Unjustified stopping	5 each
Unjustified delay in starting after a green light	5
Lane change without signaling	10

Appendix

Table 13 : Driving-score deduction schedule. Repeated detections are grouped into events or episodes before scoring.

Field	Effect
Equipment profile	Selects a named set of cabin and sensor modules.
enable_modules / disable_modules	Adds or removes named optional modules for one vehicle. Required world and driving modules cannot be removed.
Chassis profile	Selects vehicle length, width, acceleration, braking, and lane-change duration.
Chassis overrides	Replaces individual positive chassis values for a task-specific vehicle variant.
Trusted extension	At run initialization, registers additional modules, equipment profiles, chassis profiles, scenario layers, or rule directories. Scenario data cannot import code.

Appendix

Table 14 : Per-vehicle configuration fields.

Group	Registered modules
World and driving	navigation, map, weather, dayNight, speedLimit
Sensors	frontRadar, rearRadar, lidar
Climate and comfort	airConditioner, seat, sunshade, readingLight
Display, media, and communication	bluetooth, broadcast, centerInformationDisplay, conversation, HUD, instrumentPanel, music, overheadScreen, radio, video, horn
Lighting and safety	fogLight, hazardLight, highBeamHeadlight, lowBeamHeadlight, positionLight, tailLight, turnSignal, wiper
Body and physical controls	door, footPedal, frontTrunk, fuelPort, rearviewMirror, steeringWheel, sunroof, trunk, window

Appendix

Table 15 : Built-in VehicleArena modules. Names are the module identifiers used by the capability resolver and tool-discovery interface.

Region	City	Local area(s)
Mainland China (57)	Beijing	Guomao; Sanlitun; Tiananmen; Wangjing; Wudaokou; Xidan; Yizhuang; Zhongguancun
	Changchun	Chaoyang
	Changsha	Furong; Wuyi
	Changzhou	Tianning
	Chengdu	Chunxi; Gaoxin; Jinjiang
	Chongqing	Jiefangbei

Appendix

Table 16 : City-level map inventory. Each local area corresponds to one lane-level map; the table does not imply that every map appears in the 112-task evaluation set.

Models	#Para	Launch date	Context (tokens)	Scaling	Corporation	License
DeepSeek-V4.1-Flash	748B	2026-09-10	1,048,576	Effort	DeepSeek	MIT
MiMo-v2.6-Pro	1.02T	2026-09-22	1,048,576	Budget	Xiaomi	MIT
Kimi-K3	2.8T	2026-07-17	1,048,576	Effort	Moonshot AI	Kimi K3
GLM-5.3-Flash	320B	2026-09	1,048,576	Effort	Z.ai	MIT
Qwen3.8-Max	2.4T	2026-08-02	1,000,000	Effort	Alibaba	Proprietary
Qwen3.8-27B	27B	2026-08-14	262,144 a	Effort	Alibaba	Apache 2.0

Appendix

Table 17 : Specifications of the large language models evaluated in our experiments.

Skill	Use
driving_control	Motion commands, perception, routing, and command receipts.
weather_transition	Equipment changes for weather and visibility.
daynight_transition	Lights, displays, and mirrors around darkness.
safety_refusal	Unsafe-request refusal and safe alternatives.

Appendix

Table 18 : Operational skill catalog in the implementation.

Tool	Function
navigation__navigation_route_plan	Display a suggested route.
navigation__navigation_minimap	Show a route or local map without replanning.
navigation__navigation_set_speed	Set target speed and optional acceleration limits.
navigation__navigation_emergency_stop	Request emergency braking.
navigation__navigation_change_lane	Request one adjacent lane.
navigation__navigation_select_maneuver	Select a legal junction connector.

Appendix

Table 19 : Core DA tools preloaded when their modules are installed.

Tool	Function
get_module_api	List a module’s callable methods.
load_tools	Load selected module APIs.
load_skill	Load procedural guidance.
todo_manage	Maintain tasks across wakes.
memory_search	Search passenger, event, and action history.
set_heartbeat_interval	Set periodic observation interval ( $0.1$ – $30$ s).

Appendix

Table 20 : DA discovery, harness, and additional tool functions.

Tool	Function
send_immediate_request	Create or replace an immediate request with outcomes and timing.
send_triggered_request	Create or replace a two-phase request with a typed trigger.
cancel_passenger_request	Cancel the active request when no pending outcome is wanted.
finish	End the wake; keep the active request unchanged.

Appendix

Table 21 : PA request-management tools.

Tool	Function
submit_passenger_check	Record intermediate criterion statuses and evidence; no grade.
submit_passenger_judgement	Submit a final A–F/NA grade, statuses, and fulfillment times.

Appendix

Table 22 : Judge submission tools.

Explore similar work

TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control
Apr 19, 2026Siqi Lai, Pan Zhang, Yuping Zhou +3Traffic Signal Control Urban Environments
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China · Shandong University, Jinan, Shandong, China
MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systems
May 11, 2026Marco Coscoy, Zewei Zhou, Seth Z. Zhao +9V2X Autonomous Driving Simulation
University of California, Los Angeles
360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agents
Aug 9, 2026Kenta Watanabe, Atsuyuki Miyai, Mizuki Takenawa +2Urban Environments Object Goal Navigation
The University of Tokyo, Tokyo, Japan

Apr 19, 2026, cs.AIJ/K move · Enter open · S save

TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control

Siqi Lai, Pan Zhang, Yuping Zhou, Jindong Han+2

The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China · Shandong University, Jinan, Shandong, China

Traffic Signal Control Urban Environments Large Language Model Agents Long-Horizon Task Planning

Open paper AskPDF arXiv

Large language model (LLM) agents have shown strong capabilities in long-horizon reasoning, tool use, and decision-making in digital environments, yet extending them to physically grounded systems remains challenging. Unlike web, code, or game environments, where objectives are often weakly coupled, physical systems evolve through tightly coupled dynamics in which local interventions propagate across interacting subsystems over time. Urban traffic control exemplifies this challenge, as traffic signals, freeways, public transit, and taxi systems continuously interact through shared spatial infrastructure and temporal mobility demand. Existing optimization, reinforcement learning (RL), and LLM-based approaches are largely designed for isolated subsystems, limiting coordinated reasoning and system-level optimization. We propose TrafficClaw, a LLM-based generalizable traffic control agent for physical urban systems. TrafficClaw operates within a unified traffic environment that exposes coupled urban dynamics and feedback, performs executable spatiotemporal reasoning with persistent memory for long-horizon adaptation, and leverages multi-stage agentic RL for coordinated system-level optimization. Experiments across three metropolitan regions and six traffic-control tasks demonstrate strong generalization, robustness, and cross-subsystem coordination. Our project is available at https://github.com/usail-hkust/TrafficClaw.

Abstract

Explore similar work

TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control

MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systems

360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agents