cs.GRNov 5, 2026

CrowdDirector: Runtime Orchestration of Crowd Simulation through Natural-Language Directives

Authors: Xiaohan Sun, Vuk Radovanovic, Merce Alvarez, Carol O'Sullivan, Yiorgos Chrysanthou, Panayiotis Charalambous

Organizations: Trinity College Dublin Dublin, Ireland · CYENS - Centre of Excellence Nicosia, Cyprus

Abstract

Natural language offers an intuitive way to direct a virtual crowd while a simulation is running. Handwritten rules execute efficiently but cannot respond flexibly to varied instructions, whereas asking a large language model (LLM) to control the crowd at every update introduces repeated latency, cost, and the risk of unsupported actions. We present CrowdDirector, a framework that uses an LLM only when the user adds or changes an instruction. The LLM translates the request into a structured instruction, and a scene validator checks whether the entities and actions it references exist in the current Unity scene. Accepted instructions are stored and remain continuously active as the simulation is running. At each simulation tick, a graph neural network uses the current scene state and active instruction to choose an available action and target for each agent, while Unity handles movement, interactions, and animation. The network is trained offline on simulated scenarios, with LLM guidance on how agents should respond to instructions. Our evaluations show the scene validator's feasibility decisions matched human judgements more often than the LLM's own assessment, and on a behavioural benchmark CrowdDirector follows user instructions more closely than handwritten rules while avoiding repeated LLM calls. It therefore provides a practical middle ground between fixed rules and continuous LLM control.

Figures & tables

Explore similar work

CardsList
  1. Controllable Crowd Generation through World-Model Planning

    Oct 7, 2026JunGyu Lee, Jisu Shin, Seunghyun Shin +1Human Behavior SimulationPedestrian Behavior Modeling

  2. Trajeform: Embedding Crowd Data

    Oct 29, 2026Marilena Lemonari, Panayiotis Charalambous, Yiorgos ChrysanthouSynthetic Data EvaluationHuman Behavior Simulation

  3. AgoraSim: A Hybrid Agent-Based Modeling Framework

    Jul 7, 2026Chung-Chi ChenHuman Behavior SimulationLLM Agent Orchestration