cs.MAOct 16, 2026

Model Capacity Frontier: A Development Framework for LLM-Supported Social Simulation

Authors: Patrick Zhao, Yuxuan Li, Hirokazu Shirado

Organizations: Simon Fraser University Burnaby, Canada · Carnegie Mellon University Pittsburgh, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language model (LLM) agents are increasingly incorporated into agent-based models (ABMs) on the premise that richer, context-sensitive human behavior can improve empirical fidelity. However, LLM-supported ABMs introduce an open-ended design space in which subtle modeling and configuration choices, such as prompt design and environment coupling, can substantially alter outcomes. Developing LLM-supported ABMs to simulate human collective behavior therefore requires navigating configuration sensitivity while identifying the limitations of the current model architecture. In this paper, we introduce the Model Capacity Frontier (MCF), a framework that characterizes the empirical performance attainable by a specified architecture across its admissible configurations. In multi-objective settings, this capacity is represented by an estimated Pareto frontier obtained through systematic configuration exploration. We demonstrate MCF in crowd simulation of university commencement events with approximately 12,000 agents, using empirical movement data on temporal occupancy, spatial distribution, and flow direction. We first characterize the capacity of a conventional crowd model and then evaluate successive LLM-supported architectural extensions. In the process, MCF reveals which dimensions of attainable performance expand and which remain limited, providing empirical targets for refinement and re-evaluation. Initial LLM integration improves flow direction but degrades temporal occupancy and spatial distribution; subsequent architectural refinement of the LLM layer improves temporal occupancy and recovers spatial performance, while retaining much of the earlier gain in flow direction. Our case study shows how MCF guides the systematic architectural development of LLM-supported ABMs in configuration-sensitive, open-ended design spaces.

Figures & tables

Figure 1. Model Capacity Frontier. Each point represents an admissible configuration from one of two simulation architectures evaluated on two empirical error metrics, and the Pareto frontier traces the nondominated performance attainable by each architecture. Architectural improvement should be judged across configurations, because conclusions based on selected configurations can be fragile, as illustrated here.
Figure 2. Empirical crowd setting and simulation environment. (a) Top-down simulation environment of the commencement venue, including student and guest seating, stage, obstacles, exits, and the evaluation region. (b) A snapshot from the 2026 commencement recording showing the corresponding real-world scene. The shaded polygon marks the same evaluation region shown in (a), projected onto the camera view for empirical comparison.
symbolmeaningrange
σd\sigma_{d}leaving-delay scale (s)0–182
vˉ0\bar{v}^{0}mean desired speed (m/s)1.0–1.6
sv0s_{v^{0}}desired-speed spread (m/s)0.2–0.4
T\mathcal{T}time gap (s)0.8–1.2
ℓ\ellstandstill spacing (m)0.3–0.5
aasteering strength0–16
Table 1. Admissible parameter ranges used in configuration exploration.
 M0M1M2
Onsetindividual delaygroup delaygroup delay
Destinationnearest exitLLM-chosen destinationLLM-chosen destination
Coordinationnonegroup coordinationPOI gathering + group coordination
Stoppingnonewait/stagegather and linger
Routingstaticcrowd-awarecrowd-aware
Table 2. Behavioral assumptions of the three architectures. Bold entries are introduced relative to the preceding architecture.
Figure 3. Estimated frontiers in the worse-year view. The left panel shows temporal versus spatial error, and the right panel shows temporal versus flow error. Lower is better on all axes. Gray points show the 512 M0 configurations; black points highlight the 16 M0 configurations at the matched Sobol locations used to evaluate M1 and M2. Lines connect the nondominated configurations of each architecture.
Figure 4. Behavioral diagnosis across architectures. (a) Observed occupancy and median simulated occupancy in the evaluation region over time. Shaded areas represent the 25th to 75th percentile over the 16 configurations. (b) Mean walking direction by spatial cell in 2026, weighted by observed pedestrian movement. Arrows are scaled by the amount of observed walking.
TransitionTemporalSpatialFlow
M0 →\rightarrow M1+81+81 (1)+0.91+0.91 (4)−0.31\mathbf{-0.31} (15)
M1 →\rightarrow M2−276\mathbf{-276} (16)−1.18\mathbf{-1.18} (14)+0.19+0.19 (3)
M0 →\rightarrow M2−223\mathbf{-223} (15)−0.45\mathbf{-0.45} (11)−0.13\mathbf{-0.13} (12)
Table 3. Changes across development over the 16 matched configurations. Entries report the median change in error, with the number of configurations that improve in parentheses. Negative values indicate better fit and are shown in bold.
Figure 5. Configuration-matched trajectories from M0 to M2 in the temporal–flow error space. Dashed lines connect configurations with identical values of Θ\Theta . The black and orange points and the corresponding frontier lines are the same as in Figure 3 .
Figure App.1. Calibration details from the video to the simulation’s map. (a) A grid in world meters projected back onto the camera’s background. (b) The simulation’s arena projected onto the same view: seating (purple), obstacles (blue), running track lanes (white) and the evaluation region (yellow). (c) The camera view projected into the top-down frame of the simulation. (d) DM-Count’s density map at t=7t=7 min on the simulation map.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

 head  CSRNetDM-Count
crowd densitydetectorCSRNetDM-Count(hybrid)(hybrid)
all37%16%20%11%7%
sparse ( << 0.3)5%3%34%6%13%
medium (0.3–1.5)21%18%9%17%7%
dense ( >> 1.5)74%19%28%5%3%
Appendix
Table App.1. Mean absolute percentage error (lower is better) of each counting setup on 11 hand-labeled image patches, sorted by crowd density (persons/m 2 ; 2, 5, and 4 images). A hybrid setup uses the head detector in sparse crowds and the network in dense ones. We settle on the DM-Count hybrid.
Figure App.2. Hand labels for the two measurements. (a) Heads in a patch, labeled by hand (yellow dots). Cyan circles are the head detector’s detections. (b) Flow direction: the 13 sampled cells of one minute. Yellow arrows are the hand labeled direction and cyan arrows are measured mean direction; dotted cells were marked as mixed or without walking.
Figure App.3. Navigation fields of two ways out, presented as examples. Different shading levels describe the travel time to the way out (m of open ground).
More in this topic

Large Language Model-Based Social Simulation

Large language model-based social simulation studies how language-model agents represent people and interact to produce social behavior.

See all 242 papers

Explore similar work

CardsList
Jul 7, 2026

AgoraSim: A Hybrid Agent-Based Modeling Framework

LLM-agent simulations make natural-language social scenarios easy to instantiate, but their outputs can be overread as predictions and are often difficult to compare with explicit social dynamics. We present AgoraSim, a hybrid agent-based modeling framework for scenario-oriented social reaction analysis. AgoraSim resolves textual or multimodal artifacts into editable ABM configurations, runs ratio-controlled populations that mix LLM, vision-language, custom-endpoint, random, and classical agents, and compares the same scenario against matched classical reference dynamics. All agents emit a shared structured decision object, enabling common action spaces, interaction protocols, metrics, and audit records. Exposed through a local UI, Python SDK/CLI, and REST API, AgoraSim helps users inspect scenario trajectories, compare modeling assumptions, and identify cases that warrant empirical validation.

Chung-Chi Chen

Human-Agent Ally Lab (HAA Lab) National Institute of Informatics, Japan

Human Behavior SimulationLLM Agent OrchestrationAgent-Based ModelingLLM AgentsLarge Language Model-Based Social Simulation
AskPDF
Jul 20, 2026

Towards Agentic Agent-based Models: Feasibility, Performance, and Statistical Model Checking

Agent-based models (ABMs) rely on simple, explicit and reproducible rules for individual decision making, while complex collective behavior emerges from interactions among agents. Recent advances in large language models (LLMs) make it tempting to replace, enrich, or perturb these rules with LLM-based agentic capabilities. However, this raises a methodological question: how does introducing LLM-driven decisions affect the reliability, computational cost, and behavior of ABM simulations? We investigate this for Mesa ABM models, a popular Python library for ABMs, analyzed by statistical model checking. Building on Mesa's integration with the statistical model checker MultiVeStA, we extend the classical Schelling segregation model with a hybrid population: ordinary agents classify neighbors using the standard symbolic rule, while one agent delegates this task to an LLM through tool calls. The LLM-enabled agent receives natural-language descriptions of neighboring agents and invokes tools that increment counters of similar/different neighbors; these counters determine its happiness according to the original Schelling dynamics. This provides a minimal but controlled setting where the semantic, operational, and computational behavior of LLM-based decisions can be studied inside an otherwise standard ABM. We report preliminary experiments with locally served LLMs of different sizes, showing that smaller models may fail simple semantic classification experiments or become operationally unusable during repeated tool-call generation, while larger tested models pass these preliminary checks. We discuss how statistical model checking can estimate classical ABM observables and quantify the impact of introducing agentic LLM components into simulation models.

Stefano Blando, Emanuele Guerrazzi, Riccardo Porcedda +3

Sant’Anna School of Advanced Studies Pisa, Italy · University of Pisa, Italy · CentraleSupélec, Université Paris-Saclay, France +2

Agent-Based ModelingLLM AgentsLLM Agent Reliability
AskPDF
Jul 7, 2026

LLM-powered reasoning in agent-based modeling

Agent-based modeling (ABM) has the capability to model millions of individuals and their interactions, which is useful for policy making. However, ABMs have traditionally relied on static prior, which prevents the models from adapting to real-time changes. Our research provides a novel approach to addressing this information gap. Large language models (LLMs) offer new opportunities to predict human decision-making. Here, we introduce a scalable Hybrid Agent-based and Language-driven Epidemic (HALE) modeling framework that leverages LLMs to predict human decision-making in an ABM simulation. As a proof-of-concept, we use HALE to simulate COVID-19 and its effects in Salt Lake County, UT.

Sifat Afroj Moon, Dakotah Maguire, Adam Spannaus +5

Oak Ridge National Laboratory · Computational Science and Engineering Division, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA · Geospatial Science and Human Security Division, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA +1

Agent-Based ModelingLarge Language Model-Based Social Simulation
AskPDF