A Framework for Context Enrichment and Selective Robustness in LLM Social Agents

Mattia Greiche; Reihaneh Rabbany; Maximilian Puelma Touzel

cs.MANov 4, 2026

A Framework for Context Enrichment and Selective Robustness in LLM Social Agents

Authors: Mattia Greiche, Reihaneh Rabbany, Maximilian Puelma Touzel

Organizations: McGill University · Mila - Quebec Artificial Intelligence Institute

Abstract

Researchers use large language models as agents in social simulations, where agents are prompted with observations and background information intended to shape their actions. Yet LLM agents have been shown to substantially change their behavior under meaning-preserving prompt variations, creating a validation problem for social simulation. One plausible hypothesis is that this brittleness reflects a lack of grounding and can be reduced by enriching the context provided to the agents. Whether richer context improves robustness remains unclear, in part because context richness and robustness are rarely defined in ways that support controlled comparisons. We address the first need with controlled context enrichment, a set of prompt substitution rules that systematically enriches distinct types of context: neutral padding, generic social background, descriptive decision grounding, and normative decision grounding. We address the second by introducing selective robustness as a validation criterion: insensitivity to meaning-preserving prompt variation combined with a directional response to decision-relevant changes in the social world. We demonstrate the framework in a Prisoner's Dilemma-like game, showing how it can be used to identify which forms of context enrichment produce more desirable and robust agent behavior. We find that these forms of enrichment produce distinct behavioral patterns that vary between the two models tested.

Figures & tables

Condition	Added context	World contrast	Decision link	Normative force
1. Minimal	None	$\checkmark$	–	–
2. Neutral padding	Unrelated background detail	$\checkmark$	–	–
3. Generic social background	Roles not tied to the decision	$\checkmark$	–	–
4. Descriptive decision grounding	Relationships, rule, applicability	$\checkmark$	$\checkmark$	–
5. Normative decision grounding	Same as 4	$\checkmark$	$\checkmark$	$\checkmark$

Table 1: Functions of the five context conditions. The role of the agent, the decision task, and the decision-relevant world contrast are always present. The enrichment (non-minimal) blocks substitute for one another rather than accumulate.

Figure 1: Selective robustness across five context constructions for (a) Sol and (b) Terra. Each labeled point represents one prompt condition. The x-axis shows mean surface sensitivity: the average absolute difference in transfer probability between canonical prose and each of the three alternative surface forms, across both worlds. The y-axis shows the regional-minus-local difference in transfer probability averaged across surface forms; positive values indicate more transfer in the regional world. Horizontal and vertical bars show 95% bootstrap intervals. Descriptive and normative grounding produce stronger world responsiveness for both models, but Sol combines this response with lower surface sensitivity than Terra.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Context	Mean surface sensitivity	World responsiveness
Sol	Minimal	34.1 [30.3, 40.3]	33.9 [28.0, 40.2]
	Neutral padding	14.2 [10.7, 17.8]	6.7 [1.2, 12.2]
	Generic background	27.9 [21.3, 33.7]	51.1 [45.2, 56.6]
	Descriptive grounding	3.7 [1.8, 6.0]	85.9 [82.4, 88.9]
	Normative grounding	3.9 [3.3, 4.6]	92.0 [91.0, 92.9]
Terra	Minimal	19.8 [14.6, 26.9]	38.6 [29.5, 47.9]

Appendix

Table 2: Selective robustness measures using ten calls per combination of model, context, world, and surface. Entries are point estimates with 95% bootstrap intervals, in percentage points. Surface sensitivity is an average of the regional and local values.

		Regional world				Local world
Model	Context	C	B	K	A	C	B	K	A
Sol	Minimal	37.8	41.0	94.8	88.9	15.3	29.8	1.5	80.5
	Neutral padding	17.0	6.8	0.4	40.2	0.6	4.6	0.5	32.0
	Generic background	15.8	48.7	87.8	66.8	1.6	8.3	$<0.1$	4.7
	Descriptive grounding	90.3	77.7	92.5	83.1	$<0.1$	0.1	$<0.1$	0.1
	Normative grounding	96.6	77.6	99.2	94.5	$<0.1$	$<0.1$	$<0.1$	$<0.1$

Appendix

Table 3: Mean transfer probability by model, context, world, and surface, in percent. C = canonical prose, B = bullets, K = key-value table, and A = alternative prose. Values below 0.05% are shown as

<0.1

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 734 papers

Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits
May 17, 2026Jinyi Ye, Lei Cao, Ding Chen +1Human Behavior Simulation Agent-Based Modeling
Thomas Lord Department of Computer Science, University of Southern California · Annenberg School for Communication and Journalism, University of Southern California · Marshall School of Business, University of Southern California
Same Game, Different Story: A Minimal Conservative Strategic Robustness Benchmark for Large Language Model Agents
Jul 22, 2026Seyed Pouyan Mousavi Davoudi, Arshia Gharagozlou, Alireza Amiri-Margavi +2AI Agent Benchmarks Framing Effects in Language Models
Independent Researcher in AI and Statistics Tehran, Iran · Mathematics & Statistics Department University of Minnesota Duluth Duluth, MN, USA · Computational Modeling and Simulation University of Pittsburgh Pittsburgh, PA, USA +1
The Butterfly Effect in LLM Social Simulations: From Architecture Sensitivity to Robustness Audits
Oct 20, 2026Jinyi Ye, Lei Cao, Ding Chen +1Large Language Model-Based Social Simulation Language Model Robustness
University of Southern California

May 17, 2026J/K move · Enter open · S save

Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits

Jinyi Ye, Lei Cao, Ding Chen, Emilio Ferrara

Thomas Lord Department of Computer Science, University of Southern California · Annenberg School for Communication and Journalism, University of Southern California · Marshall School of Business, University of Southern California

Human Behavior Simulation Agent-Based Modeling Large Language Model-Based Social Simulation Language Model Robustness Sensitivity Analysis

Open paper AskPDF arXiv

The scientific claims drawn from LLM social simulations should be no stronger than the robustness audits that support them. Generative agents bring new expressive power to agent-based modeling, enabling simulations of collective social processes like cooperation, polarization, and norm formation. Yet they also introduce complexity through additional architectural choices, such as agent specification, memory representation, interaction protocols, and environment design. Small perturbations that appear minor to researchers can cascade into macro-level outcomes through repeated interaction, creating a "butterfly effect." Consequently, scientific claims drawn from LLM social simulations may reflect implementation artifacts rather than the social mechanisms being modeled. We support this position with two case studies: a repeated Prisoner's Dilemma and a social media echo chamber simulation. Across multiple models, minor perturbations in persona format and game-instruction framing shift cooperation rates by up to 76 percentage points, while network homophily and hub assignment produce significant and consistent shifts in polarization metrics. We also find that sensitivity is unevenly distributed across both architectural choices and model families: the same perturbation that produces the 76 pp shift in one frontier model only shifts another by 1 pp. Robustness is therefore a property that should be measured per claim and per model, not assumed. To address this validation gap, we introduce TRAILS (Taxonomy for Robustness Audits In LLM Simulations), a robustness-audit taxonomy spanning three levels of simulation design: agent (micro-level), interaction (meso-level), and system (macro-level). We call for robustness to become a first-order validation requirement before LLM social simulations are used to explain mechanisms, evaluate interventions, or inform decisions.

Abstract

LLM Agents

Explore similar work

Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits

Same Game, Different Story: A Minimal Conservative Strategic Robustness Benchmark for Large Language Model Agents

The Butterfly Effect in LLM Social Simulations: From Architecture Sensitivity to Robustness Audits