Large Language Models (LLMs) are increasingly used to simulate social attitudes and behaviors, offering scalable "silicon samples" that can approximate human data. However, current simulation practice often collapses diversity into an "average persona," masking subgroup variation that is central to social reality. This study introduces audience segmentation as a systematic approach for restoring heterogeneity in LLM-based social simulation. Using U.S. climate-opinion survey data, we compare six segmentation configurations across two open-weight LLMs (Llama 3.1-70B and Mixtral 8x22B), varying segmentation identifier granularity, parsimony, and selection logic (theory-driven, data-driven, and instrument-based). We evaluate simulation performance with a three-dimensional evaluation framework covering distributional, structural, and predictive fidelity. Results show that increasing identifier granularity does not produce consistent improvement: moderate enrichment can improve performance, but further expansion does not reliably help and can worsen structural and predictive fidelity. Across parsimony comparisons, compact configurations often match or outperform more comprehensive alternatives, especially in structural and predictive fidelity, while distributional fidelity remains metric dependent. Identifier selection logic determines which fidelity dimension benefits most: instrument-based selection best preserves distributional shape, whereas data-driven selection best recovers between-group structure and identifier-outcome associations. Overall, no single configuration dominates all dimensions, and performance gains in one dimension can coincide with losses in another. These findings position audience segmentation as a core methodological approach for valid LLM-based social simulation and highlight the need for heterogeneity-aware evaluation and variance-preserving modeling strategies.
Figures & tables
Dimension
Theoretical focus
Key metrics
Distributional fidelity
Overall distribution alignment
Accuracy, Precision, Recall, F1 score; Mean Absolute Error (MAE); Kullback-Leibler divergence (KLD)
Structural fidelity
Subgroup variance preservation
Within-group: Standard Deviation (SD), Coefficient of Variation (CV); Between-group: Normalized Earth mover’s distance (nEMD), Multidimensional scaling (MDS), Procrustes distance
Predictive fidelity
Relational pattern correspondence
Cramér’s V
Table 1: Three-dimensional framework for evaluating heterogeneity restoration
Segmentation configuration
Granularity
Parsimony
Identifier selection logic
Addressed RQs
Demo
Low (single-dimensional)
High (5 identifiers)
Empirical default
Baseline for comparison
Demo+Theory-59
Very high (multidimensional)
Low (59 identifiers)
Theory-driven
RQ1, RQ2
Demo+Theory-15
High (multidimensional)
Medium (15 identifiers)
Theory-driven
RQ1–RQ3
Data-driven
High (multidimensional)
Medium (15 identifiers)
Data-driven (GBM)
RQ3
Item-15
Medium (attitude-focused)
Medium (15 identifiers)
Pre-validated instrument-based
RQ2, RQ3
Item-4
Medium (attitude-focused)
High (4 identifiers)
Pre-validated instrument-based
RQ2, RQ3
Table 2: Segmentation configurations
Figure 1: Comparison of distributional fidelity metrics across segmentation configurations and LLMs.
Figure 2: Comparison of structural fidelity metrics across segmentation configurations and LLMs.
Figure 3: MDS maps of empirical and simulated subgroup structures across segmentation configurations and LLMs.
Segmentation configuration
Human benchmark
Llama
Mixtral
Demo
.19
.27
.08
Demo+Theory-59
.25
.19
.21
Demo+Theory-15
.26
.23
.18
Data-driven
.28
.29
.33
Item-15
.34
.24
.21
Item-4
.39
.40
.44
Table 3: Summary of Cramér’s V values
Figure 4: Comparison of Cramér’s V across segmentation configurations and LLMs. Brackets and numbers indicate the differences from the human benchmark.