Simulating how organized groups (e.g., corporations) make decisions (e.g., responding to a competitor's move) is essential for understanding real-world dynamics and could benefit relevant applications (e.g., market prediction). In this paper, we formalize this problem as a concrete research platform for group behavior understanding, providing: (1) a task formalization with benchmark and evaluation criteria, (2) a structured and adaptive analytical framework, and (3) detailed temporal and cross-group analysis. Specifically, we propose Organized Group Behavior Simulation, a task that models organized groups as collective entities from a practical perspective: given a group facing a particular situation (e.g., AI Boom), predict the decision it would take. To support this task, we present GROVE (GRoup Organizational BehaVior Evaluation), a benchmark covering 44 organized groups with 8,052 real-world context-decision pairs collected from Wikipedia and TechCrunch across 9 domains, with an end-to-end evaluation protocol assessing consistency, initiative, scope, magnitude, and horizon. Beyond straightforward prompting pipelines, we propose a structured analytical framework that converts collective decision-making events into an interpretable, adaptive, and traceable behavioral model, achieving stronger performance than summarization- and retrieval-based baselines. It further introduces an adapter mechanism for time-aware evolution and group-aware transfer, and traceable evidence nodes grounding each decision rule in originating historical events. Our analysis reveals temporal behavioral drift within individual groups, which the time-aware adapter effectively captures for stronger prediction, and structured cross-group similarity that enables knowledge transfer for data-scarce organizations.
Figures & tables
Figure 1: Framework for Simulating Organized Group Behavior.
Figure 2: Curation and Evaluation of GROVE Benchmark
Figure 3
Figure 4: Adaptation settings and results for groups with significant drift ( ∗ ), non-significant drift (n.s.), and cross-source transfer from Wikipedia to TechCrunch.
Figure 5
Figure 6: Cases for temporal behavior drift (left), group similarity and knowledge transfer (right) with traceable evidence node for validation
Appendix figures & tables37 assets
Supplementary material from the paper’s appendix.
Appendix
Dom.
Group
Tr
Te
Dom.
Group
Tr
Te
Aero.
Boeing
111
48
Spo.*
CR Vasco
148
64
Brit. Caledonian
195
84
Dallas Cowboys
142
61
Cubana
99
43
Bayern Munich
72
32
United Airlines
103
45
Fluminense
296
127
Auto.
Alfa Romeo
133
58
Grêmio
129
56
Chrysler
105
46
McLaren
180
78
Appendix
Table 3: Experiment dataset statistics.
Figure 7: Group behavior filtering prompt with three criteria: attribution, successor, and significance tests.
Source
Question
Agree (%)
κ
Final Human–GPT (%)
Wikipedia
Q1: Group Behavior
89.0
0.76
92.0
Q2: Rewrite Fidelity
96.0
0.91
97.0
Q3: Question Quality
93.0
0.82
94.0
TechCrunch
Q4: Summary Accuracy
92.0
0.78
90.0
Q5: No Leakage
94.0
0.84
96.0
Appendix
Table 4: Benchmark quality validation. “Agree” is agreement between the two primary annotators. “Final Human–GPT” is agreement between the adjudicated human label and GPT-4.1 over all sampled items.
Comparison
Agree (%)
κ
Spearman ρ
H–H (primary annotators)
78.0
0.67
0.74
Final H–GPT-4.1
73.0
0.60
0.71
Appendix
Table 5: Consistency-judge validation. H–H compares the two primary annotators; Final H–GPT compares the adjudicated human label against GPT-4.1 over all 100 pairs. The lower block gives per-class Final H–GPT agreement.
Figure 8: CDT Visualization for Nintendo
Figure 9: A running case for one data sample from Nintendo
Symbol
Description
Value
Dmax
Maximum tree depth
3
R
Clustering rounds (guide suffixes)
4
m
Observations sampled per centroid
8
k
Hypotheses generated per cluster
3
τaccept / τkeep
Precision threshold to accept / keep a statement
0.65
τreject / τdelete
Precision threshold to reject / delete a statement
0.35
Appendix
Table 6: Hyperparameters for CDT construction (§ A.1 ) and adaptation (§ 3.4 ). Thresholds τaccept and τkeep share the same value, as do τreject and τdelete .
Figure 10: Example of a human-written Wikipedia profile used as background knowledge (Boeing).
Prediction Model
Vanilla
Human Profile
Summarization
RAG
Ours (CDT)
Qwen3-4B-Instruct-2507
57.9 / 79.8
59.4 / 83.0
64.6 / 86.2
56.8 / 83.8
68.2 / 86.6
Qwen3.5-4B
55.0 / 82.9
53.8 / 81.7
58.7 / 85.7
58.1 / 81.3
63.1 / 88.3
gpt-5-nano-2025-08-07
76.1 / 88.8
78.1 / 88.8
77.7 / 90.5
76.2 / 88.5
83.1 / 91.6
gpt-5-mini-2025-08-07
80.4 / 89.8
80.2 / 90.1
81.6 / 92.1
77.9 / 86.0
84.4 / 93.0
gpt-5-2025-08-07
83.0 / 89.7
82.8 / 89.9
84.2 / 90.3
81.5 / 85.2
87.2 / 92.0
gemini-3-flash-preview
64.0 / 85.9
67.4 / 86.2
78.0 / 88.0
74.1 / 75.9
79.2 / 89.4
Appendix
Table 7: Consistency results across prediction models. Each entry reports Wikipedia / TechCrunch average.
Prediction Model
Fixed CDT
Retrained CDT
Adapted CDT
Qwen3-4B-Instruct-2507
66.2
66.6
68.5
Qwen3.5-4B
61.1
61.4
65.1
gpt-5-nano-2025-08-07
80.1
82.0
86.0
gpt-5-mini-2025-08-07
83.6
83.4
85.3
gpt-5-2025-08-07
85.5
86.1
88.3
gemini-3-flash-preview
79.8
79.7
80.1
Appendix
Table 8: Temporal adaptation consistency results across prediction models.
Config.
τaccept
τreject
gpt-5-nano
gpt-5-mini
Qwen3-4B-Inst.
Average
C0
0.65
0.35
80.2
84.2
64.6
76.33
C1
0.55
0.35
82.3
85.5
63.5
77.10
C2
0.75
0.35
80.4
84.6
63.1
76.03
C3
0.85
0.35
83.6
83.2
60.5
75.77
C4
0.65
0.25
85.1
82.4
65.7
77.73
C5
0.65
0.45
81.2
80.9
63.9
75.33
Appendix
Table 9: Threshold sensitivity on five representative groups. C0 is the default configuration.
Wikipedia
TC
Method
Aero.
Auto.
Broad.
Edu.
Ener.
Game
Ret.
Spo.
Tech.
Avg
Avg
Vanilla
80.0
80.7
79.8
78.7
86.5
74.2
72.2
77.2
82.7
79.1
89.9
Human Profile
81.0
78.8
78.7
75.3
80.2
75.8
66.7
77.9
81.5
77.3
88.0
Summary-based
80.2
78.9
79.6
79.0
87.5
74.2
69.4
77.9
80.5
78.6
88.7
RAG
77.8
74.5
80.9
80.1
85.4
77.5
69.4
75.0
75.6
77.4
85.9
Ours (CDT)
82.8
82.8
85.8
81.0
92.7
79.2
68.1
82.7
78.9
81.6
90.2
Appendix
Table 10: Consistency results on GROVE benchmark (gpt-4o-mini).
Domain
Source
Target
Vanilla
Tgt CDT
Transfer
Aero.
Brit. Cal.
Airbus
84.1
79.5
79.5
Auto.
Volkswagen
Ford
81.0
81.0
88.1
Spo.
Flumin.
Derry City
80.0
80.0
83.3
SE_Palmeiras
GELP
87.5
70.8
85.4
Edu.
Cornell
Princeton
100.0
96.2
96.2
Cornell
UPenn
66.7
76.7
80.0
Appendix
Table 11: CDT Transfer Experiment (gpt-4o-mini, Consistency Score)
Domain
Source
Target
Vanilla
Tgt CDT
Transfer
Aero.
Brit. Cal.
Airbus
71.6
62.5
76.1
Auto.
Volkswagen
Ford
79.2
75.0
82.1
Spo.
Flumin.
Derry City
74.2
68.3
71.2
SE_Palmeiras
GELP
79.2
60.9
72.9
Edu.
Cornell
Princeton
74.0
76.9
80.8
Cornell
UPenn
81.7
77.5
83.3
Appendix
Table 12: CDT Transfer Experiment (Qwen2.5-7B-Instruct, 4-Dim Score)
Figure 11: Event-level Analysis for Group Similarity on Technology, Aerospace, Education, Sports and Automotive
Figure 12: Event-level Analysis for Group Similarity on Broadcast, Energy, Game, Retail
Wikipedia
Method
Aero.
Auto.
Broad.
Edu.
Ener.
Game
Ret.
Spo.
Tech.
Avg
Ours (CDT)
74.1
77.1
74.4
74.8
76.6
75.6
67.2
74.4
78.0
74.7
Ours (CDT retrained)
72.9
75.9
74.5
71.6
78.1
75.6
61.5
74.3
79.9
73.8
Ours (CDT + Adaptation)
74.1
76.7
73.2
78.0
76.6
80.0
62.5
76.3
81.0
75.4
Appendix
Table 13: Temporal Adaptation Study — 4-Dim (Qwen2.5-7B-Instruct).
Figure 13: Hypothesis generation prompt (Step 2 of CDT construction). Given clustered scene–action pairs and existing statements, the LLM proposes k behavioral statements and k corresponding gate questions via chain-of-thought reasoning.
Figure 14: Hypothesis summarization and deduplication prompt (Step 3). Merges overlapping hypotheses from multiple clusters into a compact, non-redundant set.
Figure 15: Ungated validation prompt (Step 4, Stage 1). The discriminator LLM judges whether a single action is consistent with a candidate behavioral statement without any gate conditioning.
Figure 16: Gate evaluation prompt (Step 4, Stage 2). Determines whether a scene satisfies a candidate gate question. Used during both construction and inference-time tree traversal.
Figure 17: Multi-candidate selection prompt (Step 5). Each of Rsel voting rounds presents the candidates in a random order; the candidate with the most votes is selected.
Figure 18: Batch NLI classification prompt (Phase 1). Classifies the relationship between one action and all statements at a node in a single call, enabling efficient incremental update of the relation matrix Mn .
Figure 19: Gate hypothesis generation prompt (Phase 2, demotion). When a statement’s precision falls below τkeep , this prompt generates candidate gate questions to separate supporting from contradicting observations.
Figure 20: Gate–statement semantic compatibility check (Phase 2, demotion). Verifies that a candidate gate question is semantically coherent with the statement being demoted to a child subtree.
Figure 21: New statement generation prompt (Phase 3). Proposes behavioral statements for observations not covered by any existing statement at the current node, conditioned on the gate path from root.
Figure 22: CDT inference prompt.
Figure 23: NLI-based evaluation prompt for consistency-level evaluation.
Figure 24: Evaluation prompt for initiative dimension.
Figure 25: Evaluation prompt for scope dimension.
Figure 26: Evaluation prompt for magnitude dimension.
Figure 27: Evaluation prompt for horizon dimension.
Figure 28: Vanilla inference prompt (no background knowledge).
Large language models are increasingly used to simulate how individuals respond to new situations, yet the behavioral reasoning behind these responses is either inherited from pretraining or learned from individual-level annotations, which offer limited behavioral diversity and little supervision of the reasoning itself. We propose to learn behavioral reasoning from prediction markets, whose price trajectories record how populations respond to real-world events at scale. We introduce macro2mind, which trains a language model with GRPO using market signals. A social behavioral decomposition makes behavioral reasoning an explicit step of forecasting: the model infers representative groups of market participants, predicts how each interprets the news and updates its beliefs, reasons about their interactions, and aggregates these responses into a price. A hindsight-regret curriculum with difficulty-aware sampling focuses training on transitions where hindsight-identified groups substantially improve the forecast while prioritizing examples that remain learnable for the current policy. The learned reasoning applies to user simulation without further training. On SWM-Bench, macro2mind achieves state-of-the-art directional accuracy and correlation on Polymarket. Trained on market data, it transfers zero-shot to four user-simulation benchmarks (Humanual, OvertonBench, PRISM, and CAD) and has competitive performance among zero-shot methods. Used as a data generator, macro2mind also raises a downstream simulator's accuracy on unseen users by 15.5 points, outperforming data generated by its backbone by 13.2 points.
Yining Zhao, Bushi Liu, Haofei Yu +5
University of Illinois Urbana-Champaign · Independent Researcher
Simulating group-level user behavior enables scalable counterfactual evaluation of merchant strategies without costly online experiments. However, building a trustworthy simulator faces two structural challenges. First, information incompleteness causes reasoning-based simulators to over-rationalize when unobserved factors such as offline context and implicit habits are missing. Second, mechanism duality requires capturing both interpretable preferences and implicit statistical regularities, which no single paradigm achieves alone. We propose Policy-Guided Hybrid Simulation (PGHS), a dual-process framework that mines transferable decision policies from behavioral trajectories and uses them as a shared alignment layer. This layer anchors an LLM-based reasoning branch that prevents over-rationalization and an ML-based fitting branch that absorbs implicit regularities. Group-level predictions from both branches are fused for complementary correction. We deploy PGHS on Meituan with 101 merchants and over 26,000 trajectories. PGHS achieves a group simulation error of 8.80%, improving over the best reasoning-based and fitting-based baselines by 45.8% and 40.9% respectively.
Ziyang Chen, Renbing Chen, Daowei Li +4
LongCat Interaction Team, Meituan Inc. Beijing, China · Independent Researcher Changsha, China
Many decision-support settings require systems that adapt to individual users, but evaluation data for this problem remain limited. Existing benchmarks for user understanding often rely on simulated users or model-generated behavior, even though recent work cautions that model-based simulations can diverge systematically from human behavior. We introduce BehaviorBench, a benchmark for evaluating personalized decision modeling from real-world behavioral traces. BehaviorBench reconstructs wallet-level decision histories from observed public prediction-market and on-chain records, and organizes them into two complementary task layers: Belief prediction, which predicts a user's final revealed stance and confidence in a market, and Trade prediction, which predicts the direction and amount of individual transactions. Across 2,000 evaluation wallets, the benchmark contains 141,445 Belief instances and 1,485,972 Trade instances, with disjoint support pools for retrieval-based evaluation. We evaluate frontier and open-weight generative models under four history interfaces: no personalization, direct recent history, generated user profiles, and retrieved support-wallet evidence. Personalization improves Belief prediction more consistently than Trade prediction, model rankings change across task layers and metrics, and different history interfaces expose different failure modes. BehaviorBench provides an evaluation setting for studying whether personalized methods can use real-world behavioral evidence rather than simulated users alone.