LLM Agents as Social Scientists: A Human-AI Collaborative Platform for Social Science Automation

Lei Wang; Yuanzi Li; Jinchao Wu; Heyang Gao; Xiaohe Bo; Xu Chen; Ji-Rong Wen

cs.AIApr 2, 2026

LLM Agents as Social Scientists: A Human-AI Collaborative Platform for Social Science Automation

Authors: Lei Wang, Yuanzi Li, Jinchao Wu, Heyang Gao, Xiaohe Bo, Xu Chen, Ji-Rong Wen

Organizations: Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China

Abstract

Traditional social science research often requires designing complex experiments across vast methodological spaces and depends on real human participants, making it labor-intensive, costly, and difficult to scale. Here we present S-Researcher, an LLM-agent-based platform that assists researchers in conducting social science research more efficiently and at greater scale by "siliconizing" both the research process and the participant pool. To build S-Researcher, we first develop YuLan-OneSim, a large-scale social simulation system designed around three core requirements: generality via auto-programming from natural language to executable scenarios, scalability via a distributed architecture supporting up to 100,000 concurrent agents, and reliability via feedback-driven LLM fine-tuning. Leveraging this system, S-Researcher supports researchers in designing social experiments, simulating human behavior with LLM agents, analyzing results, and generating reports, forming a complete human-AI collaborative research loop in which researchers retain oversight and intervention at every stage. We operationalize LLM simulation research paradigms into three canonical reasoning modes (induction, deduction, and abduction) and validate S-Researcher through systematic case studies: inductive reproduction of cultural dynamics consistent with Axelrod's theory, deductive testing of competing hypotheses on teacher attention validated against survey data, and abductive identification of a cooperation mechanism in public goods games confirmed by human experiments. S-Researcher establishes a new human--AI collaborative paradigm for social science, in which computational simulation augments human researchers to accelerate discovery across the full spectrum of social inquiry.

Figures & tables

Figure 1: Overview of S-Researcher. (a) The complete workflow of S-Researcher: users input their research topics, after which simulation scenarios are automatically constructed, executed, and summarized into comprehensive reports. Researchers can intervene at every stage. (b) Summary of three case studies organized by reasoning paradigm: induction, deduction, and abduction.

Figure 2: Platform capability validation of YuLan-OneSim. a , Human expert ratings for auto-generated code across eight social science domains. b , Error type distribution across domains. c , Ablation study on workflow scores: OneSim vs. w/o G-Valid. d , Code quality ablation: four variant comparison. e , Runtime scaling with agent count. f , Distributed vs. single-node deployment efficiency. g , Feedback-driven optimization trajectories for two backbone models (Qwen2.5-1.5B, Llama-3.2-1B) under SFT and DPO strategies.

Figure 3: Inductive paradigm: S-Researcher autonomously reproduces Axelrod’s cultural dissemination dynamics, confirming coexistence of local convergence and global polarization. a , Research question input to S-Researcher. b , Simulation protocol following the ODD standard. c , Experimental setup: 100 LLM agents on a $10\times 10$ grid, 5 cultural features $\times$ 5 values, 100 rounds, 3 replicates. d , Dual-metric time series: local convergence (blue, left axis) increases from $\sim$ 0.20 to $\sim$ 0.24 (+21.0%); cultural diversity (red, right axis) decreases from $\sim$ 1.0 to $\sim$ 0.65. Shaded areas show variability across replicates. e , Distribution of neighbor cultural similarity across rounds, with cumulative high-similarity proportion (sim $\geq$ 0.6) increasing from 12.0% to 50.0%. f , Cultural cluster composition over time: singleton agents decline while large clusters ( $4+$ ) grow from 0% to 38%. g , Dominant value share evolution across five cultural dimensions, rising from the uniform baseline (20%) to a range of 32-37%. h , Pairwise cultural similarity matrices at four time points (Rounds 1, 10, 50, 100), showing progressive sharpening of cultural boundaries.

Figure 4: Deductive paradigm: bottom-up classroom simulation independently recovers the empirically established dominance of expressive ability in teacher attention allocation. a , Research question input to S-Researcher. b , Simulation protocol following the ODD standard. c , Experimental setup: 221 simulated classrooms, 5,525 student agents with profiles from CEPS, 30 rounds $\times$ 3 replicates. d , Spearman $\rho$ between simulated and empirical CEPS attention rankings ( $\uparrow$ ): Expression ( $0.152$ ) $>$ Merit ( $0.122$ ) $>$ Elite ( $0.113$ ). e , Root-mean-square error ( $\downarrow$ ): Expression ( $0.846$ ) yields the best fit. f , Convergence of $\rho$ over 30 simulation rounds; hypothesis ranking stabilizes after round 5. g , Independent validation via CEPS regression: communicative ability ( $\beta=0.349$ , $R^{2}=12.1\%$ ) explains substantially more variance than academic achievement or SES, confirming the simulation-derived ordering. h , Attention dynamics comparison: transition matrices from simulation (early vs. late phase) and CEPS two-wave panel data (Wave 1 vs. Wave 2) both exhibit diagonal-dominant persistence; simulation reproduces the qualitative structure but overestimates rigidity (e.g., Low $\to$ High: $1.8\%$ vs. $16.6\%$ ), consistent with its role as a mechanism-isolating experiment.

Figure 5: Abductive paradigm: counterfactual decomposition of follower cooperation in public goods games reveals behavioral anchoring as the dominant mechanism and uncovers an unexpected forced

>

voluntary effect. a , Research question: what causal mechanisms drive follower cooperation when leaders contribute first? b , Simulation protocol following the ODD standard. c , Experimental setup:

2\times 3

between-subjects design (voluntary/forced

\times

low/medium/high contribution), 100 agent followers per condition, 3 replicates; parallel human experiment (

N=120

). d , LLM agent simulation box plots (

N_{F}=100

n=3

): follower contributions under voluntary (blue) and forced (orange) conditions across leader contribution levels; diamonds mark means. Forced conditions elicit higher contributions at every level. e , Human-agent alignment scatter plot across 6 conditions, Pearson

r=0.915

. f , Human experiment box plots (

N=120

): humans show the same forced

>

voluntary pattern, with the gap widening at medium and high contribution levels. g , Effect size comparison forest plot: both agents and humans show significant effects of leader contribution (

\beta_{\text{agent}}=0.794

\beta_{\text{human}}=0.491

) and decision mechanism (

\beta_{\text{agent}}=0.104

\beta_{\text{human}}=0.251

), with leader contribution consistently the dominant factor.

Figure 6: Cross-paradigm expert evaluation of S-Researcher outputs. a , Radar chart comparing expert scores across 12 evaluation dimensions (grouped into four categories: Scene Design, Experiment Design, Result Analysis, and Report Writing) for each experiment paradigm. Scores are on a 1–5 Likert scale (3 reviewers per paradigm). The inductive paradigm (3.89/5) and abductive paradigm (3.82/5) receive higher overall ratings than the deductive paradigm (3.47/5). All three paradigms score highest on Feasibility and Relevance, while Accuracy and Insight show the most room for improvement. b , Word cloud of reviewer-identified strengths: dominant themes include sound conclusions, clear research questions, logical flow, and appropriate analytical methods. c , Word cloud of reviewer-identified weaknesses: recurring concerns include insufficient literature engagement, shallow discussion, lack of conceptual depth, and incomplete operationalization of key constructs.

Explore similar work

Jun 11, 2026

AgentSociety 2: An Integrated Research Environment for Executable Social Science

AI scientist systems are beginning to automate parts of scientific research, but social science poses a distinct challenge: its objects of inquiry are not merely datasets or laboratory protocols, but integrated social processes involving situated participants, interaction contexts, interventions, and outcomes. Yet a critical link is missing: existing systems either assist isolated research tasks or simulate agents as experimental subjects, leaving the research workflow and simulated society decoupled. Here we introduce AgentSociety 2, an Integrated Research Environment for executable social science. It couples two roles of LLM agents in the same runtime: AI social scientists that coordinate literature grounding, hypothesis generation, experiment design, simulation execution, result interpretation, and manuscript drafting; and silicon participants that generate behavioral responses within configurable social environments. This dual-role design turns hypotheses into auditable agent behaviors, environment rules, interventions, and measurements, thereby supporting an end-to-end workflow. Across seven illustrative studies spanning micro-level social-science laboratory experiments, meso-level dynamics in social media, and macro-level urban scenarios, we demonstrate its capacity to support diverse disciplinary questions, reproduce major qualitative patterns from prior studies, identify informative deviations, and enable large-scale simulations through optimized agent-environment interactions. By preserving human researchers' high-level agency while delegating procedural orchestration to agentic systems, it provides a human-in-the-loop and controllable infrastructure for next-generation computational social science, with broader applications in scalable computational social experimentation and AI-enabled social governance platforms.

Jinghua Piao, Jun Zhang, Haoyu Huang +16

Department of Electronic Engineering, Tsinghua University · School of Environment, Tsinghua University · School of Public Policy and Management, Tsinghua University +2

Multi-Agent LLM Systems AI Agents for Scientific Discovery Human Behavior Simulation Computational Social Science Large Language Model-Based Social Simulation

Ask PDF

May 28, 2026

EASE Configuration Facilitates A Reproducible Science of LLM Social Simulations

LLMs are increasingly deployed to simulate social interactions, yet many of the existing simulators remain ad hoc and monolithic. This lack of architectural standardization prevents reproducible research and complicates downstream evaluation. We advance a rigorous science of LLM-based multi-agent simulation by modularizing core components into Environments, Agents, Simulation engines, and Evaluation metrics (EASE). We demonstrate the utility of EASE configuration by wrapping it in an experimental study schema for orchestrating workflows centered around answering explicit research questions in generated scenarios. We contribute SiliSocS, an open-source, research-ready Silicon Society Sandbox implementing a study-structured EASE configuration to enable highly configurable and reproducible LLM-based social simulations. Using SiliSocS and EASE, we present three case studies, showcasing the system's comprehensive assessment of existing questions, ability to dive deeper into complex questions, and elaboration of existing studies, respectively. Together, these case studies highlight the limitations of current modeling approaches and isolate the impacts of design choices on key results.

Sneheel Sarangi, Maximilian Puelma Touzel, Aurélien Bück-Kaeffer +3

McGill University · Mila - Quebec Artificial Intelligence Institute · Université de Montréal +1

Multi-Agent LLM Systems Human Behavior Simulation Agent-Based Modeling Large Language Model-Based Social Simulation

Ask PDF

Jun 8, 2026

MASS: Deep Research for Social Sciences with Memory-Augmented Social Simulation

Deep Research agents powered by Large Language Models (LLMs) have exhibited extraordinary potential in automated paper writing tasks. However, existing systems rely heavily on literature retrieval and synthesis through internet and local knowledge bases, often resulting research in lacking insight and creativity in social science. To address this issue, we propose "Memory-Augmented Social Simulation (MASS)", an innovative paradigm that leverages highly realistic and research-oriented social simulations to enhance the creativity and empirical founding of LLMs-generated research. Specifically, MASS integrates three core components: dynamic goal-path planning with multi-level social norm restraint to guide the simulation, a multi-disciplinary behavior dataset for agent memory cold-start, and a structured forgetting mechanism inspired by the Ebbinghaus curve. Together, these ensure simulation authenticity and provide a robust empirical foundation for generating innovative scholarly papers. Experimental results demonstrate the effectiveness of our method, showing a 6.81% improvement in generation overall quality over foundation LLMs and 17.19% gain in Insight over strong baselines.

Yongrui Liu, Deyi Xiong

The International Joint Institute of Tianjin University, Fuzhou, Tianjin University, China · TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China

Human Behavior Simulation Large Language Model-Based Social Simulation Large Language Model-Guided Scientific Discovery Memory-Augmented Agents Autonomous Research Agents

Ask PDF