Probe What-If Worlds Before You Ship: Persona Simulation for Real-World Decision Making
Abstract
User research and system testing reveal whether products and services meet user needs, yet recruiting specialized participants, designing controlled what-if comparisons, and repeating studies as designs evolve often bottleneck design iteration and data-driven decision-making. We present a framework and execution infrastructure that translates real-world industry inquiries into controlled experiments with simulated users. Our methodology formally defines the target cohort, experimental stimuli, and decision-relevant metrics, while the pipeline supports large-scale concurrent runs over cohort-stimulus-environment combinations (survey, web, chatbot, and OS app), auditable trial-level logging, cohort-level outcome aggregation, and cross-cohort and cross-variant statistical reports. Across twelve industry partner cases spanning five domains, we show how constructing hard-to-reach or what-if persona cohorts supports exploratory studies and distills simulated responses into testable design hypotheses. Cross-model comparisons on selected cases largely preserve primary conclusions, whereas perturbing or ablating persona attributes alters service outcomes, effect sizes, and inter-group differences, indicating that grounding target cohorts in detailed personas materially shapes simulation results. Together, these studies show that persona-enabled agents can test specified cohorts at design speed, diagnose systems under test, and deliver directional evidence that focuses problem definition, steers design and strategic decisions, and guides further study.
Figures & tables
| Application domains | Cases | Test environment | Principal finding |
|---|---|---|---|
| Health & well-being | C1: Music-wellbeing onboarding | Survey | Asking how users felt built more trust than emphasizing science or adopting a peer-companion tone. |
| C10: Healthcare access barriers | Survey | Cost barriers were widespread, but rural users faced greater travel barriers and adults aged 31–50 reported greater time pressure. | |
| Marketing & consumer behavior | C3: Subscription pricing and renewal | Survey | A cheaper introductory offer attracted interest, but recurring payments discouraged subscription. |
| C5: Remote-service marketplace | Survey | Users were more concerned about trusting the person performing the service than about its price. | |
| C9: Engagement-letter evaluation | Survey | Revising payment and liability terms improved acceptance, but distrust remained a barrier to signing. | |
| Technology & product design | C6: Professional-network signup | Web | Users understood how to sign up but hesitated over identity verification and document privacy. |
User Simulation
User simulation models human behavior and interaction to test, train, and evaluate AI systems with synthetic users.
See all 138 papers