cs.CYAug 16, 2025

Integrating Large Language Models with Human Data to Design Personalized Decoy Nudges

Authors: Vladimir Maksimenko, Leonard Lee, Prateek Bansal

Organizations: Department of Civil and Environmental Engineering, National University of Singapore

Abstract

Although nudge strategies can influence behavior, their effectiveness varies across individuals, highlighting the need for personalized nudge design. We hypothesize that large language models (LLMs) can facilitate the development of individual-specific nudges while reducing the financial and time costs associated with human data collection by augmenting small human samples or by replacing human participants. Using the decoy effect as a test case, we evaluate these hypotheses in a preregistered field experiment involving 3,495 participants across five countries. We show that decoy parameters identified by LLMs as optimal for a given country generate stronger nudging effects than non-optimal parameters in the United States and, marginally, in Germany, but not in China, India, or Singapore. We find no evidence that LLM-informed personalized decoys produce a stronger nudging effect than country-level optimal decoys. Finally, we demonstrate that LLM-generated data can serve as informative priors for Bayesian choice models, improving predictive performance and parameter estimation with smaller human samples. At matched predictive performance, this approach yields estimated reductions of 37-75%\% in human sample size. Overall, our findings suggest that LLMs hold considerable promise as low-cost tools for piloting and optimizing personalized nudge strategies, particularly when complemented with a small sample of human participants.

Explore similar work

Nov 6, 2026

Nudging Decision Models: Do Digital Nudges Steer Fast, Generation-Free AI Agents?

LLM-based agents increasingly automate business and consumer tasks but are slow and expensive. Decision models, a new class that chooses among options without generating text, are far faster and less costly. Developers call them "System One models," referencing dual process theory. We ask whether such models can be steered by digital nudges online and whether, as the theory predicts, automatic nudges affect them more than reflective ones. We adapted an online shopping experiment, previously run with humans and LLM agents, to three decision models (Jev, Decision-1, d1). Across 600 runs per model, they chose a preselected default in 97.5-99.9% and a product with social influence information in 75-96% of choices, versus 47-49% in the control. All responded more strongly to the automatic nudge, the opposite of what has been reported for reasoning-capable LLM agents. We provide first evidence that non-generative decision models are susceptible to digital nudges in a System 1 pattern, implying agent-type-specific nudge effects.
Feb 17, 2026

This human study did not involve human subjects: Validating LLM simulations as behavioral evidence

A growing literature uses large language models (LLMs) as synthetic participants to generate cost-effective and nearly instantaneous responses in social science experiments. However, there is limited guidance on when such simulations support valid inference about human behavior. We contrast two strategies for obtaining valid estimates of causal effects and clarify the assumptions under which each is suitable for exploratory versus confirmatory research. Heuristic approaches seek to establish that simulated and observed human behavior are interchangeable through prompt engineering, model fine-tuning, and other repair strategies designed to reduce LLM-induced inaccuracies. While useful for many exploratory tasks, heuristic approaches lack the formal statistical guarantees typically required for confirmatory research. In contrast, statistical calibration combines auxiliary human data with statistical adjustments to account for discrepancies between observed and simulated responses. Under explicit assumptions, statistical calibration preserves validity and provides more precise estimates of causal effects at lower cost than experiments that rely solely on human participants. Yet the potential of both approaches depends on how well LLMs approximate the relevant populations. We consider what opportunities are overlooked when researchers focus myopically on substituting LLMs for human participants in a study.
Mar 8, 2026

Evaluating LLMs as Human Surrogates in Controlled Experiments

Large language models (LLMs) are increasingly used to simulate human responses in behavioral research, yet it remains unclear when LLM-generated data support the same experimental inferences as human data. We evaluate this by directly comparing off-the-shelf LLM-generated responses with human responses from a canonical survey experiment on accuracy perception. Each human observation is converted into a structured prompt, and models generate a single 0--10 outcome variable without task-specific training; identical statistical analyses are applied to human and synthetic responses. We find that LLMs reproduce several directional effects observed in humans, but effect magnitudes and moderation patterns vary across models. Off-the-shelf LLMs therefore capture aggregate belief-updating patterns under controlled conditions but do not consistently match human-scale effects, clarifying when LLM-generated data can function as behavioral surrogates.