cs.AISep 12, 2026

Synthetic Data in Marketing Research: How to Evaluate and When to Trust

Authors: Oded Netzer, Rajan Sambandam

Organizations: Arthur J. Samberg Professor of Business, Columbia Business School · President, TRC Insights

Abstract

Debate over synthetic data in marketing research has polarized between claims that large language models (LLMs) make human respondents obsolete and calls to avoid them entirely. We argue that both positions obscure the more useful question: not whether synthetic respondents work, but when. Building on Brand, Israeli, and Ngwe (2026), we make three contributions. First, we distinguish three types of synthetic data (ungrounded LLM responses, segment-level personas, and individual-level digital twins) and map each to the decisions it can support. Second, we develop a taxonomy of four families of accuracy measures and suggest that the wide range of reported twin accuracy, from near-perfect to near-chance, largely reflects differences in what is being measured rather than in method quality. Aggregate measures often perform well even when little information is supplied to the LLM, and can mask a complete absence of respondent-level differentiation. Third, we introduce the forgotten question problem, in which a question is omitted from a fielded study, as a setting for twin-based augmentation of existing data. We propose an ex-ante answerability diagnostic that requires no ground truth: the R^2 of a random forest predicting twin outputs from the data used to construct the twins. Across 108 attitude questions from a nationally representative survey (N = 3,063), screening at R^2 above 0.7 raises the mean twin-human individual-level correlation by 15% and reduces the share of poorly answered questions from 25.9% to 4.3%. Embedding similarity and experienced-researcher judgment provide correlated but weaker screens.

Explore similar work

CardsList
  1. Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?

    Jun 3, 2026Leonard Kinzinger, Jochen HartmannComputational Social ScienceUser Modeling

  2. Stochastic Parrots or Singing in Harmony? Testing Five Leading LLMs for their Ability to Replicate a Human Survey with Synthetic Data

    Feb 10, 2026Jason Miklian, Kristian Hoelscher, John E. KatsosHuman Behavior SimulationSynthetic Data Generation

  3. Synthetic Consumer Insight Generation with Large Language Models

    Jul 7, 2026Stephen L. France, Pia. A. AlbinssonLLM PromptingSynthetic Data Generation