stat.APNov 11, 2026

Can Model Answer Probabilities Represent Survey Populations? Conditional Prediction and Joint Validity in Jev Survey Simulation

Authors: Zengqing Wu, Chuan Xiao

Organizations: University of Osaka, Japan · Nagoya University, Japan

Abstract

Survey simulations require realistic combinations of answers, but predictive accuracy alone does not establish that a model can generate them. We evaluate the option probabilities returned by Jev, a TypeSafe decision model, using linked responses to 20 binary items from five archived survey waves. We compare population-only prediction, prediction given recorded answers, and sequential completion that uses generated answers to predict later responses. Joint comparisons hold the starting population information fixed, and two three-question cases are completed in both question orders. We evaluate predictive scores, answer frequencies, and associations against the survey records. Recorded answers can improve predictive scores while associations remain distorted. Large conditional errors also occur. For example, among men who described themselves as having traditional values, the model predicted that 16% would see men and women as similar in workplace abilities, compared with 56% in the survey. Sequential completion has mixed effects on joint validity, and reversing question order produces different joint distributions in both tested cases. Order checks and classical marginal bounds clarify what can be assessed before linked validation answers are available. The findings support validating the joint events a simulation uses and separating empirical constraints from model assumptions. The evidence concerns selected binary tasks and one model version.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Simulating Respondents, Not Single Questions: Coherent Survey Generation with Large Language Models

    Sep 28, 2026Ji Huang, Mengfei Li, Shuai ShaoLarge Language Model-Based Social SimulationConditional Generative Modeling

  2. Probable but Not Valid: A Psychometric Audit of Decision Models as Synthetic Survey Respondents

    Oct 10, 2026Mantas Lukauskas, Viktorija ŠarkauskaitėSynthetic Data GenerationSynthetic Data Evaluation