cs.LGOct 26, 2026

When Respondents Are Scarce: LLM-Derived Semantic Priors for Bayesian Confirmatory Factor Analysis

Authors: Shaohuai Feng, Mohd Wira Mohd Shafiei, Mohd Sukri Shafie, Siyang Tian

Organizations: Centre for Global Sustainability Studies (CGSS), Universiti Sains Malaysia, Minden, Penang, Malaysia. · Centre for Education and Training in Renewable Energy, Energy Efficiency and Green Technology (CETREE), Universiti Sains Malaysia, Minden, Penang, Malaysia. · Centre for Instructional Technology and Multimedia, Universiti Sains Malaysia, Minden, Penang, Malaysia.

Abstract

Across social and behavioral research, measuring latent constructs can be challenging when target populations are small or difficult to reach. Recruiting additional respondents may be costly or infeasible, leaving researchers with limited data for estimating questionnaire factor structures. Yet item wording contains structural expectations before data collection, and whether these can improve small-sample estimation without replacing empirical validation remains unclear. We introduce the Semantic-Empirical Updating Framework (SEUF), which converts large language model paraphrases and embeddings, aligned with hypothesized item-factor assignments, into prespecified priors for Bayesian confirmatory factor analysis. We evaluated SEUF using a 38-item survey of 617 industrial-park stakeholders and two public surveys, FEVS and RIASEC. Semantic structure persisted after wording controls across all three instruments. With 50 respondents, SEUF achieved loading precision comparable to conventional CFA with 70-82 respondents, equivalent to 29-39% fewer respondents at the specified precision benchmark. Results showed lower loading errors and improved credible-interval coverage relative to conventional CFA in both public surveys. Gains were greatest when respondent data were limited, highlighting the value of semantic priors under information scarcity. By harnessing structural information already embedded in questionnaires, SEUF advances a more resource-efficient approach to measurement and opens new directions for study design and empirical inference in hard-to-reach populations.

Explore similar work

CardsList
  1. semanticfa: An R Package for Response-Free Semantic Factor Analysis of Psychometric Scales

    Oct 27, 2026Devon Yanitski, Chris WestburyPsychometric ValidationLatent Variable Models

  2. Bayesian Elicitation with LLMs: Model Size Helps, Extra "Reasoning" Doesn't Always

    Apr 2, 2026Luka Hobor, Mario Brcic, Mihael Kovac +1Probability Elicitation from Language ModelsLLM Uncertainty Estimation