Analytical Standard Errors for Exploratory Factor Solutions
Authors: Xingwei Hu, Caihong Hu, Cheng-Kuang Wu
Organizations: International Monetary Fund, Washington, DC, 20431, USA · Zhangjiajie College, Zhangjiajie, Hunan, 427000, China · Zhejiang Industry and Trade Vocational College, Wenzhou, Zhejiang, 325700, China
Inference for factor models is often hampered by the lack of tractable and accurate variance estimates, which can materially distort downstream analyses. In practice, uncertainty in the residual covariance matrix is frequently either ignored or addressed through computationally intensive resampling methods that tend to be unstable. This paper develops a unified analytical framework for inference in exploratory factor analysis under several widely used extraction rules, including least-squares, principal-factor, iterative principal-component, alpha, and image factoring. By treating these estimators as implicitly defined functions of the sample covariance matrix, we derive closed-form Jacobians that translate perturbations in the covariance matrix into changes in the resulting factor solutions. Combined with the delta method and consistent estimators of the sample covariance matrix, the proposed approach yields standard errors that are straightforward to compute and remain valid under non-Gaussianity, heteroskedasticity, and serial or cross-sectional dependence. Simulation evidence confirms that the analytical standard errors accurately capture finite-sample variability while avoiding both the instability of bootstrap procedures and the restrictive assumptions underlying Fisher information-based inference. An application to a factor-augmented structural vector autoregressive (SVAR) model further demonstrates how accounting for this source of uncertainty can substantially affect impulse-response inference. Taken together, the results provide a practical and general tool for propagating estimation uncertainty in settings where factor extraction serves as an intermediate step.
Factor-based Structural Equation Modeling (SEM) relies on likelihood-based estimation assuming a nonsingular sample covariance matrix, which breaks down in small-sample settings with p>n. To address this, we propose a novel estimation principle that reformulates the covariance structure into self-covariance and cross-covariance components. The resulting framework defines a likelihood-based feasible set combined with a relative error constraint, enabling stable estimation in small-sample settings where p>n for sign and direction. Experiments on synthetic and real-world data show improved stability, particularly in recovering the sign and direction of structural parameters. These results extend covariance-based SEM to small-sample settings and provide practically useful directional information for decision-making.
In partially exploratory factor analysis (PEFA), the loading structure and factor numbers are weakly specified. The regularized variational approximation for partially confirmatory factor analysis (PCFA VA) recovers this structure via Bayesian variable selection, using spike and slab priors to assign inclusion probabilities to unspecified loadings. This research introduces a post selection assessment framework for this approach. We convert converged solutions into covariance models using either hard selection (thresholding probabilities into a sparse pattern) or soft selection (retaining them as weights for effective parameter counts). We derive the resulting degrees of freedom, absolute fit diagnostics (RMSEA, SRMR, CFI, TLI), and relative criteria (AIC, BIC, ELBO). To determine factor numbers, we propose a scale free gain rule with a sustained drop guard. Simulations show absolute indices successfully track loading recovery and flag under factoring. While raw criteria over factor, our gain rule accurately recovers true dimensionality, with the ELBO variant proving most robust. Finally, a 100 item PID 5 example demonstrates that our model fits better than a confirmatory 25 facet model and concordantly recovers major structures across disjoint specifications.
Whether an additional general dimension is necessary beyond correlated first-order factors is a property of the population covariance matrix, not of any estimator or design. This research establishes when that property can be decided. Where the general and group loadings are proportional within every cluster the bifactor structure is covariance-equivalent to correlated factors, so no sample size separates them (Proposition 1); where that proportionality fails in every cluster, three items per cluster and some mild regularities leave no K-factor model with diagonal uniquenesses able to reproduce the covariance matrix (Theorem 1); and between them lies a mixed boundary, located numerically here and turning on cluster resistance. Distinguishability is therefore graded, measured by the population distance to the K-factor class. Because that question is conditional on a first-order structure which is itself uncertain, a two-step procedure is developed within partially exploratory factor analysis, delivering a structure only when it reproduces across adjacent counts and treating non-delivery as legitimate. Simulation shows that a unanimous count can accompany a structure that fails to reproduce, and that absorbed local dependence can imitate a general factor, the error growing with sample size while stability indicators stay clean. Four empirical datasets illustrate the possible outcomes.