Goodness-of-Fit Tests for Latent Class Models with Ordinal Categorical Data

Huan Qing

stat.MLFeb 25, 2026

Goodness-of-Fit Tests for Latent Class Models with Ordinal Categorical Data

Authors: Huan Qing

Organizations: School of Economics and Finance, Chongqing University of Technology, Chongqing, 400054, China

Abstract

Ordinal categorical data are widely collected in psychology, education, and other social sciences, appearing commonly in questionnaires, assessments, and surveys. Latent class models provide a flexible framework for uncovering unobserved heterogeneity by grouping individuals into homogeneous classes based on their response patterns. A fundamental challenge in applying these models is determining the number of latent classes, which is unknown and must be inferred from data. In this paper, we propose one test statistic for this problem. The test statistic centers the largest singular value of a normalized residual matrix by a simple sample-size adjustment. Under the null hypothesis that the candidate number of latent classes is correct, its upper bound converges to zero in probability. Under an under-fitted alternative, the statistic itself exceeds a fixed positive constant with probability approaching one. This sharp dichotomous behavior of the test statistic yields two sequential testing algorithms that consistently estimate the true number of latent classes. Extensive experimental studies confirm the theoretical findings and demonstrate their accuracy and reliability in determining the number of latent classes.

Figures & tables

$N$	$T_{K_{0}}$				Ratios
$N$	$K_{0}=1$ ( $H_{1}$ )	$K_{0}=2$ ( $H_{1}$ )	$K_{0}=3$ ( $H_{1}$ )	$K_{0}=4$ ( $H_{0}$ )	$r_{2}=\|T_{1}/T_{2}\|$	$r_{3}=\|T_{2}/T_{3}\|$	$r_{4}=\|T_{3}/T_{4}\|$
200	2.020 (0.171)	1.706 (0.147)	1.418 (0.149)	-0.037 (0.026)	1.19 (0.12)	1.21 (0.13)	255.42 (2952.98)
400	2.135 (0.167)	1.835 (0.147)	1.551 (0.153)	-0.024 (0.018)	1.17 (0.10)	1.19 (0.12)	157.51 (486.75)
600	2.214 (0.174)	1.891 (0.147)	1.620 (0.157)	-0.018 (0.014)	1.18 (0.10)	1.18 (0.12)	229.62 (676.48)
800	2.257 (0.167)	1.933 (0.152)	1.662 (0.153)	-0.017 (0.011)	1.17 (0.11)	1.17 (0.12)	388.22 (2547.19)
1000	2.278 (0.169)	1.956 (0.150)	1.690 (0.161)	-0.014 (0.010)	1.17 (0.10)	1.16 (0.12)	450.91 (2527.84)

Table 1: Behavior of

T_{K_{0}}

and ratios

r_{2},r_{3},r_{4}

under

H_{0}

(

K_{0}=4

) and under - fitted models (

K_{0}=1,2,3

) for true

K=4

. Values are mean (standard deviation) over 200 replications.

True $K$	Algorithm	Stopping $K_{0}$
		1	2	3	4	5	6	7	8	9	10
2	GoF-LCM	0.000	1.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
2	RGoF-LCM	0.000	1.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
3	GoF-LCM	0.000	0.000	1.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
3	RGoF-LCM	0.000	0.000	1.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
4	GoF-LCM	0.000	0.000	0.000	1.000	0.000	0.000	0.000	0.000	0.000	0.000

Table 2: Proportion of times GoF-LCM and RGoF-LCM stop at each candidate

K_{0}

for true

K=2,3,4,5,6

. Columns cover

K_{0}=1

10

Parameters				GoF-LCM	RGoF-LCM	Spec
$K$	$N$	$J$	$\delta$	Accuracy (S.E.)	Accuracy (S.E.)	Accuracy (S.E.)
1	200	60	0.1	1.000 (0.000)	1.000 (0.000)	1.000 (0.000)
1	200	60	0.2	1.000 (0.000)	1.000 (0.000)	1.000 (0.000)
1	200	60	0.3	1.000 (0.000)	1.000 (0.000)	1.000 (0.000)
1	200	100	0.1	1.000 (0.000)	1.000 (0.000)	1.000 (0.000)
1	200	100	0.2	1.000 (0.000)	1.000 (0.000)	1.000 (0.000)

Table 3: Accuracy (with standard error in parentheses) of GoF-LCM, RGoF-LCM, and the spectral thresholding method (Spec) for all parameter combinations.

Panel of Figure 1 — Figure 1: Accuracy (left) and running time (right) of GoF-LCM and RGoF-LCM for $K=8$ , $J=60$ , $\delta=0.3$ , with varying $N$ .

GoF-LCM: $\tau_{N}=N^{-\epsilon}$
$\epsilon$	Accuracy
0.1	1.000
0.2	1.000
0.3	1.000
0.4	1.000
0.5	0.995

Table 4: Sensitivity of GoF-LCM to

\epsilon

\tau_{N}=N^{-\epsilon}

and of RGoF-LCM to multiplier

a

\gamma_{N}=a\log N

Figure 2: Accuracy of GoF - LCM and RGoF - LCM $K=8$ , $N=600$ , $\delta=0.3$ , with varying $J$ .

Panel of Figure 3 — Figure 3: Test statistic $T_{K_{0}}$ (left) and ratio $r_{K_{0}}=|T(K_{0}-1)/T(K_{0})|$ (right) versus candidate number of latent classes $K_{0}$ for the BFPT dataset.

Latent Variable Models

Latent variable models infer hidden factors, classes, or states from observed data to explain structure, uncertainty, and variation.

See all 153 papers

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks
Oct 26, 2025Peiyu Li, Xiuxiu Tang, Si Chen +4LLM Evaluation Psychometric Validation
University of Notre Dame, Notre Dame, USA
Classifier Pooling for Modern Ordinal Classification
Mar 18, 2026Noam H. Rotenberg, Andreia V. Faria, Brian CaffoOrdinal Regression
Department of Biomedical Engineering, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USA. · Department of Radiology, School of Medicine, Johns Hopkins University, Baltimore, MD, USA. · 3*Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA.
On the Identifiability of Mixed Ordinal and Exponential Family Causal DAGs under Linear Parametric Models
Sep 16, 2026Sambit Mishra, Urbashi MitraStructural Causal Models Causal Discovery
Ming Hsieh Department of Electrical and Computer Engineering University of Southern California Los Angeles, CA 90089, USA

Oct 26, 2025J/K move · Enter open · S save

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks

Peiyu Li, Xiuxiu Tang, Si Chen, Ying Cheng+3

University of Notre Dame, Notre Dame, USA

LLM Evaluation Psychometric Validation Computerized Adaptive Testing Item Response Theory

Open paper AskPDF arXiv

Evaluating large language models (LLMs) typically requires thousands of benchmark items, making the process expensive, slow, and increasingly impractical at scale. Existing evaluation protocols rely on average accuracy over fixed item sets, treating all items as equally informative despite substantial variation in difficulty and discrimination. We introduce ATLAS, an adaptive testing framework based on Item Response Theory (IRT) that estimates model ability using Fisher information-guided item selection. ATLAS reduces the number of required items by up to 90% while maintaining measurement precision. For instance, it matches whole-bank ability estimates using only 41 items (0.157 MAE) on HellaSwag (5,600 items). We further reconstruct accuracy from ATLAS's ability estimates and find that reconstructed accuracies closely match raw accuracies across all five benchmarks, indicating that ability

θ

preserves the global performance structure. At the same time,

θ

provides finer discrimination within accuracy-equivalent models: among more than 3,000 evaluated models, 23-31% shift by more than 10 rank positions, and models with identical accuracies receive meaningfully different ability estimates. Code and calibrated item banks are available at https://github.com/Peiyu-Georgia-Li/ATLAS.git.

Abstract

Latent Variable Models

Explore similar work

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks

Classifier Pooling for Modern Ordinal Classification

On the Identifiability of Mixed Ordinal and Exponential Family Causal DAGs under Linear Parametric Models