Deep Neural Mixed Models for Nonlinear Longitudinal Prediction with Subject-Specific Heterogeneity

Mahmud Hasan; Niloofar Ramezani; Hrishikesh Chakraborty

stat.MEOct 11, 2026

Deep Neural Mixed Models for Nonlinear Longitudinal Prediction with Subject-Specific Heterogeneity

Authors: Mahmud Hasan, Niloofar Ramezani, Hrishikesh Chakraborty

Organizations: Department of Mathematics and Statistics, Penn State Altoona, PA, USA · Department of Biostatistics, Virginia Commonwealth University, VA, USA · Department of Biostatistics & Bioinformatics, Duke University, NC, USA

Abstract

Longitudinal health data often exhibit within-subject correlation, subject-specific heterogeneity, and complex nonlinear predictor--response relationships that may not be adequately captured by conventional mixed models. We propose a Deep Generalized Neural Network Mixed Model (DGNMM) that extends the GLMM framework by using a deep neural network to learn nonlinear population-level relationships while incorporating directly trainable, regularized subject-specific effects to capture individual heterogeneity. The model is estimated using prediction-oriented stochastic optimization with separate regularization of the neural-network parameters and subject-specific effects. This formulation allows us to investigate when increased nonlinear representation capacity provides additional predictive benefit in the presence of subject-specific heterogeneity. A simulation study varies population-level nonlinearity and subject-specific heterogeneity and compares DGNMM with GLMM, artificial neural network (ANN), Maity--Pal (MP), and GNMM alternatives for continuous and binary longitudinal outcomes. In the real-data application, DGNMM achieved the lowest MSPE for continuous depression and anxiety prediction, while alternative models remained competitive for binary outcomes. Exploratory subject-level analyses showed heterogeneous predictive gains across individuals, but the magnitude of the estimated subject-specific effects was not significantly associated with predictive gain. Overall, increased nonlinear representation capacity was most beneficial for continuous prediction under nonlinear response structures, while greater subject-specific heterogeneity did not systematically yield larger predictive gains.

Figures & tables

Figure 1: Architecture of the proposed Deep Generalized Neural Network Mixed Model (DGNMM). The population-level nonlinear component $f_{\theta}^{Deep}(\mathbf{x}_{ij})$ is represented by a multilayer feed-forward neural network, while a regularized subject-specific latent effect $e_{i}$ captures persistent individual deviation from the population-level response surface. The resulting predictor is $\eta_{ij}=f_{\theta}^{Deep}(\mathbf{x}_{ij})+e_{i}$ , with the conditional mean obtained through the inverse link function. Both components are estimated jointly for longitudinal prediction.

\eta_{ij}=f_{\theta}^{Deep}(\mathbf{x}_{ij})+e_{i}.

Algorithm 1 Parameter Estimation Procedure for DGNMM

Figure 2: MSPE comparison of DGNMM, GNMM-1, GNMM-2, ANN, GLMM, and MP across different levels of population-level nonlinearity

\gamma

and subject-specific heterogeneity

\tau

for continuous outcomes. Results are presented for competing neural-network specifications with 5 nodes in the first hidden layer. The proposed DGNMM employs a deep neural network architecture with hidden layers of sizes 64, 32, and 16. For each combination of

\gamma

and

\tau

, the final observation from each subject was reserved as a holdout test sample, and prediction performance was evaluated using MSPE over 500 simulation replications. Lower MSPE indicates better predictive performance.

Figure 3: MSPE comparison of DGNMM, GNMM-1, GNMM-2, ANN, GLMM, and MP across different levels of population-level nonlinearity

\gamma

and subject-specific heterogeneity

\tau

for continuous outcomes. Results are presented for competing neural-network specifications with 10 nodes in the first hidden layer. The proposed DGNMM employs a deep neural network architecture with hidden layers of sizes 64, 32, and 16. For each combination of

\gamma

and

\tau

Figure 4: AUC comparison of DGNMM, GNMM-1, GNMM-2, ANN, GLMM, and MP across different levels of population-level nonlinearity

\gamma

and subject-specific heterogeneity

\tau

for binary outcomes. Results are presented for competing neural-network specifications with 5 nodes in the first hidden layer. The proposed DGNMM employs a deep neural network architecture with hidden layers of sizes 64, 32, and 16. For each combination of

\gamma

and

\tau

, the final observation from each subject was reserved as a holdout test sample, and prediction performance was evaluated using AUC over 500 simulation replications. Larger AUC indicates better discrimination.

Figure 5: AUC comparison of DGNMM, GNMM-1, GNMM-2, ANN, GLMM, and MP across different levels of population-level nonlinearity

\gamma

and subject-specific heterogeneity

\tau

for binary outcomes. Results are presented for competing neural-network specifications with 10 nodes in the first hidden layer. The proposed DGNMM employs a deep neural network architecture with hidden layers of sizes 64, 32, and 16. For each combination of

\gamma

and

\tau

Outcome	Method	MSPE
Depression	DGNMM	0.324665
	GNMM-1	0.731966
	GNMM-2	0.748917
	ANN	0.986315
	GLMM Main Effects	0.546328
	GLMM + 3 Interactions	0.608985

Table 1: MSPE comparison of DGNMM, GNMM-1, GNMM-2, GLMM, ANN, and MP for depression and anxiety prediction. Lower MSPE indicates better predictive performance.

Outcome	Method	AUC
Depression	DGNMM	0.989
	GNMM-1	0.982
	GNMM-2	0.961
	ANN	0.870
	GLMM Main Effects	0.875
	GLMM + 3 Interactions	1.000

Table 2: AUC comparison of DGNMM, GNMM-1, GNMM-2, GLMM, ANN, and MP for classifying high versus low depression and anxiety symptoms. Larger AUC values indicate better classification performance.

Panel of Figure 6 — Figure 6: Relationship between the magnitude of the estimated DGNMM subject-specific effect $|\hat{e}_{i}|$ and subject-specific predictive gain relative to the main-effects GLMM for (a) depression and (b) anxiety. Positive predictive gain indicates a smaller squared prediction error for DGNMM, whereas negative gain indicates a smaller squared prediction error for GLMM. The Spearman correlations between $|\hat{e}_{i}|$ and predictive gain were $\rho=-0.346$ ( $p=0.206$ ) for depression and $\rho=-0.407$ ( $p=0.132$ ) for anxiety. Neither association was statistically significant.

Mixed-Effects Models

Mixed-effects models separate population-wide patterns from group- or subject-specific variation in clustered and repeated-measures data.

See all 20 papers

Neural Generalized Mixed-Effects Models
Apr 13, 2026Yuli Slavutsky, Sebastian Salazar, David M. BleiMixed-Effects Models Generalized Linear Models
Department of Statistics, Columbia University, New York, USA · Department of Computer Science, Columbia University, New York, USA
Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data
Aug 6, 2026Nina van Gerwen, Dimitris Rizopoulos, Manon Hillegers +2Learning with Missing Data Mixed-Effects Models
Department of Biostatistics, Erasmus University Medical Center, the Netherlands · Department of Epidemiology, Erasmus University Medical Center, the Netherlands · Department of Child Psychiatry, Erasmus University Medical Center, the Netherlands +1
Neural ODE enhanced linear mixed effect models for estimating complex association patterns of time-varying covariates with the marker trajectory
Aug 30, 2026Zhe Aurore Li, Quentin Clairon, Cécilia Samieri +3Mixed-Effects Models
Univ. Bordeaux, Inserm, BPH, UMR 1219, 33000 Bordeaux, France · INRIA SISTM, 33000 Bordeaux, France · Vaccine Research Institute, 94010 Creteil, France

Apr 13, 2026J/K move · Enter open · S save

Neural Generalized Mixed-Effects Models

Yuli Slavutsky, Sebastian Salazar, David M. Blei

Department of Statistics, Columbia University, New York, USA · Department of Computer Science, Columbia University, New York, USA

Mixed-Effects Models Generalized Linear Models Latent Variable Models

Open paper AskPDF arXiv

Generalized linear mixed-effects models (GLMMs) are widely used to analyze grouped and hierarchical data. In a GLMM, each response is assumed to follow an exponential-family distribution where the natural parameter is given by a linear function of observed covariates and a latent group-specific random effect. Since exact marginalization over the random effects is typically intractable, model parameters are estimated by maximizing an approximate marginal likelihood. In this paper, we replace the linear function with neural networks. The result is a more flexible model, the neural generalized mixed-effects model (NGMM), which captures complex relationships between covariates and responses. To fit NGMM to data, we introduce an efficient optimization procedure that maximizes the approximate marginal likelihood and is differentiable with respect to network parameters. We show that the approximation error of our objective decays at a Gaussian-tail rate in a user-chosen parameter. On synthetic data, NGMM improves over GLMMs when covariate-response relationships are nonlinear, and on real-world datasets it outperforms prior methods. Finally, we analyze a large dataset of student proficiency to demonstrate how NGMM can be extended to more complex latent-variable models.

Abstract

Mixed-Effects Models

Explore similar work

Neural Generalized Mixed-Effects Models

Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data

Neural ODE enhanced linear mixed effect models for estimating complex association patterns of time-varying covariates with the marker trajectory