Fused Multinomial Logistic Regression Utilizing Summary-Level External Machine-learning Information
Organizations: Department of Statistics, National Cheng Kung University, Tainan, 701, Taiwan · Department of Statistics, University of Wisconsin-Madison, Madison, 53706, USA
Abstract
In many modern applications, a carefully designed primary study provides individual-level data for interpretable modeling, while summary-level external information is available through black-box, efficient, and nonparametric machine-learning predictions. Although summary-level external information has been studied in the data integration literature, there is limited methodology for leveraging external nonparametric machine-learning predictions to improve statistical inference in the primary study. We propose a general empirical-likelihood framework that incorporates external predictions through moment constraints. An advantage of nonparametric machine-learning prediction is that it induces a rich class of valid moment restrictions that remain robust to covariate shift under a mild overlap condition without requiring explicit density-ratio modeling. We focus on multinomial logistic regression as the primary model and address common data-quality issues in external sources, including coarsened outcomes, partially observed covariates, covariate shift, and heterogeneity in generating mechanisms known as concept shift. We establish large-sample properties of the resulting fused estimator, including consistency and asymptotic normality under regularity conditions. Moreover, we provide mild sufficient conditions under which incorporating external predictions delivers a strict efficiency gain relative to the primary-only estimator. Simulation studies and an application to the National Health and Nutrition Examination Survey on multiclass blood-pressure classification.
Figures & tables
| Shift | Metric | Method | |||||||||||
| None | Bias | MLE | 0.010 | 0.025 | 0.025 | 0.011 | 0.003 | 0.012 | 0.010 | 0.005 | 0.007 | 0.041 | |
| FMLE | 0.010 | 0.022 | 0.023 | 0.011 | 0.003 | 0.008 | 0.014 | 0.002 | 0.021 | 0.026 | |||
| SD | MLE | 0.143 | 0.255 | 0.268 | 0.241 | 0.252 | 0.156 | 0.305 | 0.300 | 0.289 | 0.288 | ||
| FMLE | 0.143 | 0.257 | 0.269 | 0.243 | 0.252 | 0.146 | 0.226 | 0.227 | 0.212 | 0.213 | |||
| SE | MLE | 0.141 | 0.258 | 0.258 | 0.250 | 0.257 | 0.158 | 0.296 | 0.296 | 0.281 | 0.296 | ||
| Shift | Method | Class 1 | Class 2 | Class 3 | Class 1 | Class 2 | Class 3 |
| None | MLE | 0.0021 | 0.0021 | 0.0018 | 0.0021 | 0.0021 | 0.0018 |
| FMLE | 0.0019 | 0.0019 | 0.0009 | 0.0019 | 0.0020 | 0.0012 | |
| Mean | MLE | 0.0020 | 0.0021 | 0.0017 | 0.0020 | 0.0021 | 0.0017 |
| FMLE | 0.0019 | 0.0019 | 0.0009 | 0.0019 | 0.0020 | 0.0012 | |
| Variance | MLE | 0.0021 | 0.0021 | 0.0018 | 0.0021 | 0.0021 | 0.0018 |
| Category | Primary | External |
| Normal | 0.704 | 0.754 |
| Prehypertension | 0.153 | 0.134 |
| Hypertension | 0.146 | 0.112 |