The prevalence of missing values in data science poses a substantial risk to any further analyses. Despite a wealth of research, principled nonparametric methods to deal with general non-monotone missingness are still scarce. Instead, ad-hoc imputation methods are often used, for which it remains unclear whether the correct distribution can be recovered. In this paper, we propose FLOWGEM, a principled iterative method for generating a complete dataset from a dataset with values Missing at Random (MAR). Motivated by convergence results of the ignoring maximum likelihood estimator, our approach minimizes the expected Kullback-Leibler (KL) divergence between the observed data distribution and the distribution of the generated sample over different missingness patterns. To minimize the KL divergence, we employ a discretized particle evolution of the corresponding Wasserstein Gradient Flow, where the velocity field is approximated using a local linear estimator of the density ratio. This construction yields a data generation scheme that iteratively transports an initial particle ensemble toward the target distribution. Simulation studies and real-data benchmarks demonstrate that FLOWGEM achieves state-of-the-art performance across a range of settings, including the challenging case of non-monotone MAR mechanisms. Together, these results position FLOWGEM as a principled and practical alternative to existing imputation methods, and a decisive step towards closing the gap between theoretical rigor and empirical performance.
Figures & tables
Figure 1: Three data matrices with missing values, each with three different patterns. The matrix on the left shows monotone missingness, while the middle and right show cases with non-monotone missingness. The middle matrix is the one used in the Example of Section 4 .
Figure 2: Standardized Energy distance in log-scale (left) and quantile estimate (right) for the simulated example with uniform distribution and n=2000 , repeated over B=20 times. The solid blue line on the right indicates the true quantile value of the uncontaminated distribution, while the dashed line shows the population quantile when missing values are ignored. Our method, FLOWGEM, clearly outperforms its competitors, minimizing the energy distance and estimating the quantile most accurately. We note that MICE also performs well, but unlike FLOWGEM, it lacks theoretical justification. Some values are cut off for readability: the energy distance for MIRI ranges from 6 to 4⋅10148 with a median of 2⋅1046 , and its quantile estimates reach as low as −2.95 .
Dataset
Ours
MICE
GAIN
Hyperimp.
MIRI
Bayes
MissDiff
NewImp
scm1d
26.26
25.53
>103
31.17
>103
–
457.23
>103
scm20d
17.65
21.40
>103
31.45
>103
–
226.51
>103
pumadyn32nm
11.41
11.20
>103
116.12
>103
–
230.36
>103
parkinsons
22.98
29.21
>103
31.53
>103
–
44.99
>103
allergens
34.29
209.30
205.81
30.05
125.09
–
89.17
>103
concrete
5.84
22.28
29.84
12.79
20.09
>103
20.21
>103
Table 1: Standardized energy distance between generated and held-out samples across ten real-world datasets (lower is better). “–” indicates the method exceeded the runtime limit. Values larger than 103 are truncated and displayed as >103 . For each dataset, the best result is marked in bold, while the second best result is underlined. Our method is consistently either best or second best.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Name
Rows
Cols
n
d
Source
scm1d
9803
296
4901
272
Vanschoren et al. (2013)
scm20d
8966
77
4482
58
Vanschoren et al. (2013)
pumadyn32nm
8192
33
4095
33
UCI
parkinsons
5875
21
2937
18
UCI
allergens
2351
112
1175
49
Société Française d’Allergologie and AllergoBioNet (2024)
concrete
1030
9
514
8
Yeh (1998)
Appendix
Table 2: Real data examples considered in Section 4.2 . For each dataset, we report the number of rows, the number of columns, the actual number of observations n and dimensions d used in the analysis, as well as their source. UCI refers to Lichman (2013) and was downloaded from https://github.com/treforevans/uci_datasets/tree/master .
Method
Mean
Std. Dev.
FLOWGEM
22.85
1.36
MICE
0.56
0.00
GAIN
2.80
0.64
Hyperimpute
8.68
5.63
MIRI
1611.91
2588.70
Bayes
2038.28
520.23
Appendix
Table 3: Average computation time (in seconds) per method for the simulation study of Section 4.1 with n=2000 , d=3 , and uniform distribution. Mean and standard deviation over B=20 replications are reported.
Figure 3: Standardized Energy distance in log-scale (left) and quantile estimate (right) for the simulated example with Gaussian distribution, n=2000 , and repeated over B=20 times. The blue line on the right indicates the true quantile value of the uncontaminated distribution. Our method, FLOWGEM, belongs to one of the best methods, minimizing the energy distance and estimating the quantile most accurately. We note that both MICE and the Bayes method essentially use Gaussian regression and Gaussian approximations respectively, giving them a somewhat unfair edge in this example. Despite this edge, it turns out that increasing T to 1500 further boosts FLOWGEM and closes the gap between our method and MICE or Bayes, but we did not include this result here.
Figure 4: Scatter plots of the first two dimensions of the generated samples for each method, for a single replication of the simulation study in Section 4.1 with n=2000 , d=3 , and uniform distribution. The black square indicates the support [0,1]2 of the true distribution.
Figure 5: Scatter plots of the first two dimensions of the generated samples for each method, for a single replication of the simulation study in Section 4.1 with n=2000 , d=3 , and Gaussian distribution. We note that both MICE and the Bayes method essentially use Gaussian regression and Gaussian approximations respectively, giving them a somewhat unfair edge in this example.
Learning with missing data develops methods to estimate, predict, and make decisions when variables, labels, modalities, or time points are unobserved.
Flow matching assumes fully observed training data, which many real-world applications rarely provide. We study Missing-Data Flow Matching, which treats the missing coordinates of training samples as latent variables and averages the flow matching loss over the values they could take. We first prove that the oracle correction is exact: completing from the true pattern-conditioned posterior recovers the complete-data flow-matching objective under any missingness mechanism. Under missing completely at random (MCAR) and missing at random (MAR), this posterior equals the ordinary complete-data conditional, while under missing not at random (MNAR) it generally depends on the missingness mechanism and may require additional structure to identify. Our finite-sample analysis then answers design questions that the algorithm leaves open. Missingness transfers estimator variance rather than adding it, and for the fixed-parameter loss estimator one completion per example matches complete-data variance and is optimal under a fixed evaluation budget. A learned completion model contributes population completion bias, which we bound by its expected conditional Wasserstein distance to the true completion law. Experiments numerically validate the theoretical predictions, show that deterministic rather than frozen imputation reduces generated conditional variability, and place our method alongside strong classical and deep imputation baselines on real tabular data.
Fairoz Nower Khan, Nabuat Zaman Nahim, Peizhong Ju
Department of Computer Science, University of Kentucky
Missing data frequently arises across diverse domains, including time-series and image domains. In the real world, missing occurrences often depend on the unobservable values themselves, which are referred to as Missing Not at Random (MNAR). In this work, we introduce the Missing Pattern Recognized Diffusion Imputation Model (PRDIM), a novel framework that explicitly captures the missing pattern and precisely imputes unobserved values. PRDIM iteratively maximizes the likelihood of the joint distribution for observed values and missing mask under an Expectation-Maximization (EM) algorithm. In this sense, we first employ a pattern recognizer, which approximates the underlying missing pattern and provides guidance during every inference toward more plausible imputations with respect to the missing information. Through extensive experiments, we demonstrate that PRDIM consistently achieves strong imputation performance under MNAR settings across multiple data modalities.
In real-world machine learning applications, incomplete observations create a fundamental challenge. Researchers have come up with several ideas to address this crucial problem. However, current models still face challenges in balancing scalability and structural consistency. This study proposes a feature imputation method, called FILLER, that deliberately searches the two-dimensional latent space produced by a generative model and fills the missing values with appropriate entries. The generative model is trained on fully observed data to generate samples from the latent space, and FILLER uses this trained model to impute the values missing in the corrupted test samples. In this study, G-NeuroDAVIS serves the purpose of the generative model. This work also presents a mathematical proof on the convergence of the iterative search. Finally, FILLER has been evaluated on several image datasets under random and structured missingness patterns with varying levels of imputation complexities. In order to justify the efficacy of FILLER, it has been compared against existing state-of-the-art solution strategies in terms of RMSE, PSNR, and SSIM. In addition, Wilcoxon signed-rank test has been carried out to validate statistical significance. Moreover, downstream analyses (classification and clustering) have also established the quality of imputation in terms of standard metrics.
Santu Mondal, Chayan Maitra, Rajat K. De
Machine Intelligence Unit Indian Statistical Institute 203 Barrackpore Trunk Road Kolkata 700108