Conditional-Independence-Regularized Distributional Autoencoders for Mixed-Type Data

Siyuan Tang; Gongjun Xu; Ji Zhu

stat.MEAug 20, 2026

Conditional-Independence-Regularized Distributional Autoencoders for Mixed-Type Data

Authors: Siyuan Tang, Gongjun Xu, Ji Zhu

Organizations: Department of Statistics University of Michigan

Abstract

Mixed-type data containing both numerical and categorical variables arise in many scientific and real-world applications. Existing representation learning and generative modeling approaches typically focus either on reconstruction accuracy or unconditional data generation, but often fail to recover the full conditional distribution of the data while preserving interpretable structural relationships between heterogeneous variable types. In this work, we introduce Conditional-Independence-Regularized Distributional Autoencoders, a framework for learning low-dimensional representations of mixed-type data through conditional distribution matching and structural regularization. Our method combines an energy-score-based objective for numerical variables, a likelihood-based objective for categorical variables, and an auxiliary conditional independence regularization term encouraging the learned representation to capture the dependence between numerical and categorical components. We provide theoretical analysis showing that the optimal representation balances unexplained numerical variability, conditional entropy of categorical variables, and residual conditional dependence. Empirically, the proposed method achieves strong performance on both synthetic and real-world datasets, substantially improving categorical distribution recovery, achieving competitive overall conditional distribution recovery, and preserving mixed-type dependence structure. The code has been made available at GitHub.

Figures & tables

Figure 1: Categorical total variation distance (log-scale) across simulation settings. Lower values indicate more accurate recovery of the categorical conditional distributions. CI-Reg achieves the best performance in most settings.

Figure 2: Overall energy distance (log-scale) between the true and learned conditional distributions across simulation settings. Lower values indicate better conditional distribution recovery.

Dataset	CI-Reg	CI-Reg-0	AE	VAE	DPA	DDPM
Adult	$\mathbf{0.071\pm 0.029}$	$0.079\pm 0.032$	$0.385\pm 0.001$	$0.389\pm 0.001$	$0.385\pm 0.001$	$0.379\pm 0.005$
Insurance	$\mathbf{0.028\pm 0.008}$	$\mathbf{0.028\pm 0.008}$	$0.074\pm 0.019$	$0.071\pm 0.012$	$0.075\pm 0.013$	$0.080\pm 0.024$
Shoppers	$\mathbf{0.021\pm 0.005}$	$0.031\pm 0.009$	$0.365\pm 0.004$	$0.372\pm 0.004$	$0.365\pm 0.004$	$0.364\pm 0.006$
PM2.5	$\mathbf{0.027\pm 0.004}$	$0.030\pm 0.004$	$0.041\pm 0.002$	$0.044\pm 0.002$	$0.041\pm 0.002$	$0.047\pm 0.002$

Table 1: Categorical TV distance on real-world datasets (mean

\pm

std). Lower is better.

Dataset	CI-Reg	CI-Reg-0	AE	VAE	DPA	DDPM
Adult	$\mathbf{0.047\pm 0.034}$	$0.060\pm 0.044$	$0.405\pm 0.002$	$0.423\pm 0.002$	$0.405\pm 0.002$	$0.420\pm 0.011$
Insurance	$0.013\pm 0.003$	$\mathbf{0.012\pm 0.002}$	$0.034\pm 0.007$	$0.090\pm 0.012$	$0.034\pm 0.007$	$0.056\pm 0.024$
Shoppers	$\mathbf{0.004\pm 0.002}$	$0.007\pm 0.003$	$0.227\pm 0.005$	$0.240\pm 0.005$	$0.229\pm 0.006$	$0.280\pm 0.030$
PM2.5	$\mathbf{0.001\pm 0.000}$	$0.002\pm 0.000$	$0.009\pm 0.000$	$0.023\pm 0.002$	$0.009\pm 0.000$	$0.034\pm 0.012$

Table 2: Overall energy distance (mean

\pm

std). Lower is better.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	# Total	# Train	# Validation	# Test	# Num	# Cat
Adult	45,222	28,941	7,236	9,045	6	9
Insurance	1,338	749	321	268	4	3
Shoppers	12,330	7,891	1,973	2,466	10	8
PM2.5	41,757	23,383	10,022	8,352	7	5

Appendix

Table 3: Details of datasets used in the real-world experiments.

Figure 3: KS statistic for numerical components (column-wise, log-scale) between the true and learned conditional distributions across simulation settings. Lower values indicate better conditional distribution recovery. CI-Reg is consistently among the best-performing methods.

Figure 4: Relative Frobenius norm of the correlation matrix (log-scale) comparing covariance structure between the true and learned conditional distributions. Lower values indicate better recovery of second-order structure. CI-Reg performs competitively across all settings.

Figure 5: Energy distance for the numerical part (log-scale) between the true and learned conditional distributions across simulation settings. Lower values indicate better conditional distribution recovery.

Figure 6: Energy distance for the categorical part between the true and learned conditional distributions across simulation settings. Lower values indicate better conditional distribution recovery.

Panel of Figure 7 — Figure 7: Sensitivity analysis with respect to $\lambda_{\mathrm{CInd}}$ for the setting $K=2$ , $d_{z}=3$ . Lower values indicate better performance. Solid lines show the mean across 10 independent repetitions and shaded regions indicate $\pm$ one standard deviation.

Panel of Figure 8 — Figure 8: Sensitivity analysis with respect to $\lambda_{\mathrm{CInd}}$ for the setting $K=8$ , $d_{z}=10$ . Lower values indicate better performance. Solid lines show the mean across 10 independent repetitions and shaded regions indicate $\pm$ one standard deviation.

Panel of Figure 9 — Figure 9: Comparison of CI-Reg, CI-Reg-0, AE, VAE, DPA, and DDPM on the structured-factor simulation study. Here CI-Reg denotes CI-Reg-0.03. Results are based on 10 independent repetitions.

Panel of Figure 10 — Figure 10: Sensitivity analysis with respect to $\lambda_{\mathrm{CInd}}$ on the structured-factor simulation study. Lower values indicate better performance. Curves show the mean over 10 independent repetitions, with shaded regions indicating $\pm$ one standard deviation.

Panel of Figure 11 — Figure 11: t-SNE visualization of the learned representations for CI-Reg variants in the structured-factor simulation study with cluster separation 5.0. Rows correspond to encoder dimensions and columns correspond to different values of $\lambda_{\mathrm{CInd}}$ . Points are colored according to the true latent Gaussian mixture component. Across all encoder dimensions and regularization strengths, the learned representations preserve the underlying latent cluster structure.

Panel of Figure 12 — Figure 12: t-SNE visualization of the learned representations for CI-Reg variants in the structured-factor simulation study with cluster separation 3.0. Rows correspond to encoder dimensions and columns correspond to different values of $\lambda_{\mathrm{CInd}}$ . Points are colored according to the true latent Gaussian mixture component.

Panel of Figure 13 — Figure 13: KS statistic for numerical components on the real-world datasets comparing CI-Reg, CI-Reg-0, AE, VAE, DPA, and DDPM. Lower values indicate better reconstruction performance.

Panel of Figure 14 — Figure 14: Combined association relative Frobenius error on the real-world datasets comparing CI-Reg, CI-Reg-0, AE, VAE, DPA, and DDPM. Lower values indicate better reconstruction performance.

Panel of Figure 15 — Figure 15: Energy distance for the numerical part on the real-world datasets comparing CI-Reg, CI-Reg-0, AE, VAE, DPA, and DDPM. Lower values indicate better reconstruction performance.

Panel of Figure 16 — Figure 16: Energy distance for the categorical part on the real-world datasets comparing CI-Reg, CI-Reg-0, AE, VAE, DPA, and DDPM. Lower values indicate better reconstruction performance.

Autoencoders

Autoencoders learn compressed representations by encoding data into latent codes and decoding them for reconstruction, generation, denoising, or anomaly detection.

See all 154 papers

Logit-Coordinate Generative Models for Mixed Continuous-Categorical Tabular Data
Jul 25, 2026Yuefei Shen, Xiaotong ShenFlow Matching Synthetic Tabular Data Generation
School of Statistics, University of Minnesota
Learning Unified Distance Metric for Heterogeneous Attribute Data Clustering
Mar 3, 2026Yiqun Zhang, Mingjie Zhao, Yizhou Chen +2Clustering Unsupervised Clustering
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, 510006, Guangdong, China · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, 361102, Fujian, China · Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China
FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching
Aug 7, 2026Suman Cha, Seongchan Lee, Dohyun Ko +1Flow Matching Synthetic Tabular Data Generation
Department of Statistics and Data Science, Yonsei University · Department of Mathematical Sciences, KAIST

Jul 25, 2026J/K move · Enter open · S save

Logit-Coordinate Generative Models for Mixed Continuous-Categorical Tabular Data

Yuefei Shen, Xiaotong Shen

School of Statistics, University of Minnesota

Flow Matching Synthetic Tabular Data Generation Generative Modeling Diffusion Models

Open paper AskPDF arXiv

Mixed continuous--categorical data pose a representation problem for continuous generative models. Flow Matching and Gaussian diffusion operate in Euclidean spaces, whereas categorical laws lie on probability simplices and may be highly imbalanced. We study a logit-coordinate framework that encodes categorical variables as smoothed natural parameters and combines them with transformed numerical variables. This yields common formulations of Logit Flow Matching and Logit Diffusion. We introduce a mixed-distribution discrepancy separating categorical marginal error from conditional continuous Wasserstein error, and derive stability bounds and imbalance-aware nonparametric rates linking vector-field or drift error to decoded mixed-distribution error. Controlled simulations show that scaled-logit coordinates improve or match one-hot coordinates, especially under severe rare-cell imbalance. Across four real-data benchmarks and ten splits per dataset, Logit FM improves the primary distributional metrics on three datasets and is comparable on Churn2; Block-Conditional Logit FM consistently improves the flat model; and Logit Diffusion generally improves over or matches One-Hot Diffusion.

Abstract

Autoencoders

Explore similar work

Logit-Coordinate Generative Models for Mixed Continuous-Categorical Tabular Data

Learning Unified Distance Metric for Heterogeneous Attribute Data Clustering

FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching