stat.MLMay 26, 2026

Identifiable Bayesian Deep Generative Copulas with Unknown Layer Widths for Data with Arbitrary Marginal Distributions

Authors: Joseph Feldman, Yuqi Gu

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Deep generative models offer powerful tools for multivariate data analysis, but their black-box architectures are often unidentified and difficult to interpret. We introduce the Deep Discrete Encoder (DDE) Copula, an identifiable and interpretable generative model for multivariate data with arbitrary marginal distributions. The model places a hierarchical directed network of binary latent variables inside a copula framework, enabling flexible dependence modeling for mixed discrete and continuous data. Estimation is based on rank likelihoods, which decouple marginal modeling from posterior inference on the DDE parameters and avoid specifying the marginal distributions. We establish conditions for identification of the DDE copula parameters, ensuring that layer-specific parameters provide meaningful summaries of multivariate dependence. We also prove quotient-space posterior consistency for continuous margins under the exact rank likelihood and treat the extended rank likelihood for tied or mixed margins as a generalized likelihood, with concentration under an additional contrast condition. For computation, we propose a stochastic expectation-maximization algorithm for \emph{maximum a posteriori} estimation, together with initialization strategies that improve convergence. To learn network dimension adaptively, we extend Bayesian rank-selection priors to infer layer-specific widths. Simulations show strong finite-sample performance, and a personality-survey analysis reveals interpretable hierarchical latent structure in complex multivariate data.

Explore similar work

CardsList
Apr 22, 2026cs.LG

Amortized Vine Copulas for High-Dimensional Density and Information Estimation

Modeling high-dimensional dependencies while keeping likelihoods tractable remains challenging. Classical vine-copula pipelines are interpretable but can be expensive, while many neural estimators are flexible but less structured. In this work, we propose Vine Denoising Copula (VDC), an amortized vine-copula pipeline for continuous-data, simplified-vine dependence modeling. VDC trains a single bivariate denoising model and reuses it across all vine edges. For each edge, given pseudo-observations, the model predicts a piecewise-constant density grid. We then apply an IPFP/Sinkhorn projection that normalizes mass and drives the marginals to uniformity. This preserves the tractable vine-likelihood structure and the usual copula interpretation while replacing repeated per-edge optimization with GPU inference. Across synthetic and real-data benchmarks, VDC delivers strong bivariate density accuracy, competitive MI/TC estimation, and faster high-dimensional vine fitting. These gains make explicit information estimation and dependence decomposition feasible when repeated vine fitting would otherwise be costly, while conditional downstream tasks remain a limitation.
Houman Safaai
Gaussian CopulaMaximum LikelihoodHigh-Dimensional DataImage Denoising
AskPDF
May 29, 2026math.ST

Bayesian Inference with Shaped Deep Non-linear MLPs

A central aim of deep learning theory is to characterize how neural networks make predictions in the regime of simultaneously large model and training set size. Since the limits of diverging number of model parameters and dataset size do not commute it is not clear a priori what limits exist. In this work, we shed new light on these questions by studying Bayesian inference in deep non-linear MLPs in the regime where the number of training samples (PP), the input dimension (N0N_0), the hidden layer width (NN), and the number of hidden layers (LL) can all be large. We build on the Neural Covariance SDE (Li et al., 2022) to analyze predictive posteriors in the regime where LP/N∈Θ(1)LP/N\inΘ(1), playing the role of an effective network depth. Our framework covers both smooth and ReLU activation functions and applies to arbitrary temperature. We find to first order in LP/NLP/N a simple criterion for which data generating processes benefit from depth in the sense that larger LP/NLP/N increases the Bayesian model evidence. We also give a novel derivation of a prior result from the physics literature that at least to first order in LP/NLP/N, the Bayesian predictive posterior is remarkably simple and is simply equivalent to that of a data-dependent kernel method.
Boris Hanin, Tianze Jiang
Bayesian Neural NetworksBayesianMultilayer PerceptronsLarge ModelsStochastic Differential Equations
AskPDF
May 22, 2026cs.LG

Archimedean Copula Inference via Taylor-Mode AD

No existing nested Archimedean copula tool handles all three of (a) arbitrary per-variable (right-)censoring in survival analysis, (b) arbitrary nesting trees, and (c) exact parameter gradients. Existing implementations handle only bivariate problems, low dimensional (i.e., d≤10d \leq 10) cases, two layers of nesting, or only hand-derived copula nestings. We present \textsc{acopula}, a JAX-native framework that, given any Archimedean generator -- classical or neural -- evaluates exact nested-copula likelihoods and parameter gradients under arbitrary censoring masks in polynomial time. The mechanism is polynomial powering of Taylor-mode automatic differentiation output, which replaces per-family hand-derived partial Bell polynomial tables with a single differentiable computation that any user-defined generator can drive. We conduct extensive simulations to verify the correctness of \textsc{acopula}. We then demonstrate (a) per-variable censoring on 85,22985{,}229 MIMIC-IV ICU admissions in high dimensions with d=53d{=}53, fit by both classical Archimedean families and nested neural Archimedean copulas; (b) an 11-sector hierarchical model on S&P~500 daily returns at d=98d{=}98; (c) family-agnostic censored MLE across ten families, five of them with no prior implementation, on a retinopathy study; and (d) a ∼650×{\sim}650\times per-density speedup over R's \texttt{nacLL} at d=35d{=}35, scaling quadratically to d=8,000d{=}8{,}000.
Cambridge Yang, Dongdong Li
Gaussian CopulaSurvival AnalysisAlzheimer
AskPDF