Stepwise Variational Inference with Vine Copulas

Elisabeth Griesbauer; Leiv Rønneberg; Arnoldo Frigessi; Claudia Czado; Ingrid Hobæk Haff

stat.MLMar 24, 2026

Stepwise Variational Inference with Vine Copulas

Authors: Elisabeth Griesbauer, Leiv Rønneberg, Arnoldo Frigessi, Claudia Czado, Ingrid Hobæk Haff

Organizations: Norwegian Computing Center, Oslo, Norway · Oslo Centre for Biostatistics and Epidemiology (OCBE), University of Oslo, Norway · Integreat - Norwegian Centre for Knowledge-driven Machine Learning, Oslo, Norway · Department of Mathematics, University of Oslo, Norway · Technical University of Munich, Germany · Munich Data Science Institute, Germany

Abstract

We propose stepwise variational inference (VI) with vine copulas: a universal VI procedure that combines vine copulas with a novel stepwise estimation procedure of the variational parameters. Vine copulas consist of a nested sequence of trees built from copulas, where more complex latent dependence can be modeled with increasing number of trees. We propose to estimate the vine copula approximate posterior in a stepwise fashion, tree by tree along the vine structure. Further, we show that the usual backward Kullback-Leibler divergence cannot recover the correct parameters in the vine copula model, thus the evidence lower bound is defined based on the Rényi divergence. Finally, an intuitive stopping criterion for adding further trees to the vine eliminates the need to pre-define a complexity parameter of the variational distribution, as required for most other approaches. Thus, our method interpolates between mean-field VI (MFVI) and full latent dependence. In many applications, in particular sparse Gaussian processes, our method is parsimonious with parameters, while outperforming MFVI.

Figures & tables

Figure 1: Contour plots of samples from NUTS (black) regarded as ground truth and variational approximations obtained with stepwise VI with vines (blue), MFVI (orange) and GC-VI (green).

Panel of Figure 2 — Figure 2: A D-vine tree sequence on 4 elements.

\hat{\bm{\lambda}}\leftarrow\hat{\bm{\lambda}}+\gamma\widehat{\nabla_{\bm{\lambda}}l_{N}^{(\alpha)}(\hat{\bm{\lambda}};\mathbf{x})}\;.

Algorithm 1 Stepwise VI with vine copulas

Figure 3: NLPD across a range of inducing point values for the pumadyn32nm dataset. We compare the mean-field SGPR (MF-SGPR), a full rank SGPR (FullRank-SGPR), and the vine at tree level $\tau=1$ to a full (non-sparse) GP fit (red dashed line).

Figure 4: Evolution of the corresponding correlation matrix associated with the variational matrix $S$ at different tree levels $t$ of vine copula approximate posterior, for the pumadyn32nm example using 50 inducing points.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Definition	Description
$[d],\;d\in\mathbb{N}$	$[d]:=\{1,....,d\},\;d\in\mathbb{N}$	natural numbers from 1 to $d$
$j:k,\;j,k\in\mathbb{N},\;j<k$	$j:k:=\{j,j+1,...,k\}$	natural numbers from $j$ to $k$
$X$	–	1-dimensional random variable $X$
$\mathbf{X}\in\mathbb{R}^{d},\;d\in\mathbb{N}$	$\mathbf{X}:=(X_{1},...,X_{d})^{T}$	$d$ -dimensional random vector
$\mathbf{X}=\mathbf{x}$	–	$d$ -dimensional random vector $\mathbf{X}$ takes on value $\mathbf{x}\in\mathbb{R}^{d}$
$\mathbf{x}_{<j},\;j\leq d$	$\mathbf{x}_{<j}:=(x_{1},...,x_{j-1})^{T}$	$j-1$ -dimensional sub-vector of $\mathbf{x}\in\mathbb{R}^{d}$

Appendix

Table 1: Notation used throughout the paper.

Panel of Figure 5 — Figure 5: A D-vine tree sequence on 4 elements.

Example	1	2	3	4	5	6	7	8	9	10
$\alpha=0.01$	0.9863	4.1335	2.2100	0.3619	0.3108	0.3112	7.6358	0.0000	9.1015	0.0000
$\alpha=0.05$	0.0000	3.3678	6.0011	0.0615	0.0000	0.0000	0.0000	0.7748	3.6409	0.8254
$\alpha=0.1$	2.4009	0.0000	0.0000	0.0050	0.1315	0.0606	5.1381	1.6675	0.0000	0.1515
$\alpha=0.2$	0.7186	1.2066	8.5026	0.0000	0.1960	0.0802	8.3610	0.9560	0.5140	1.0075
$\alpha=0.3$	3.4847	6.0026	5.8918	0.5133	0.5074	0.5757	25.4941	1.3103	6.0731	2.3233
$\alpha=0.4$	5.8391	12.2899	18.6095	0.4655	1.0352	0.6295	45.1336	2.1350	10.9958	3.9044

Appendix

Table 2:

\Delta KL_{rel}^{(\alpha,k)}

defined in Equation ( 29 ) between

q

, a stepwise D-vine, and the true posterior obtained with NUTS, by

\alpha

in the VR-IWAE (rows) and simulated observed data coming from a multivariate normal (columns). The minimum

\Delta KL_{rel}^{(\alpha,k)}

value per Example (column) is marked in bold indicating the best

\alpha

value per example.

Example	1	2	3	4	5	6	7	8	9	10
$\alpha=0.01$	0.2106	0.4120	0.2483	0.6312	0.5207	0.6121	0.5505	0.5237	0.4068	0.4344
$\alpha=0.05$	0.1935	0.4209	0.2723	0.6335	0.5287	0.6102	0.5416	0.5268	0.4187	0.4438
$\alpha=0.1$	0.2135	0.3794	0.2671	0.6220	0.5170	0.6317	0.5547	0.5161	0.4259	0.4399
$\alpha=0.2$	0.2265	0.4503	0.3100	0.6206	0.5428	0.6242	0.5914	0.5259	0.4512	0.4731
$\alpha=0.3$	0.2717	0.4572	0.3238	0.6337	0.5579	0.6354	0.6096	0.5569	0.4855	0.4461
$\alpha=0.4$	0.3219	0.5061	0.3784	0.6437	0.5691	0.6644	0.6585	0.5872	0.5195	0.5007

Appendix

Table 3: Mean relative RMSE of standard deviation ( 31 ) of

q

, a Gaussian MF, and the true posterior obtained with NUTS, by

\alpha

in the VR-IWAE (rows) and simulated observed data coming from a multivariate normal (columns). The minimum value per Example (column) is marked in bold indicating the best

\alpha

value per example.

Example	1	2	3	4	5	6	7	8	9	10
$\alpha=0.01$	9.0e-05	2.6e-04	2.9e-04	0.0e+00	2.4e-04	1.5e-04	3.0e-04	2.6e-04	2.5e-04	7.7e-05
$\alpha=0.05$	0.0e+00	1.4e-04	0.0e+00	2.3e-04	0.0e+00	3.4e-04	0.0e+00	0.0e+00	3.2e-04	2.8e-05
$\alpha=0.1$	3.0e-04	0.0e+00	2.2e-05	8.6e-05	4.7e-04	0.0e+00	1.0e-03	1.6e-04	0.0e+00	0.0e+00
$\alpha=0.2$	3.8e-04	2.0e-05	4.5e-04	3.9e-06	4.4e-04	3.0e-04	1.8e-04	5.7e-05	9.0e-05	2.8e-04
$\alpha=0.3$	9.4e-04	1.3e-04	6.9e-04	3.9e-04	1.2e-03	1.3e-03	7.5e-04	2.5e-04	1.4e-04	1.9e-04
$\alpha=0.4$	1.6e-03	3.1e-04	1.5e-03	1.1e-03	1.9e-03	1.8e-03	1.5e-03	8.5e-04	5.3e-04	3.8e-04

Appendix

Table 4:

\Delta KL_{rel}^{(\alpha,k)}

defined in Equation ( 29 ) between

q

, a stepwise D-vine, and the true posterior obtained with NUTS, by

\alpha

in the VR-IWAE (rows) and simulated observed data coming from a multivariate Student-t distribution (columns). The minimum

\Delta KL_{rel}^{(\alpha,k)}

value per Example (column) is marked in bold indicating the best

\alpha

value per example.

Example	1	2	3	4	5	6	7	8	9	10
$\alpha=0.01$	0.0e+00	7.1e-04	0.0e+00	1.2e-03	2.2e-04	1.2e-03	1.6e-03	0.0e+00	0.0e+00	0.0e+00
$\alpha=0.05$	8.6e-05	1.6e-03	8.7e-04	3.7e-04	0.0e+00	1.3e-03	9.7e-04	8.0e-05	3.5e-04	3.0e-03
$\alpha=0.1$	3.0e-05	0.0e+00	1.1e-03	0.0e+00	1.4e-03	0.0e+00	0.0e+00	8.8e-04	8.2e-04	2.0e-03
$\alpha=0.2$	7.7e-04	2.2e-03	2.9e-03	7.8e-04	2.2e-03	1.6e-03	5.7e-03	8.5e-04	1.1e-03	2.5e-03
$\alpha=0.3$	9.9e-04	4.1e-03	2.6e-03	1.6e-03	3.5e-03	1.7e-03	5.9e-03	1.2e-03	3.7e-03	4.5e-03
$\alpha=0.4$	1.5e-03	1.1e-02	2.6e-03	2.5e-03	2.9e-03	2.3e-03	1.1e-02	2.6e-03	4.1e-03	4.8e-03

Appendix

Table 5:

\Delta KL_{rel}^{(\alpha,k)}

defined in Equation ( 29 ) between

q

, a Gaussian MF, and the true posterior obtained with NUTS, by

\alpha

in the VR-IWAE (rows) and simulated observed data coming from a multivariate Student-t distribution (columns). The minimum

\Delta KL_{rel}^{(\alpha,k)}

value per Example (column) is marked in bold indicating the best

\alpha

value per example.

Example	1	2	3	4	5	6	7	8	9	10
$\alpha=0.01$	0.0000	1.4084	0.7960	0.0000	0.0000	0.0663	0.0016	4.3844	0.3249	0.6476
$\alpha=0.05$	0.0876	1.4489	0.0000	0.0093	0.0228	0.0726	0.0000	0.7574	1.3431	1.8231
$\alpha=0.1$	0.6093	1.4973	0.0443	0.0015	0.0282	0.0579	0.0327	0.3312	0.0000	0.0000
$\alpha=0.2$	0.2182	1.5363	0.1981	0.0031	0.3240	0.1062	0.0041	0.0000	1.4434	0.4692
$\alpha=0.3$	0.3218	1.7829	0.2536	0.0054	0.7951	0.0000	0.0159	2.5615	6.8313	1.9801
$\alpha=0.4$	1.0896	0.0000	0.9100	0.0135	1.0328	0.4799	0.0192	4.6749	8.2737	1.8281

Appendix

Table 6:

\Delta KL_{rel}^{(\alpha,k)}

defined in Equation ( 29 ) between

q

, a stepwise D-vine, and the true posterior obtained with NUTS, by

\alpha

in the VR-IWAE (rows) and simulated observed data coming from a vine copula with Gaussian marginals and Clayton and Gaussian pair copulas (columns). The minimum

\Delta KL_{rel}^{(\alpha,k)}

value per Example (column) is marked in bold indicating the best

\alpha

value per example.

Example	1	2	3	4	5	6	7	8	9	10
$\alpha=0.01$	0.1288	0.1356	0.1245	0.0762	0.3238	0.1872	0.2329	0.0650	0.1316	0.2368
$\alpha=0.05$	0.1804	0.1598	0.1143	0.0752	0.3049	0.2171	0.2084	0.0579	0.1684	0.2538
$\alpha=0.1$	0.1441	0.1447	0.1632	0.0861	0.2875	0.1165	0.2638	0.0519	0.1506	0.3116
$\alpha=0.2$	0.1798	0.1618	0.1442	0.0544	0.3399	0.1891	0.2713	0.1058	0.1784	0.3075
$\alpha=0.3$	0.1642	0.1793	0.1613	0.1088	0.3360	0.1865	0.3046	0.1648	0.2274	0.3230
$\alpha=0.4$	0.2232	0.2288	0.2155	0.1628	0.3766	0.2459	0.3860	0.2227	0.2794	0.3673

Appendix

Table 7: Mean relative RMSE of standard deviation ( 31 ) of

q

, a Gaussian MF, and the true posterior obtained with NUTS, by

\alpha

in the VR-IWAE (rows) and simulated observed data coming from a vine copula with Gaussian marginals and Clayton and Gaussian pair copulas (columns). The minimum value per Example (column) is marked in bold indicating the best

\alpha

value per example.

Figure 6: Contour plots of samples from NUTS (black) regarded as ground truth and variational approximations obtained with stepwise VI with vines (blue), MFVI (orange), GC-VI (green) and MAF (red) for the independence example (top) and the needle example (bottom) of Section 4.2 .

Figure 7: RMSE across a range of inducing point values for the pumadyn32nm dataset. We compare the mean-field SGPR (MF-SGPR), a full rank SGPR (FullRank-SGPR), and the vine at tree level $\tau=1$ to a full (non-sparse) GP fit (red dashed line).

Explore similar work

Amortized Vine Copulas for High-Dimensional Density and Information Estimation
Apr 22, 2026Houman SafaaiCopula Models Amortized Inference
Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University
Structured Nonparametric Variational Inference for Dependent Latent Modeling
Jun 13, 2026Yuda Shao, Zhiling Gu, Shan YuLatent Variable Models Variational Inference
Department of Statistics, University of Virginia · Department of Biostatistics, Yale University
Generalized Matheron Variational Implicit Processes
Oct 6, 2026Luis A. Ortega, Andrés R. Masegosa, Thomas D. NielsenBayesian Inference Gaussian Processes
Aalborg University

Apr 22, 2026J/K move · Enter open · S save

Amortized Vine Copulas for High-Dimensional Density and Information Estimation

Houman Safaai

Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University

Copula Models Amortized Inference Density Estimation

Open paper AskPDF arXiv

Modeling high-dimensional dependencies while keeping likelihoods tractable remains challenging. Classical vine-copula pipelines are interpretable but can be expensive, while many neural estimators are flexible but less structured. In this work, we propose Vine Denoising Copula (VDC), an amortized vine-copula pipeline for continuous-data, simplified-vine dependence modeling. VDC trains a single bivariate denoising model and reuses it across all vine edges. For each edge, given pseudo-observations, the model predicts a piecewise-constant density grid. We then apply an IPFP/Sinkhorn projection that normalizes mass and drives the marginals to uniformity. This preserves the tractable vine-likelihood structure and the usual copula interpretation while replacing repeated per-edge optimization with GPU inference. Across synthetic and real-data benchmarks, VDC delivers strong bivariate density accuracy, competitive MI/TC estimation, and faster high-dimensional vine fitting. These gains make explicit information estimation and dependence decomposition feasible when repeated vine fitting would otherwise be costly, while conditional downstream tasks remain a limitation.

Abstract

Explore similar work

Amortized Vine Copulas for High-Dimensional Density and Information Estimation

Structured Nonparametric Variational Inference for Dependent Latent Modeling

Generalized Matheron Variational Implicit Processes