BayesNDE: Bayesian Generative Modeling for Neural Density Estimation

Chenglin Li; Qiao Liu

stat.MLSep 30, 2026

BayesNDE: Bayesian Generative Modeling for Neural Density Estimation

Authors: Chenglin Li, Qiao Liu

Organizations: Department of Biostatistics, Yale University New Haven, Connecticut, USA

Abstract

Density estimation is a fundamental problem in statistics and machine learning. In this work, we introduce BayesNDE, a neural density estimator based on Bayesian generative modeling. BayesNDE learns a Bayesian generative model and evaluates its density without requiring invertible networks or Jacobian-determinant computation. For each observation, it infers a sample-specific latent posterior to construct an adaptive proposal that focuses computation on regions contributing most to its density. Bridge sampling then combines samples from this proposal with separate posterior samples to estimate the density. Experiments on nonlinear and multimodal synthetic datasets show improved estimation of density values and better recovery of the density structure compared to the state-of-the-art neural density estimators. Applications to real-world datasets further demonstrate improved anomaly detection. Together, these results highlight BayesNDE as a flexible and effective neural density estimator, demonstrating how posterior inference can turn generative models into tools for density estimation. The code and tutorials are available at https://github.com/liuq-lab/BayesNDE.

Figures & tables

Figure 1 : Overview of BayesNDE.

Figure 2 : True and estimated densities for the two-dimensional involute distribution and independent Gaussian mixture.

Method	Unconditional		Conditional
	BANK	ParkTele	Pendigits10	Vehicle	EEGEye
MADE	14.38 $\pm$ 0.27	34.34 $\pm$ 0.67	-54.56 $\pm$ 0.96	-51.84 $\pm$ 1.41	-50.25 $\pm$ 0.18
MAF	-53.15 $\pm$ 0.20	36.99 $\pm$ 0.66	-52.29 $\pm$ 0.78	-52.30 $\pm$ 2.25	-49.47 $\pm$ 0.21
Real NVP	27.21 $\pm$ 0.38	36.45 $\pm$ 0.68	-54.59 $\pm$ 1.16	-63.03 $\pm$ 3.61	-49.87 $\pm$ 0.22
Resflow	23.68 $\pm$ 0.18	36.25 $\pm$ 0.63	-47.77 $\pm$ 0.64	-45.97 $\pm$ 1.13	-48.33 $\pm$ 0.29
CFMs	10.23 $\pm$ 0.07	16.32 $\pm$ 0.19	-54.40 $\pm$ 0.36	-49.65 $\pm$ 0.87	-51.45 $\pm$ 0.14

Table 1 : Average test log-likelihood (nats) on real-world datasets. Uncertainties are two standard errors of the mean test log-likelihood across test points. The best result for each dataset is shown in bold.

	OC-SVM	I-Forest	Real NVP	MAF	CFMs	Resflow	Roundtrip	BayesNDE
Shuttle	0.953	0.958	0.735	0.861	0.932	0.477	0.958	0.965
Cardio	0.533	0.556	0.178	0.200	0.511	0.244	0.311	0.556
Pendigits	0.154	0.128	0.000	0.000	0.000	0.000	0.000	0.180

Table 2 : Precision@k on the ODDS anomaly-detection benchmarks, where k equals the number of outliers in the test split. Results are means over the same three random seeds for every method; OC-SVM is deterministic. Higher values are better.

Figure 3 : Conditional image generation on MNIST. The first column shows real MNIST examples; the remaining columns show samples generated by BayesNDE and MAF. For each digit class, representative generated images are shown in decreasing order of conditional density $p_{\bm{\theta}}({\bm{x}}\mid y)$ .

Panel of Figure 4 — Figure 4 : Empirical evaluation across benchmarks. (a) Density estimation accuracy across varying dimensions; (b) Classification performance on MNIST.

Method	Spearman $\uparrow$
BayesNDE (2D)	0.9214
w.o. EGM init.	0.0129
w.o. var. reg.	0.8929
BayesNDE (15D)	0.6311
w.o. BS	0.5932
w.o. EGM reg. (MMD)	0.5848

Table 3 : Ablation results on synthetic data.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Dim( $z$ )	Dim( $x$ )	Outliers (%)	Test outliers ( $k$ )	Train	Validation	Test
					# examples
Shuttle	3	9	7.15	365	39770	4418	4909
Cardio	5	21	9.61	15	1484	164	183
Pendigits	5	16	2.27	13	5565	618	687

Appendix

Table 4 : Basic descriptions of the three datasets used for outlier detection in our study.

Dataset	Domain	Dim( ${\bm{z}}$ )	Dim( ${\bm{x}}$ )	# of examples
Dataset	Domain	Dim( ${\bm{z}}$ )	Dim( ${\bm{x}}$ )	Train	Validation	Test
Vehicle	Transportation	6	18	684	77	85
ParkTele	Medicine	8	14	4,760	528	587
Pendigits10	Handwriting	8	16	8,902	990	1,100
EEGEye	Neuroscience	5	14	12,130	1,348	1,498
BANK	Finance	8	17	36,621	4,069	4,521

Appendix

Table 5 : Basic descriptions of the 6 datasets for density estimation used in our study. BANK and ParkTele are used for unconditional density estimation; Pendigits10, Vehicle and EEGEye for conditional density estimation, where

\dim({\bm{x}})

counts only the continuous features and the class label is the conditioning variable.

Conditional decoder $G_{{\bm{\theta}}}$	Image discriminator $D_{x}$
Inputs: $z\in\mathbb{R}^{10}$ and $y\in\{0,1\}^{10}$	Inputs: $x\in[0,1]^{784}$ and $y\in\{0,1\}^{10}$
$\operatorname{Concat}(z,y)\in\mathbb{R}^{20}$	Reshape $x$ to $28\times 28\times 1$ ; tile $y$ to $28\times 28\times 10$
FC $6272$ , LReLU	$\operatorname{Concat}(x,\operatorname{tile}(y))$ : $28\times 28\times 11$
Reshape to $7\times 7\times 128$	Conv ( $k=5$ , $s=2$ , $c=32$ ), LReLU: $14\times 14\times 32$
Tile $y$ to $7\times 7\times 10$ and concatenate: $7\times 7\times 138$	Conv ( $k=5$ , $s=2$ , $c=64$ ), LReLU: $7\times 7\times 64$
Transposed conv ( $k=3$ , $s=2$ , $c=64$ ), BN, LReLU: $14\times 14\times 64$	Conv ( $k=3$ , $s=2$ , $c=128$ ), LReLU: $4\times 4\times 128$

Appendix

Table 6 : Network architecture used by BayesNDE for conditional MNIST image generation and density estimation. Images are scaled to

[0,1]^{784}

, the class label

y\in\{0,1\}^{10}

is one-hot encoded, and the latent variable is

z\in\mathbb{R}^{10}

. For convolutional layers,

k

s

, and

c

denote kernel size, stride, and number of output channels, respectively. All convolutions use same padding. BN denotes batch normalization and LReLU has negative slope

0.2

. The decoder has two output heads that parameterize a Gaussian distribution in logit space.

		EGM			Iterative
Dataset	$z$	$\alpha$	$\lambda_{\mathrm{var}}$	$\lambda_{\mathrm{corr}}$	$\lambda^{\mathrm{iter}}_{\mathrm{var}}$	$\lambda^{\mathrm{iter}}_{\mathrm{corr}}$	$\lambda_{\mathrm{PP}}$
BANK	8	0.1	0.01	0.3	0	0.3	1000
ParkTele	8	0	0	0.3	0	0.3	1000
Pendigits10	8	0	0	0.3	0.01	0.3	0
Vehicle	6	0	0	0	0	0	0
EEGEye	5	0.1	0.01	0	0.01	0	0

Appendix

Table 7 : Regularization weights for the real datasets, fixed on validation data before any test point is scored.

Figure 5 : (a) Sampling-budget curve on the

15

-dimensional independent GMM benchmark: rank correlation between

\log\hat{p}

and the ground-truth log-density against the posterior budget

M

. Shaded bands are

\pm 1

standard error over three estimator replicates, and the dotted horizontal line marks the highest accuracy importance sampling attains at any budget, which bridge sampling already exceeds at

M=400

. (b) Running time on BANK: the time to fit each estimator plus the time the fitted estimator takes to score

500

test points, drawn uniformly at random without replacement from the

4521

-row test split. All six methods use all

17

columns and were timed on one node of the same type (Intel Xeon 6442Y, four cores, a single NVIDIA A40). The vertical axis is logarithmic.

$M$	Bridge	IS	$\Delta$
	Spearman( $\log p$ )
$100$	$\mathbf{0.5804}\pm 0.0049$	$0.5270\pm 0.0033$	$+0.0535$
$200$	$\mathbf{0.6034}\pm 0.0090$	$0.5501\pm 0.0146$	$+0.0534$
$400$	$\mathbf{0.6311}\pm 0.0080$	$0.5932\pm 0.0053$	$+0.0379$
$800$	$\mathbf{0.6300}\pm 0.0042$	$0.6014\pm 0.0022$	$+0.0286$
$1600$	$\mathbf{0.6207}\pm 0.0011$	$0.6039\pm 0.0009$	$+0.0168$

Appendix

Table 8 : Sampling-budget sweep at

p=15

, mean

\pm

standard error over three estimator replicates on

500

test points.

\Delta

is bridge minus importance sampling.

Explore similar work

In-Context Density Estimation for Tabular Data
Aug 10, 2026Patryk Marszałek, Jacek Tabor, Marek ŚmiejaDensity Ratio Estimation Out-Of-Distribution Detection
Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland
Adaptive Kernel Density Estimation with Pre-training
May 13, 2026Ruitong Zhang, Ke DengDensity Ratio Estimation Kernel Method
Department of Statistics and Data Science Tsinghua University Beijing, China
Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation
May 9, 2026Zhijun Zeng, Yixuan Jiang, Pipi Hu +1Density Ratio Estimation Energy Resources
Qiuzhen College Tsinghua University Beijing 100084, China · Yanqi Lake Beijing Institute of Mathematical Sciences and Applications Beijing 101408, China · Yau Mathematical Sciences Center Tsinghua University, Beijing 100084, China and Yanqi Lake Beijing Institute of Mathematical Sciences and Applications Beijing 101408, China

Aug 10, 2026, cs.LGJ/K move · Enter open · S save

In-Context Density Estimation for Tabular Data

Patryk Marszałek, Jacek Tabor, Marek Śmieja

Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland

Density Ratio Estimation Out-Of-Distribution Detection Dense Prediction Unsupervised Tabular Data In-Context

Open paper AskPDF arXiv

Density estimation underlies many unsupervised tasks on tabular data such as anomaly detection, out-of-distribution detection, and data augmentation. Although all these problems reduce to questions about where probability mass lies, they are typically solved individually by fitting a separate model to each dataset, with its own hyperparameters and tuning budget. We introduce ICED, an in-context, energy-based density estimator that removes this per-dataset cost. ICED is a transformer-based model pretrained once on a synthetic prior built specifically for density estimation under an objective that fits log-density where it is informative and preserves its ordering elsewhere. In the inference, it reads a dataset as context and returns an unnormalized log-density for any query point in a single forward pass, with no fitting, sampling, or hyperparameter selection. A single frozen ICED model then drives four tasks usually handled by four specialized pipelines: density estimation, out-of-distribution detection, unsupervised anomaly detection, and generative augmentation. Across all four, it is competitive with the strongest task-specific method, while being the only approach that needs no retraining, no tuning, and no labels to move between them. The code is available at https://github.com/gmum/iced.

Abstract

Explore similar work

In-Context Density Estimation for Tabular Data

Adaptive Kernel Density Estimation with Pre-training

Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation