Amortised adaptive importance sampling

Arran Carter; Víctor Elvira; Esmeralda S. Whitammer

stat.MLOct 12, 2026

Amortised adaptive importance sampling

Authors: Arran Carter, Víctor Elvira, Esmeralda S. Whitammer

Organizations: University of Edinburgh

Abstract

Monte Carlo sampling algorithms aim to produce samples from a target distribution with a given unnormalised probability density function. The high cost of running Monte Carlo algorithms, with many simulation steps or particles needed to obtain a good empirical approximation to the target, can be reduced by introducing learnable components into the sampler -- a form of amortisation. Once these components are trained, amortised samplers can produce high-quality samples with less computation at sampling time than training-free algorithms would require. We propose an amortised sampler built upon the framework of adaptive importance sampling -- a hierarchical importance sampling procedure where the proposal is iteratively adapted. Our amortised adaptive importance sampling algorithm fits a lightweight neural network that adapts the proposal using information obtained over the course of sampling. By training our sampler over many different targets taken from the same family of distributions, we obtain a sampler that can adapt to a specific target at sampling time without a priori access to the target's parameters. We show that our procedure can efficiently sample from targets within the family the sampler was trained on, giving improvements over target-specific amortised samplers and training-free adaptive importance sampling algorithms. We also explore the potential for generalisation, where the trained sampler is used to sample from targets outwith its training family.

Figures & tables

Panel of Figure 2 — Figure 2: Samples generated by the final step proposal for each method for $50$ -dimensional funnel (top row) with $N=50$ components in the proposal mixture, and $5$ -dimensional banana (bottom row) with $N=10$ components in the proposal mixture, each projected into their first two coordinates.

	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	ESS $\uparrow$	$\log Z$ err. $\downarrow$
$d=5$	AAIS	0.4658 $\pm$ 0.002 (0.978)	0.2605 $\pm$ 0.008 (0.015)	0.1250 $\pm$ 0.005 (0.241)
	DM-PMC	0.9592 (1.69)	0.0301 (0.002)	0.2691 (0.537)
	M-PMC	0.7222 (1.43)	0.0965 (0.004)	0.1777 (0.347)
	GAPIS	1.678 (2.44)	0.0055 (0.001)	0.4400 (1.05)
	True samples	0.4451 (0.848)	1.000 (1.00)	0.0000 (0.000)
$d=25$	AAIS	0.5069 $\pm$ 0.006 (1.08)	0.1953 $\pm$ 0.006 (0.003)	0.2672 $\pm$ 0.007 (0.413)

Table 1: Experimental results for Funnel

d=5,25,50

. We report the mean over 256 test targets from the family and (for AAIS) the std.dev. of this mean over

3

training seeds. The worst

5\%

quantile of the metric over the batch of targets is shown in parentheses. Best in green , second-best in blue .

	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	ESS $\uparrow$	$\log Z$ err. $\downarrow$
$d=5$	AAIS	0.8718 $\pm$ 0.029 (1.50)	0.0358 $\pm$ 0.003 (0.002)	0.5812 $\pm$ 0.007 (0.972)
	DM-PMC	1.594 (3.16)	0.0120 (0.001)	0.8585 (1.69)
	M-PMC	2.005 (4.31)	0.0081 (0.000)	1.674 (3.81)
	GAPIS	138.7 (543.3)	0.0016 (0.000)	2.449 (5.90)
	True samples	0.3458 (0.565)	1.000 (1.00)	0.0000 (0.000)
$d=25$	AAIS	0.9877 $\pm$ 0.020 (2.41)	0.0125 $\pm$ 0.001 (0.001)	1.279 $\pm$ 0.022 (2.19)

Table 2: Experimental results for the Bayesian hierarchical model

d=5,25

. Metrics reported follow the convention of Table 1 .

Target Family	Amortised over	Evaluated on	$\log Z$ err. $\downarrow$
Funnel	Single target	Single target	$0.2656$ $\pm 0.0062$ $(0.3187)$
	Family	Single target	$0.2647$ $\pm 0.0081$ $(0.3222)$
	Single target	Family	$0.2661$ $\pm 0.0049$ $(0.4080)$
	Family	Family	$0.2672$ $\pm 0.0074$ $(0.4131)$
Banana	Single target	Single target	$1.096$ $\pm 0.0005$ $(1.529)$
	Family	Single target	$1.500$ $\pm 0.0244$ $(2.102)$

Table 3: Performance of AAIS amortised over a single target vs over a whole family, all targets are at

d=25

. Metrics reported follow the convention of Table 1 .

Base Target	Shift $\downarrow$ Method $\rightarrow$	AAIS	DM-PMC	M-PMC	GAPIS
Banana ( $d=5$ )	Curvature range: $\left[\frac{1}{10},\frac{1}{2}\right]\to\left[\frac{1}{2},1\right]$	$1.365$ $\pm 0.0344$ $(2.459)$	$1.530$ ( $3.040$ )	$1.518$ ( $2.951$ )	$1.387$ ( $1.996$ )
Banana ( $d=5$ )	Curvature range: $\left[\frac{1}{10},\frac{1}{2}\right]\to[1,2]$	$2.912$ $\pm 0.0520$ $(5.382)$	$3.370$ ( $6.450$ )	$3.429$ ( $6.192$ )	$2.278$ ( $4.172$ )
BLR ( $d=5$ )	$\beta^{}\sim\mathcal{N}(0,4I_{r})\to\beta^{}\sim\mathcal{N}(0,64I_{r})$	$3.980$ $\pm 0.0428$ $(7.859)$	$1.323$ $(4.059)$	$5.108$ $(9.767)$	$131.1$ $(611.8)$
	$\beta^{}\sim\mathcal{N}(0,4I_{r})\to\beta^{}\sim\mathcal{N}(0,400I_{r})$	$12.231$ $\pm 0.0404$ $(20.850)$	6.793 (16.909)	13.203 (22.528)	684.2 $(3.8\!\times\!10^{3})$
	$n$ range: $[3r,12r]\to[12r,48r]$	$0.7802$ $\pm 0.0302$ $(2.521)$	$1.152$ $(2.197)$	$2.624$ $(4.347)$	$132.9$ $(668.3)$
	$n$ range: $[3r,12r]\to[48r,192r]$	$0.8607$ $\pm 0.0162$ $(2.648)$	$1.190$ $(2.101)$	$2.753$ $(4.494)$	$515.2$ $(2.5\!\times\!10^{3})$

Table 4: Generalisation of a trained AAIS algorithm to shifted targets of the same form, see family definitions in § B.2 . Sliced 1-Wasserstein distance of the final proposal to the target is reported and follows the convention of Table 1 . A “-” indicates that the mean could not be calculated due to numerical divergence.

Target Family	Objective $\downarrow$ Metric $\rightarrow$	$\log Z$ err. $\downarrow$
Funnel	$\KL(q_{t}\mid\mid p)$	$0.2672$ $\pm 0.0074$ $(0.4131)$
	$\KL(p\mid\mid q_{t})$ (SNIS)	$4.496$ $\pm 0.7189$ $(6.445)$
	$\KL(p\mid\mid q_{t})$ (oracle draws)	$0.2145$ $\pm 0.0077$ $(0.3352)$
Banana	$\KL(q_{t}\mid\mid p)$	$2.127$ $\pm 0.0138$ $(3.341)$
	$\KL(p\mid\mid q_{t})$ (SNIS)	$6.705$ $\pm 0.6376$ $(11.362)$
	$\KL(p\mid\mid q_{t})$ (oracle draws)	$1.130$ $\pm 0.0392$ $(2.398)$

Table 5: Effect of three different divergences in the AAIS objective. Metrics reported follow the convention of Table 1 . All targets are at

d=25

. A “-” indicates that the mean could not be calculated due to numerical divergence.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Feature	Dimension
$h_{n}$ – the pooled particle encodings.	$H$
$\frac{\left(\mu_{n}-\bar{\mu}\right)}{\zeta}$ , position relative to the ensemble.	$d$
$\frac{1}{2}\log\Sigma_{n,jj}-\log\zeta$ , variance relative to the ensemble.	$d$
Correlations of $\Sigma_{n}$ .	$d(d{-}1)/2$
$\log Nm_{n}$ – EM mass share.	$1$
$\log N\alpha_{n}$ – Current mixture weight multiplied by $N$ .	$1$

Appendix

Table 6: Summary of component level features.

Target Family	$d$	$N$	$T$	$K$ (AAIS)	$K$ (DM-PMC)	$K$ (M-PMC)	$K$ (GAPIS)	$\lambda$ (GAPIS)	$\Delta\mu_{\max}$	H	Num. params.
BLR d5	5	10	15	15	32	32	25	1e-05	3.0	32	5802
Hierarchical d5	5	10	15	15	32	32	25	1e-05	1.0	32	5802
Funnel d5	5	10	15	15	32	32	25	0.01	1.0	32	5802
Banana d5	5	10	15	15	32	32	25	0.03	1.0	32	5802
BLR d25	25	25	15	15	32	32	6	1e-05	3.0	64	31346
Hierarchical d25	25	25	15	15	32	32	6	1e-05	1.0	64	31346

Appendix

Table 7: Hyperparameter settings for experiments. The final three columns are specific to AAIS, the Num. params column was calculated using the formula from § A.1 .

Target Family	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
BLR	With gradient features	$1.174$ $\pm 0.0333$ $(2.600)$	$409.9$ $\pm 3.154$ $(1.0\!\times\!10^{3})$	$374.4$ $\pm 4.126$ $(994.0)$	$6.832$ $\pm 0.2652$ $(15.645)$
BLR	No gradient features	$5.300$ $\pm 1.431$ $(6.316)$	$1.1\!\times\!10^{3}$ $\pm 157.7$ $(1.8\!\times\!10^{3})$	$790.1$ $\pm 22.984$ $(1.4\!\times\!10^{3})$	$19.601$ $\pm 1.304$ $(25.856)$
Hierarchical	With gradient features	$0.9877$ $\pm 0.0201$ $(2.414)$	$2.157$ $\pm 0.0160$ $(3.137)$	$1.279$ $\pm 0.0215$ $(2.194)$	$4.531$ $\pm 0.0769$ $(12.805)$
Hierarchical	No gradient features	$1.151$ $\pm 0.0501$ $(2.963)$	$2.515$ $\pm 0.0460$ $(4.160)$	$1.426$ $\pm 0.0569$ $(2.583)$	$5.134$ $\pm 0.0410$ $(15.701)$
Funnel	With gradient features	$0.5069$ $\pm 0.0061$ $(1.082)$	$0.1535$ $\pm 0.0032$ $(0.2379)$	$0.2672$ $\pm 0.0074$ $(0.4131)$	$0.6550$ $\pm 0.1238$ $(2.128)$
Funnel	No gradient features	$0.5200$ $\pm 0.0059$ $(1.095)$	$0.2003$ $\pm 0.0055$ $(0.3083)$	$0.2784$ $\pm 0.0061$ $(0.4373)$	$0.5170$ $\pm 0.0218$ $(1.560)$

Appendix

Table 8: Experimental results showing the effect of removing target gradients as features in the adaptation function ( § A.1 ) note that this still uses target gradients when performing gradient descent in training. Metrics reported follow the convention of Table 1 . All targets are at

d=25

Target Family	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
BLR	With EM features	$1.174$ $\pm 0.0333$ $(2.600)$	$409.9$ $\pm 3.154$ $(1.0\!\times\!10^{3})$	$374.4$ $\pm 4.126$ $(994.0)$	$6.832$ $\pm 0.2652$ $(15.645)$
BLR	No EM features	$0.9059$ $\pm 0.0132$ $(1.908)$	$357.5$ $\pm 5.075$ $(960.2)$	$335.0$ $\pm 4.293$ $(916.3)$	$5.496$ $\pm 0.0735$ $(11.569)$
Hierarchical	With EM features	$0.9877$ $\pm 0.0201$ $(2.414)$	$2.157$ $\pm 0.0160$ $(3.137)$	$1.279$ $\pm 0.0215$ $(2.194)$	$4.531$ $\pm 0.0769$ $(12.805)$
Hierarchical	No EM features	$1.255$ $\pm 0.0103$ $(3.175)$	$2.683$ $\pm 0.0571$ $(4.568)$	$1.512$ $\pm 0.0402$ $(2.838)$	$5.239$ $\pm 0.0698$ $(13.861)$
Funnel	With EM features	$0.5069$ $\pm 0.0061$ $(1.082)$	$0.1535$ $\pm 0.0032$ $(0.2379)$	$0.2672$ $\pm 0.0074$ $(0.4131)$	$0.6550$ $\pm 0.1238$ $(2.128)$
Funnel	No EM features	$0.5486$ $\pm 0.0096$ $(1.218)$	$0.2895$ $\pm 0.0444$ $(0.4191)$	$0.2871$ $\pm 0.0020$ $(0.4446)$	$0.6850$ $\pm 0.0509$ $(2.014)$

Appendix

Table 9: Experimental results showing the effect of the EM features in the adaptation model. Metrics reported follow the convention of Table 1 . All targets are at

d=25

Target Family	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
BLR	Full model	$1.174$ $\pm 0.0333$ $(2.600)$	$409.9$ $\pm 3.154$ $(1.0\!\times\!10^{3})$	$374.4$ $\pm 4.126$ $(994.0)$	$6.832$ $\pm 0.2652$ $(15.645)$
	Terminal objective only	$2.706$ $\pm 0.6514$ $(3.839)$	$717.5$ $\pm 101.8$ $(1.3\!\times\!10^{3})$	$604.1$ $\pm 97.197$ $(1.1\!\times\!10^{3})$	$12.982$ $\pm 3.012$ $(20.797)$
	Step-wise	$0.8079$ $\pm 0.0069$ $(1.701)$	$320.2$ $\pm 2.461$ $(871.4)$	$310.9$ $\pm 2.342$ $(858.9)$	$5.005$ $\pm 0.0412$ $(10.651)$
Hierarchical	Full model	$0.9877$ $\pm 0.0201$ $(2.414)$	$2.157$ $\pm 0.0160$ $(3.137)$	$1.279$ $\pm 0.0215$ $(2.194)$	$4.531$ $\pm 0.0769$ $(12.805)$
	Terminal objective only	$2.184$ $\pm 0.1925$ $(3.297)$	$7.913$ $\pm 0.6277$ $(11.317)$	$2.727$ $\pm 0.2046$ $(4.947)$	$10.343$ $\pm 0.5920$ $(18.960)$
	Step-wise	$1.227$ $\pm 0.0190$ $(2.463)$	$2.377$ $\pm 0.0102$ $(4.292)$	$1.404$ $\pm 0.0219$ $(2.434)$	$4.881$ $\pm 0.1296$ $(10.874)$

Appendix

Table 10: Experimental results showing the effect of three different objective structures, a terminal only objective that uses pathwise gradients that flow through the whole trajectory, and a step-wise objective with no gradients flowing between steps in the trajectory, full model has step-wise objectives and also gradients flowing through the whole trajectory. Metrics reported follow the convention of Table 1 . All targets are at

d=25

Target Family	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
BLR	$\KL(q_{t}\mid\mid p)$	$1.174$ $\pm 0.0333$ $(2.600)$	$409.9$ $\pm 3.154$ $(1.0\!\times\!10^{3})$	$374.4$ $\pm 4.126$ $(994.0)$	$6.832$ $\pm 0.2652$ $(15.645)$
	$\KL(p\mid\mid q_{t})$ (SNIS)	$118.2$ $\pm 94.077$ $(128.3)$	$1.3\!\times\!10^{14}$ $\pm 1.8\!\times\!10^{14}$ $(2.6\!\times\!10^{35})$	$1.6\!\times\!10^{3}$ $\pm 1.4\!\times\!10^{3}$ $(5.0\!\times\!10^{3})$	$21.066$ $\pm 1.232$ $(27.944)$
	$\KL(p\mid\mid q_{t})$ (oracle draws)	$2.344$ $\pm 0.4549$ $(3.144)$	$1.5\!\times\!10^{19}$ $\pm 2.7\!\times\!10^{19}$ $(1.1\!\times\!10^{14})$	$583.8$ $\pm 41.198$ $(1.1\!\times\!10^{3})$	$11.520$ $\pm 1.502$ $(18.700)$
Hierarchical	$\KL(q_{t}\mid\mid p)$	$0.9877$ $\pm 0.0201$ $(2.414)$	$2.157$ $\pm 0.0160$ $(3.137)$	$1.279$ $\pm 0.0215$ $(2.194)$	$4.531$ $\pm 0.0769$ $(12.805)$
	$\KL(p\mid\mid q_{t})$ (SNIS)	-	-	-	-
	$\KL(p\mid\mid q_{t})$ (oracle draws)	-	-	-	-

Appendix

Table 11: Experimental results showing the effect of three different divergences in the objective. Metrics reported follow the convention of Table 1 . All targets are at

d=25

. A “-” indicates that the mean could not be calculated due to a numerical divergence.

Target Family	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
BLR	Diagonal	$1.174$ $\pm 0.0333$ $(2.600)$	$409.9$ $\pm 3.154$ $(1.0\!\times\!10^{3})$	$374.4$ $\pm 4.126$ $(994.0)$	$6.832$ $\pm 0.2652$ $(15.645)$
	Isotropic	$1.956$ $\pm 0.2392$ $(2.927)$	$615.5$ $\pm 45.231$ $(1.1\!\times\!10^{3})$	$495.5$ $\pm 40.974$ $(1.0\!\times\!10^{3})$	$9.346$ $\pm 1.027$ $(16.318)$
	Full	$262.0$ $\pm 270.3$ $(288.0)$	$1.4\!\times\!10^{5}$ $\pm 2.5\!\times\!10^{5}$ $(3.8\!\times\!10^{5})$	$827.1$ $\pm 34.365$ $(1.5\!\times\!10^{3})$	$21.699$ $\pm 1.606$ $(29.616)$
Hierarchical	Diagonal	$0.9877$ $\pm 0.0201$ $(2.414)$	$2.157$ $\pm 0.0160$ $(3.137)$	$1.279$ $\pm 0.0215$ $(2.194)$	$4.531$ $\pm 0.0769$ $(12.805)$
	Isotropic ∗	$2.710$ $\pm 0.0008$ $(5.984)$	$14.136$ $\pm 0.2302$ $(22.366)$	$5.371$ $\pm 0.0832$ $(13.536)$	$11.226$ $\pm 0.1195$ $(27.474)$
	Full	$8.314$ $\pm 11.169$ $(12.147)$	$157.2$ $\pm 256.7$ $(328.0)$	$11.193$ $\pm 14.756$ $(17.452)$	$16.326$ $\pm 14.053$ $(29.943)$

Appendix

Table 12: Experimental results showing the effect of using different parametrisations for the proposal covariance. Metrics reported follow the convention of Table 1 . An ∗ indicates that only

2

out of the

3

training seeds were stable. All targets are at

d=25

Target Family	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
BLR	Scale-equivariant	$1.174$ $\pm 0.0333$ $(2.600)$	$409.9$ $\pm 3.154$ $(1.0\!\times\!10^{3})$	$374.4$ $\pm 4.126$ $(994.0)$	$6.832$ $\pm 0.2652$ $(15.645)$
BLR	Not scale-equivariant	$1.461$ $\pm 0.0976$ $(2.529)$	$542.4$ $\pm 16.950$ $(1.2\!\times\!10^{3})$	$466.2$ $\pm 15.175$ $(1.0\!\times\!10^{3})$	$8.135$ $\pm 0.4538$ $(14.913)$
Hierarchical	Scale-equivariant	$0.9877$ $\pm 0.0201$ $(2.414)$	$2.157$ $\pm 0.0160$ $(3.137)$	$1.279$ $\pm 0.0215$ $(2.194)$	$4.531$ $\pm 0.0769$ $(12.805)$
Hierarchical	Not scale-equivariant	$1.052$ $\pm 0.0421$ $(3.274)$	$2.192$ $\pm 0.1499$ $(3.985)$	$0.9876$ $\pm 0.0388$ $(1.950)$	$4.132$ $\pm 0.0887$ $(14.853)$
Funnel	Scale-equivariant	$0.5069$ $\pm 0.0061$ $(1.082)$	$0.1535$ $\pm 0.0032$ $(0.2379)$	$0.2672$ $\pm 0.0074$ $(0.4131)$	$0.6550$ $\pm 0.1238$ $(2.128)$
Funnel	Not scale-equivariant	$0.5076$ $\pm 0.0036$ $(1.105)$	$0.1486$ $\pm 0.0018$ $(0.2277)$	$0.2304$ $\pm 0.0044$ $(0.3747)$	$0.5341$ $\pm 0.0807$ $(1.515)$

Appendix

Table 13: Experimental results showing the effect of using a model that is not scale-equivariant § A.3 . Metrics reported follow the convention of Table 1 . All targets are at

d=25

Base Target	Shift $\downarrow$ Method $\rightarrow$	Equivariant	Non-equivariant
Banana ( $d=5$ )	Curvature range: $\left[\frac{1}{10},\frac{1}{2}\right]\to\left[\frac{1}{2},1\right]$	$1.365$ $\pm 0.0344$ $(2.459)$	$1.395$ $\pm 0.0184$ $(2.593)$
Banana ( $d=5$ )	Curvature range: $\left[\frac{1}{10},\frac{1}{2}\right]\to[1,2]$	$2.912$ $\pm 0.0520$ $(5.382)$	$3.197$ $\pm 0.0251$ $(5.970)$
BLR ( $d=5$ )	$\beta^{}\sim\mathcal{N}(0,4I_{p})\to\beta^{}\sim\mathcal{N}(0,64I_{p})$	$3.980$ $\pm 0.0428$ $(7.859)$	$4.331$ $\pm 0.0666$ $(8.379)$
	$\beta^{}\sim\mathcal{N}(0,4I_{p})\to\beta^{}\sim\mathcal{N}(0,400I_{p})$	$12.231$ $\pm 0.0404$ $(20.850)$	$12.897$ $\pm 0.1075$ $(21.379)$
	$n$ range: $[3p,12p]\to[12p,48p]$	$0.7802$ $\pm 0.0302$ $(2.521)$	$0.8313$ $\pm 0.0859$ $(2.477)$
	$n$ range: $[3p,12p]\to[48p,192p]$	$0.8607$ $\pm 0.0162$ $(2.648)$	$1.086$ $\pm 0.2062$ $(2.863)$

Appendix

Table 14: Experimental results showing the generalisation of the equivariant AAIS and a non-equivariant AAIS § A.3 to shifted targets of the same form. Metrics reported follow the convention of Table 1 .

Target Family	Hidden dim. $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
BLR	$H=64$	$1.174$ $\pm 0.0333$ $(2.600)$	$409.9$ $\pm 3.154$ $(1.0\!\times\!10^{3})$	$374.4$ $\pm 4.126$ $(994.0)$	$6.832$ $\pm 0.2652$ $(15.645)$
	$H=16$	$3.119$ $\pm 0.1201$ $(3.784)$	$758.6$ $\pm 16.843$ $(1.3\!\times\!10^{3})$	$644.2$ $\pm 16.823$ $(1.2\!\times\!10^{3})$	$14.417$ $\pm 0.8210$ $(21.133)$
	$H=32$	$1.808$ $\pm 0.1249$ $(2.609)$	$602.8$ $\pm 22.304$ $(1.1\!\times\!10^{3})$	$484.6$ $\pm 20.098$ $(992.2)$	$8.506$ $\pm 0.5306$ $(14.841)$
	$H=128$	$1.687$ $\pm 0.1518$ $(3.390)$	$527.6$ $\pm 24.223$ $(1.2\!\times\!10^{3})$	$448.7$ $\pm 22.353$ $(1.1\!\times\!10^{3})$	$8.904$ $\pm 0.9462$ $(18.417)$
Hierarchical	$H=64$	$0.9877$ $\pm 0.0201$ $(2.414)$	$2.157$ $\pm 0.0160$ $(3.137)$	$1.279$ $\pm 0.0215$ $(2.194)$	$4.531$ $\pm 0.0769$ $(12.805)$
	$H=16$	$2.813$ $\pm 0.0464$ $(3.642)$	$13.661$ $\pm 0.6366$ $(18.976)$	$3.318$ $\pm 0.1224$ $(5.651)$	$11.879$ $\pm 0.2681$ $(20.676)$

Appendix

Table 15: Experimental results showing the effect of increasing the hidden dimension used in the MLP layers of the adaptation function ( § A.1 ). Metrics reported follow the convention of Table 1 . All targets are at

d=25

	Trained on $\downarrow$ Evaluated on $\rightarrow$	Banana	BLR	Funnel	Hierarchical
$d=5$	Banana	$0.5072$ $\pm 0.0042$ ( $1.132$ )	$1.790$ $\pm 0.1044$ ( $3.365$ )	$0.9594$ $\pm 0.0191$ ( $1.709$ )	$3.040$ $\pm 0.0285$ ( $6.533$ )
	BLR	$1.516$ $\pm 0.0204$ ( $2.158$ )	$0.7486$ $\pm 0.0271$ ( $2.397$ )	$1.598$ $\pm 0.0177$ ( $2.336$ )	$3.607$ $\pm 0.0194$ ( $6.645$ )
	Funnel	$0.7655$ $\pm 0.0175$ ( $1.349$ )	$2.415$ $\pm 0.1542$ ( $4.008$ )	$0.4658$ $\pm 0.0016$ ( $0.9783$ )	$2.877$ $\pm 0.0391$ ( $6.540$ )
	Hierarchical	$0.9649$ $\pm 0.0458$ ( $1.698$ )	$4.585$ $\pm 1.097$ ( $8.091$ )	$1.107$ $\pm 0.0319$ ( $1.843$ )	$0.8718$ $\pm 0.0289$ ( $1.500$ )
$d=25$	Banana	1.093 $\pm 0.0069$ $(1.492)$	$1.581$ $\pm 0.0354$ $(2.745)$	1.045 $\pm 0.0203$ $(1.872)$	$3.985$ $\pm 0.0198$ $(8.518)$
	BLR	$1.610$ $\pm 0.0280$ $(1.955)$	$1.174$ $\pm 0.0334$ $(2.602)$	$1.480$ $\pm 0.0490$ $(2.378)$	$4.234$ $\pm 0.0647$ $(8.567)$

Appendix

Table 16: Experimental results showing the generalisation of a trained AAIS algorithm to targets from different families. Metrics reported follow the convention of Table 1 .

	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	sliced $W_{1}$ $\downarrow$	rev. KL $\downarrow$	ESS $\uparrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$	$\mathbb{E}\\|x\\|^{2}$ err. $\downarrow$
$d=5$	AAIS	0.4658 $\pm$ 0.002 (0.978)	0.5919 $\pm$ 0.004 (1.21)	0.1224 $\pm$ 0.003 (0.194)	0.2605 $\pm$ 0.008 (0.015)	0.1250 $\pm$ 0.005 (0.241)	0.3619 $\pm$ 0.006 (1.07)	70.126 $\pm$ 0.267 (246.6)
	DM-PMC	0.9592 (1.69)	0.9823 (1.83)	11.360 (31.36)	0.0301 (0.002)	0.2691 (0.537)	0.5487 (1.31)	83.221 (267.1)
	M-PMC	0.7222 (1.43)	0.7588 (1.57)	0.7444 (1.52)	0.0965 (0.004)	0.1777 (0.347)	0.5617 (1.43)	78.737 (258.9)
	GAPIS	1.678 (2.44)	1.425 (2.44)	$1.6\!\times\!10^{25}$ ( $1.1\!\times\!10^{12}$ )	0.0055 (0.001)	0.4400 (1.05)	0.9232 (1.94)	71.511 (259.7)
	True samples	0.4451 (0.848)	0.5206 (0.967)	0.0000 (0.000)	1.000 (1.00)	0.0000 (0.000)	0.1113 (0.241)	15.528 (73.76)
$d=25$	AAIS	0.5069 $\pm$ 0.006 (1.08)	0.5561 $\pm$ 0.008 (1.16)	0.1535 $\pm$ 0.003 (0.238)	0.1953 $\pm$ 0.006 (0.003)	0.2672 $\pm$ 0.007 (0.413)	0.6550 $\pm$ 0.124 (2.13)	428.2 $\pm$ 7.106 ( $1.6\!\times\!10^{3}$ )

Appendix

Table 17: Experimental results for Funnel

d=5,25,50

. Metrics reported follow the convention of Table 1 .

	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	sliced $W_{1}$ $\downarrow$	rev. KL $\downarrow$	ESS $\uparrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$	$\mathbb{E}\\|x\\|^{2}$ err. $\downarrow$
$d=5$	AAIS	0.5072 $\pm$ 0.004 (1.13)	0.5685 $\pm$ 0.016 (1.25)	0.4092 $\pm$ 0.008 (0.867)	0.1301 $\pm$ 0.003 (0.006)	0.1988 $\pm$ 0.011 (0.453)	0.9676 $\pm$ 0.029 (2.44)	9.373 $\pm$ 0.085 (27.18)
	DM-PMC	0.7214 (1.38)	0.6353 (1.35)	3.493 (6.80)	0.0466 (0.003)	0.2104 (0.473)	0.9608 (2.17)	9.273 (27.58)
	M-PMC	0.5467 (1.25)	0.5637 (1.27)	2.151 (6.55)	0.0619 (0.002)	0.2597 (0.688)	0.9823 (2.41)	8.910 (25.39)
	GAPIS	1.214 (1.69)	0.9583 (1.57)	162.3 (675.6)	0.0053 (0.001)	0.5060 (1.09)	1.207 (2.30)	6.848 (22.08)
	True samples	0.2041 (0.339)	0.2487 (0.408)	0.0000 (0.000)	1.000 (1.00)	0.0000 (0.000)	0.0610 (0.117)	0.4696 (1.55)
$d=25$	AAIS	1.093 $\pm$ 0.007 (1.49)	1.275 $\pm$ 0.011 (1.90)	6.112 $\pm$ 0.056 (7.82)	0.0032 $\pm$ 0.000 (0.000)	2.127 $\pm$ 0.014 (3.34)	4.295 $\pm$ 0.023 (6.51)	70.088 $\pm$ 0.358 (113.7)

Appendix

Table 18: Experimental results for Banana

d=5,25,50

. Metrics reported follow the convention of Table 1 .

	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	sliced $W_{1}$ $\downarrow$	rev. KL $\downarrow$	ESS $\uparrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$	$\mathbb{E}\\|x\\|^{2}$ err. $\downarrow$
$d=2$	Learnt	$0.6752$ $\pm 0.0028$ $(1.579)$	$0.4618$ $\pm 0.0042$ $(1.200)$	$0.2126$ $\pm 0.0032$ $(0.5842)$	$0.4159$ $\pm 0.0054$ $(0.0371)$	$0.1191$ $\pm 0.0032$ $(0.4545)$	$0.4452$ $\pm 0.0067$ $(1.262)$	$0.9646$ $\pm 0.0329$ $(3.482)$
	DM-PMC	$1.116$ $(2.179)$	$0.8316$ $(1.856)$	$1.199$ $(2.092)$	$0.1341$ $(0.0073)$	$0.1933$ $(0.4949)$	$0.6694$ $(1.910)$	$1.110$ $(2.738)$
	M-PMC	$0.3899$ $(0.9256)$	$0.3840$ $(0.9017)$	$0.2054$ $(0.6037)$	$0.4153$ $(0.0352)$	$0.0840$ $(0.3159)$	$0.3541$ $(1.278)$	$0.6824$ $(2.382)$
	GAPIS	$0.7924$ $(1.595)$	$0.5031$ $(1.269)$	$0.4298$ $(0.9473)$	$0.3072$ $(0.0115)$	$0.1297$ $(0.4825)$	$0.4465$ $(1.477)$	$0.9507$ $(3.325)$
	True samples	$0.1862$ $(0.3086)$	$0.2578$ $(0.4077)$	$0.0000$ $(0.0000)$	$1.0000$ $(1.0000)$	$0.0000$ $(0.0000)$	$0.0505$ $(0.1023)$	$0.1148$ $(0.2929)$
$d=5$	Learnt	$0.7183$ $\pm 0.0043$ $(1.200)$	$0.5029$ $\pm 0.0048$ $(0.9176)$	$0.3738$ $\pm 0.0023$ $(0.7057)$	$0.2325$ $\pm 0.0012$ $(0.0095)$	$0.2625$ $\pm 0.0030$ $(0.5819)$	$0.9929$ $\pm 0.0063$ $(2.060)$	$2.545$ $\pm 0.0308$ $(7.899)$

Appendix

Table 19: Experimental results for GMM5

d=2,5,25

. Metrics reported follow the convention of Table 1 .

	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	sliced $W_{1}$ $\downarrow$	rev. KL $\downarrow$	ESS $\uparrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$	$\mathbb{E}\\|x\\|^{2}$ err. $\downarrow$
$d=5$	AAIS	0.7489 $\pm$ 0.027 (2.39)	0.7158 $\pm$ 0.031 (2.41)	25.102 $\pm$ 1.016 (90.71)	0.0140 $\pm$ 0.001 (0.000)	21.971 $\pm$ 1.398 (85.87)	1.869 $\pm$ 0.081 (6.14)	3.852 $\pm$ 0.125 (17.16)
	DM-PMC	1.099 (2.20)	0.7844 (2.07)	$2.0\!\times\!10^{3}$ ( $6.4\!\times\!10^{3}$ )	0.0005 (0.000)	14.359 (54.53)	1.419 (4.65)	3.311 (11.96)
	M-PMC	2.263 (4.17)	2.223 (4.18)	114.1 (262.1)	0.0016 (0.000)	111.3 (257.0)	5.941 (11.14)	30.751 (91.58)
	GAPIS	34.469 (163.6)	2.999 (4.71)	$6.8\!\times\!10^{3}$ ( $1.5\!\times\!10^{4}$ )	0.0003 (0.000)	69.437 (144.8)	5.895 (10.22)	30.323 (75.57)
	True samples	0.0169 (0.030)	0.0199 (0.036)	0.0000 (0.000)	1.000 (1.00)	0.0000 (0.000)	0.0048 (0.011)	0.0104 (0.027)
$d=25$	AAIS	1.174 $\pm$ 0.033 (2.60)	1.091 $\pm$ 0.044 (2.49)	409.9 $\pm$ 3.154 ( $1.0\!\times\!10^{3}$ )	0.0004 $\pm$ 0.000 (0.000)	374.4 $\pm$ 4.126 (994.0)	6.832 $\pm$ 0.265 (15.64)	33.606 $\pm$ 5.746 (140.9)

Appendix

Table 20: Experimental results for Bayesian linear regression

d=5,25

. Metrics reported follow the convention of Table 1 .

	Method $\downarrow$ Metric $\rightarrow$	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	sliced $W_{1}$ $\downarrow$	rev. KL $\downarrow$	ESS $\uparrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$	$\mathbb{E}\\|x\\|^{2}$ err. $\downarrow$
$d=5$	AAIS	0.8718 $\pm$ 0.029 (1.50)	1.077 $\pm$ 0.017 (2.23)	1.120 $\pm$ 0.021 (1.43)	0.0358 $\pm$ 0.003 (0.002)	0.5812 $\pm$ 0.007 (0.972)	1.678 $\pm$ 0.066 (4.08)	39.313 $\pm$ 1.213 (94.82)
	DM-PMC	1.594 (3.16)	1.650 (3.33)	81.933 (316.4)	0.0120 (0.001)	0.8585 (1.69)	2.575 (6.85)	62.201 (182.2)
	M-PMC	2.005 (4.31)	1.999 (3.93)	10.592 (28.46)	0.0081 (0.000)	1.674 (3.81)	3.707 (9.78)	68.642 (187.8)
	GAPIS	138.7 (543.3)	4.140 (8.03)	$1.8\!\times\!10^{6}$ ( $8.9\!\times\!10^{6}$ )	0.0016 (0.000)	2.449 (5.90)	6.160 (14.54)	86.651 (242.6)
	True samples	0.3458 (0.565)	0.4084 (0.665)	0.0000 (0.000)	1.000 (1.00)	0.0000 (0.000)	0.0900 (0.182)	1.432 (4.17)
$d=25$	AAIS	0.9877 $\pm$ 0.020 (2.41)	1.201 $\pm$ 0.028 (3.01)	2.157 $\pm$ 0.016 (3.14)	0.0125 $\pm$ 0.001 (0.001)	1.279 $\pm$ 0.022 (2.19)	4.531 $\pm$ 0.077 (12.81)	193.4 $\pm$ 1.506 (763.2)

Appendix

Table 21: Experimental results for the Bayesian hierarchical model

d=5,25

. Metrics reported follow the convention of Table 1 .

Target Family	Amortised over	Evaluated on	sliced $W_{1}^{\mathrm{prop}}$ $\downarrow$	rev. KL $\downarrow$	$\log Z$ err. $\downarrow$	$\mathbb{E}[x]$ err. $\downarrow$
Banana	Single-target	Single-target	$0.6679$ $\pm 0.0009$ $(0.7235)$	$3.445$ $\pm 0.0024$ $(3.675)$	$1.096$ $\pm 0.0005$ $(1.529)$	$2.821$ $\pm 0.0426$ $(5.602)$
	Family	Single-target	$0.8139$ $\pm 0.0048$ $(0.8673)$	$4.671$ $\pm 0.0535$ $(4.920)$	$1.500$ $\pm 0.0244$ $(2.102)$	$3.160$ $\pm 0.0164$ $(5.759)$
	Single-target	Family	$0.8439$ $\pm 0.0201$ $(1.162)$	$147.0$ $\pm 17.239$ $(500.2)$	$2.563$ $\pm 0.0315$ $(4.500)$	$4.627$ $\pm 0.0680$ $(7.142)$
	Family	Family	$1.093$ $\pm 0.0069$ $(1.492)$	$6.112$ $\pm 0.0565$ $(7.823)$	$2.127$ $\pm 0.0138$ $(3.340)$	$4.295$ $\pm 0.0227$ $(6.514)$
Funnel	Single-target	Single-target	$0.3999$ $\pm 0.0002$ $(0.5568)$	$0.1336$ $\pm 0.0009$ $(0.1866)$	$0.2656$ $\pm 0.0062$ $(0.3187)$	$0.6306$ $\pm 0.0416$ $(2.064)$
	Family	Single-target	$0.4041$ $\pm 0.0032$ $(0.5610)$	$0.1309$ $\pm 0.0023$ $(0.1845)$	$0.2647$ $\pm 0.0081$ $(0.3222)$	$0.6437$ $\pm 0.0836$ $(2.051)$

Appendix

Table 22: Experimental results showing the performance of AAIS amortised over a single target vs over a whole family, all targets are at

d=25

. Metrics reported follow the convention of Table 1 .

Explore similar work

Amortised Inference through One-Step Implicit Sampling
Oct 8, 2026Vincent Pauline, Kirill Tamogashev, Arran Carter +3Amortized Inference Posterior Sampling
Technical University of Munich · Munich Center for Machine Learning · University of Edinburgh +1
Adaptive teachers for amortized samplers
Oct 2, 2024Minsu Kim, Sanghyeok Choi, Taeyoung Yun +7Amortized Inference RL Exploration
Mila, KAIST · KAIST · Recursion +1
Beyond Effective Sample Size: Effective Number of Proposals for Adaptive Importance Sampling
Aug 15, 2026Ali Mousavi, Victor ElviraImportance Sampling
Department of Computer Engineering, Islamic Azad University, Neyshabur Branch, Neyshabur, Iran · School of Mathematics, University of Edinburgh, United Kingdom

Oct 8, 2026J/K move · Enter open · S save

Amortised Inference through One-Step Implicit Sampling

Vincent Pauline, Kirill Tamogashev, Arran Carter, Sanghyeok Choi+2

Technical University of Munich · Munich Center for Machine Learning · University of Edinburgh+1

Amortized Inference Posterior Sampling Markov Chain Monte Carlo One-Step Generative Modeling

Open paper AskPDF arXiv

Efficient sampling from Bayesian posterior distributions in high dimensions remains a hard problem, despite advances in theory and algorithms. Multiple steps are usually required to draw unbiased samples, for example, iterations of an MCMC algorithm or integration steps in a diffusion-based amortised sampler. However, scalable one-step amortised inference remains a challenging open problem. In this work, we build upon recent progress in data-driven generative modelling to offer a possible solution for one-step amortised sampling. Our one-step implicit sampler (OSIS) uses short-time MCMC dynamics to define a local target for the evolution of an implicitly modelled distribution. In the asymptotic limit, the modelled distribution is a fixed point of the MCMC kernel and thus samples from the target. We analyse the convergence properties of OSIS and show connections with other generative modelling and sampling algorithms. We study various design choices and show that OSIS performs well compared to multi-step amortised samplers and non-amortised MCMC baselines while being far more efficient at sampling time.

Abstract

Explore similar work

Amortised Inference through One-Step Implicit Sampling

Adaptive teachers for amortized samplers

Beyond Effective Sample Size: Effective Number of Proposals for Adaptive Importance Sampling