Density Ratio Estimation with Stein Displacement Fields

Song Liu

stat.MLOct 8, 2026

Density Ratio Estimation with Stein Displacement Fields

Authors: Song Liu

Organizations: University of Bristol

Abstract

Density ratios quantify distribution shift from a probability-mass point of view, whereas displacement fields describe, from a dynamical point of view, how one distribution is transported onto another. Although both offer complementary insights, they are usually estimated separately, and converting one into the other requires post-processing. In this paper, we estimate the density ratio between a target and a base distribution by parametrizing it through a displacement field acting on the base: the log-ratio is modeled as minus the Stein operator of the base applied to the field, up to a normalizing constant. This gives both statistical and dynamical descriptions of the distribution shift through a single convex optimization problem. Iterating this estimate-and-move step gives two inference algorithms: push-forward moves the model and corrects a pretrained sampler without retraining it, whereas pull-back moves the data closer to the base and fits a transformation model one layer at a time. Applications to distribution shift in simulation-based inference and to nonlinear independent component analysis illustrate the benefits and limitations of the approach.

Figures & tables

Panel of Figure 1 — Figure 1: Layer construction in the push-forward and pull-back algorithms. Push-forward appends $T_{k}$ to the model output; pull-back applies $T_{k}^{-1}$ to the current latent samples and refits the base between layers. In both panels, $\Phi_{k}$ maps the latent base to data.

	Prior change			Simulator change		Partial coverage
Method	unimodal	bimodal	GLM	$n_{\rm real}{=}500$	$n_{\rm real}{=}2000$	covered	uncovered
Frozen NPSE	0.724 $\pm$ .001	0.760 $\pm$ .001	0.619 $\pm$ .002	0.619 $\pm$ .004	0.619 $\pm$ .004	0.632 $\pm$ .001	0.638 $\pm$ .003
PriorGuide, ODE	0.638 $\pm$ .028	0.634 $\pm$ .017	0.665 $\pm$ .014	$\times$	$\times$	$\times$	$\times$
PriorGuide + Langevin ∗†	0.511 $\pm$ .002	0.512 $\pm$ .001	0.543 $\pm$ .020	$\times$	$\times$	$\times$	$\times$
Classifier DRE + SIR ∗	0.816 $\pm$ .005	0.862 $\pm$ .003	0.717 $\pm$ .005	0.722 $\pm$ .009	0.721 $\pm$ .007	0.727 $\pm$ .004	0.759 $\pm$ .007
DG + Langevin ∗	0.531 $\pm$ .005	0.555 $\pm$ .002	0.559 $\pm$ .005	0.589 $\pm$ .002	0.575 $\pm$ .004	0.607 $\pm$ .009	0.859 $\pm$ .036

Table 1: Posterior correction in simulation-based inference: C2ST against reference posteriors (

0.5

= indistinguishable; mean

\pm

std). Best per column in bold, second best underlined (reference rows excluded). ∗ No tractable density. † Hyperparameters selected by test C2ST.

\times

: not applicable; –: not evaluated. The last row reports retraining on new target simulations as a reference. Experimental details are in Appendix I .

	Synthetic ( $d{=}5$ )		CMU ARCTIC		LibriSpeech
Method	$n_{c}{=}200$	$n_{c}{=}1000$	$d{=}3$	$d{=}5$	$d{=}3$	$d{=}5$
iVAE	0.484 $\pm$ 0.045	0.492 $\pm$ 0.067	0.538 $\pm$ 0.090	0.383 $\pm$ 0.034	0.578 $\pm$ 0.131	0.461 $\pm$ 0.069
GIN	0.602 $\pm$ 0.087	0.635 $\pm$ 0.055	0.645 $\pm$ 0.084	0.440 $\pm$ 0.037	0.575 $\pm$ 0.077	0.438 $\pm$ 0.037
SDRE (from scratch)	0.689 $\pm$ 0.061	0.673 $\pm$ 0.062	0.621 $\pm$ 0.150	0.518 $\pm$ 0.047	0.770 $\pm$ 0.068	0.537 $\pm$ 0.026
OSF ∗	0.795 $\pm$ 0.068	0.762 $\pm$ 0.084	0.854 $\pm$ 0.024	0.747 $\pm$ 0.046	0.850 $\pm$ 0.014	0.736 $\pm$ 0.046
GIN + SDRE	0.660 $\pm$ 0.104	0.687 $\pm$ 0.066	0.747 $\pm$ 0.090	0.587 $\pm$ 0.081	0.588 $\pm$ 0.056	0.467 $\pm$ 0.043

Table 2: Nonlinear ICA: validation MCC (mean

\pm

std). Top: standalone estimators; bottom: pretrained models refined by SDRE . Best per column within each block is bold, excluding the starred standalone row. ∗ Uses oracle knowledge of the mixing architecture. Experimental details are in Appendix H .

	Synthetic		ARCTIC		LibriSpeech
	$n_{c}{=}200$	$n_{c}{=}1000$	$d{=}3$	$d{=}5$	$d{=}3$	$d{=}5$	wins
GIN	0.602	0.635	0.645	0.440	0.575	0.438
Extended training	0.619	0.658	0.623	0.433	0.587	0.445
+ SDRE	0.660	0.687	0.747	0.587	0.588	0.467	19/22
OSF	0.795	0.762	0.854	0.747	0.850	0.736
Extended training	0.837	0.856	0.858	0.771	0.833	0.790

Table 3: Refinement versus extended training: validation MCC (seed means). Extended training uses

4\times

the original training steps; “wins” counts seeds on which refinement beats extended training. Best per column within each model is bold. Experimental details are in Appendix H .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 2 — Figure 2: Iteration loops for the two inference algorithms of Section 4 .

	Synthetic		ARCTIC		LibriSpeech
	$n_{c}{=}200$	$n_{c}{=}1000$	$d{=}3$	$d{=}5$	$d{=}3$	$d{=}5$	wins
Structured flow (true arch.)	0.591 $\pm$ 0.071	0.574 $\pm$ 0.061	0.767 $\pm$ 0.144	0.614 $\pm$ 0.013	0.750 $\pm$ 0.139	0.651 $\pm$ 0.075
Extended training	0.604 $\pm$ 0.074	0.595 $\pm$ 0.072	0.759 $\pm$ 0.134	0.670 $\pm$ 0.095	0.756 $\pm$ 0.143	0.673 $\pm$ 0.065
+ SDRE	0.768 $\pm$ 0.076	0.734 $\pm$ 0.073	0.783 $\pm$ 0.166	0.704 $\pm$ 0.074	0.792 $\pm$ 0.160	0.712 $\pm$ 0.123	19/22

Appendix

Table 4: The true-architecture structured flow (leaky-ReLU activation), complementing Tables 2 and 3 : validation MCC (mean

\pm

std over mixing seeds). Last column: seeds on which refinement beats extended training.

Pretrained model	no refinement	all 40 layers	oracle layer	$\kappa=1$	$\kappa=4$	$\kappa=16$
all	0.706	0.770	0.777	0.736 (15.2)	0.717 (5.2)	0.705 (1.4)
GIN	0.589	0.662	0.664	0.630 (8.6)	0.617 (3.4)	0.595 (0.4)
Structured (true)	0.706	0.769	0.788	0.726 (23.7)	0.694 (7.4)	0.691 (2.4)
Structured (tanh)	0.824	0.878	0.879	0.851 (13.4)	0.841 (4.9)	0.829 (1.3)

Appendix

Table 5: Stopping refinement by

\hat{J}<\kappa\,m/(2n)

, post hoc on 42 traced 40-layer refinements (ARCTIC and LibriSpeech

d=3

, ARCTIC 20 s

d=3

, synthetic

n_{c}=200

). Mean validation MCC at the stopping layer (mean number of accepted layers).

	$d=3$		$d=5$
Method	5 s	20 s	5 s	20 s
iVAE	0.538 $\pm$ 0.090	0.573 $\pm$ 0.033 †	0.383 $\pm$ 0.034	0.494 $\pm$ 0.014
GIN	0.645 $\pm$ 0.084	0.525 $\pm$ 0.057	0.440 $\pm$ 0.037	0.432 $\pm$ 0.019
Structured flow (true arch.)	0.767 $\pm$ 0.144	0.792 $\pm$ 0.073	0.614 $\pm$ 0.013	0.627 $\pm$ 0.058
Structured flow (tanh)	0.854 $\pm$ 0.024	0.816 $\pm$ 0.017	0.747 $\pm$ 0.046	0.711 $\pm$ 0.062
SDRE (from scratch)	0.621 $\pm$ 0.150	0.598 $\pm$ 0.057	0.518 $\pm$ 0.047	0.480 $\pm$ 0.007

Appendix

Table 6: ARCTIC with 5 s versus 20 s per source (validation MCC, mean

\pm

std over 3 seeds). The 20 s setting uses seven speakers with a different speaker subset per seed. † One iVAE run diverged; mean over 2 seeds.

Method	smooth_polarizers	color_mix
Supervised bound	0.946 $\pm$ 0.002	0.949 $\pm$ 0.008
Principal components (no unmixing)	0.754 $\pm$ 0.002	0.582 $\pm$ 0.013
FastICA (linear)	0.878 $\pm$ 0.001	0.815 $\pm$ 0.006
iVAE	0.798 $\pm$ 0.034	0.583 $\pm$ 0.016
GIN	0.638 $\pm$ 0.001	0.533 $\pm$ 0.026
Structured flow (true arch.)	0.665 $\pm$ 0.004	0.615 $\pm$ 0.006

Appendix

Table 7: Causal Chambers light tunnel ( lt_walks_v1 ): photodiode readings projected on their top three principal components (the six sensors come in near-duplicate pairs), 20 time segments, validation MCC against the recoverable actuators, mean

\pm

std over 3 train/validation splits. Targets: red, blue and

\cos^{2}

of the polarizer angle ( smooth_polarizers ); red, green, blue ( color_mix ). The supervised bound regresses each target on the inputs with random Fourier features.

Explore similar work

Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression
Sep 14, 2026Ren-Rui Liu, Zheng-Chu GuoKernel Regression Distribution Shift
School of Mathematical Sciences, Zhejiang University, Hangzhou 310058, China
Augmented Inverse Hybrid Weighting: Robust Inference under Deterministic and Random Distribution Shifts
Aug 1, 2026Ying Jin, Ying Jin, Dominik RothenhäuslerInverse Probability Weighting Distribution Shift Robustness
Factorizable joint shift revisited
Jan 21, 2026Dirk TascheDistribution Shift Robustness Label Shift
Centre for Business Mathematics and Informatics, North-West University, South Africa

Sep 14, 2026, stat.MLJ/K move · Enter open · S save

Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression

Ren-Rui Liu, Zheng-Chu Guo

School of Mathematical Sciences, Zhejiang University, Hangzhou 310058, China

Kernel Regression Distribution Shift Density Ratio Estimation

Open paper AskPDF arXiv

We study density ratio estimation and importance-weighted regression under target shift with continuous outputs. Under target shift, the conditional distribution of the inputs given the outputs remains invariant across the training and test distributions, while the output marginal distribution may change. Although this problem has been extensively studied for discrete outputs, the continuous setting is substantially less understood: the importance weights are determined by an unknown density ratio function, for which existing estimation methods lack explicit finite-sample convergence rates. We propose a spectral regularization method in a reproducing kernel Hilbert space (RKHS) for estimating the continuous density ratio from labeled training samples and unlabeled test inputs. Under a source condition with regularity parameter

ι>0

, we establish high-probability finite-sample guarantees and show that the estimator achieves the capacity-independent minimax-optimal RKHS-norm rate

O(n_η^{-ι/(2ι+2)})

. We then incorporate the estimated density ratio into importance-weighted regression and characterize the propagation of density-ratio estimation error to the final predictor. When sufficiently many samples are available for density ratio estimation, the resulting regression estimator attains the minimax-optimal rates of standard kernel regression. These results establish a finite-sample theory for continuous density ratio estimation and importance-weighted learning under target shift.

Abstract

Explore similar work

Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression

Augmented Inverse Hybrid Weighting: Robust Inference under Deterministic and Random Distribution Shifts

Factorizable joint shift revisited