When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning

Wang Ru

cs.LGOct 20, 2026

When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning

Authors: Wang Ru

Organizations: Department of Chemical Engineering, Tsinghua University

Abstract

Latent diffusion models almost universally place a Gaussian prior on the VAE latent space. For base generation this default is unproblematic; under reinforcement-learning fine-tuning, however, it carries a structural limitation. The standard VAE's KL term does not merely ignore mode structure - it actively taxes separation and tightness. When data are multimodal, the ELBO optimum compresses distinct modes into an overlapping latent. RL fine-tuning amplifies this overlap: the score network is under-resolved on the saddle region between modes, dynamic-to-variational amplification rises sharply, and the collapse boundary beta_c is pushed far above its variational scale. On the real alanine-dipeptide Boltzmann distribution, a Gaussian-prior VAE collapses at beta_c = 0.224 while a mode-tracked Gaussian-mixture prior collapses at 0.00316, a 71x gap from the same pipeline. We trace this gap to a single geometric mediator, d_min/(2 sigma_0), and verify the causal chain through controlled interventions: moving geometry at fixed prior, and switching prior at matched geometry (full mediation; prior-label coefficient 0.995x, 95% CI [0.79, 1.25]). A three-source decomposition localises the amplification to the learned policy (zero ODE-solver contribution, bounded anchor mismatch); the coverage plateau is an unconditional benefit of mode-separating priors that no amount of RL capacity recovers. The remedy is conditional: a non-Gaussian prior is necessary precisely when the latent geometry would fall into overlap and the task is coverage-sensitive. A machine-checked theory skeleton supports the KL pricing structure - a divergent compactness tax under a Gaussian prior versus a finite assignment fee under a mixture - and the position-versus-plateau distinction delimits what is finite-resource from what requires changing the prior.

Figures & tables

Figure 1: Overview. (a) The data are multimodal: several separated modes in data space. (b) The VAE prior sets the latent separation-to-width ratio

d_{\min}/(2\sigma_{0})

: under a Gaussian prior the modes can overlap, while a mixture prior keeps them separated. (c) Under KL-anchored RL fine-tuning, the collapse boundary

\beta_{c}

and the recovery plateau depend on that geometry (schematic; measured curves in §5).

condition	non-Gaussian prior needed?
multimodal data $+$ tight latent dimension	yes — Gaussian ELBO packs modes into overlap
RL fine-tuning with a single-target reward	yes — RL amplifies whatever overlap exists
unimodal data, or a wide latent dimension	no — the Gaussian prior suffices
base generation only, no RL fine-tuning	no — overlap does not hurt the base model

Table 1: Decision rule for prior choice. The failure mode is joint: multimodality creates the overlap and RL fine-tuning amplifies it. Neither factor alone requires changing the prior.

Figure 2: Experiment Z: latent geometry during VAE training. Identical architecture and pack (

K{=}10

d{=}8\to 2

d_{\rm sep}{=}5

) trained under

\mathcal{N}(0,I)

and under a label-tracked GMM prior (VaDE-style, switched on at step 3000; 3 seeds, shaded min–max). The arms coincide until the switch, then diverge: Gaussian stalls at the overlap threshold (

1.14\pm 0.07

), the mixture reaches

5.0

(

3.3

–

6.6

Quantity	Gaussian prior	GMM prior
$\sigma_{\mathrm{lat}}$	0.96	0.027
$\beta_{\mathrm{cross}}=2\sigma_{0}^{2}$	1.84	$1.4\times 10^{-3}$
empirical $\beta_{c}^{\mathrm{lat}}$	0.224	$3.16\times 10^{-3}$
$\alpha=\beta_{c}/\beta_{\mathrm{cross}}$	0.12	2.24
$d_{\min}/\sigma$	1.61	2.06

Table 2: Crossover scale vs. empirical

\beta_{c}

(Exp7 seed 0). All quantities are in the latent 2D space.

Quantity	Gaussian	GMM	Ratio
$\beta_{c}^{\mathrm{var}}$ (analytic)	$2.41\times 10^{-5}$	$3.49\times 10^{-5}$	$0.69\times$
$\beta_{c}^{\mathrm{dyn}}$ (empirical)	$0.224$	$3.16\times 10^{-3}$	$70.8\times$
dyn / var	—	$83\times$	—

Table 3: Variational vs. dynamic

\beta_{c}

(Exp7 seed 0). Variational

\beta_{c}

is computed analytically from the reverse-KL exact optimum using the non-central chi-square ball-mass integral. Dynamic

\beta_{c}

is the empirical score-matching + ODE sampling result. Note: the Gaussian prior’s

\beta_{c}^{\mathrm{var}}

lies in the geometric-saturation regime (

d_{\min}/(2\sigma_{0})<1

), where the ball-mass integral loses geometry dependence; the

0.69\times

variational ratio is therefore unreliable as a quantitative predictor for the Gaussian prior, and the entire empirical

71\times

gap must be attributed to the dynamic error amplification mechanism (( dyn-var-mech )).

Figure 3: Theory validation:

\beta_{c}

vs. reward temperature

\lambda

on the three-basin alanine torus (20 ns TorusKDE; nearest basin centres

1.35

rad apart, KDE bandwidth

0.185

rad). Left : empirical

\beta_{c}

(blue circles, reverse-KL) scales linearly with

\lambda

(slope

0.96

, theory

1.0

), matching the prediction

\beta_{\mathrm{cross}}=2\lambda\sigma_{0}^{2}

(orange squares). Forward-KL

\beta_{c}

(green triangles) has a steeper resolved slope and a non-monotone offset (the

\lambda=0.1

point is censored below

10^{-6}

; Appendix C ). Right : empirical vs. theoretical

\beta_{c}

for all four

\lambda

values. Points lie within

2.5\times

of the identity line (dashed), confirming that for well-separated basins the variational bound is quantitatively predictive. Data from boundary_l{0.1,1,10,100}.json .

Figure 4: Comprehensive scaling law:

\beta_{c}

vs. mode geometry parameters across 72 well-separated configurations (

K=5

–

20

d=3

–

5

, dim=1–5D,

\lambda=0.1

–

100

\beta_{c}

decreases with normalised mode separation

d/\sigma_{0}

(each panel, log-vertical scale), consistent with the theoretical crossover-scale prediction. Figure from fig3_scaling_law.pdf .

Figure 5: Score-network approximation error at mode saddle points. The 2-layer MLP score network’s MSE (color) peaks at saddle-point manifolds between modes, where the true tilted-GMM gradient is most non-linear. Well-separated geometries with small saddle MSE show

\mathrm{dyn}/\mathrm{var}\approx 2\times

. The peaks carry negligible density-weighted mass (Appendix D ), a direct controlled-trajectory test (Appendix E ) shows collapsing fine-tunes do not generally reroute through saddle strips, and the three-source decomposition (Appendix F ) locates the boundary gap in a static policy-side representation/optimisation deficit instead. Figure from figA_saddle_mse.pdf .

Quantity	Gaussian	GMM	Interpretation
$\beta_{c}$ (collapse position)	$100$	$100$	Main Line 1: prior-independent
Plateau coverage ( $\beta=10000$ )	$0.70$	$1.00$	Main Line 2: prior matters
Raw-prior coverage	$0.80$	$1.00$	GMM target captures all modes

Table 4: Exp5A: 1D

K=10

mode target, prior comparison. Collapse position is identical (

\beta_{c}=100

) for both priors, but plateau coverage differs dramatically. Results from prior_beta_c.json + prior_beta_c_extra.json (extended scan to

\beta=10000

Figure 6: Exp5A: Coverage vs.

\beta

for Gaussian and GMM priors on a 1D

K=10

mode target. Collapse position is identical at

\beta_{c}=100

(vertical dashed line, both priors cross

0.5

at the same

\beta

), validating Main Line 1. Plateau coverage differs at

\beta\gg 1000

: GMM recovers

100\%

, Gaussian plateaus at

70\%

, validating Main Line 2. Data from prior_beta_c.json + extended scan.

Seed	$\beta_{c}^{\mathrm{latent}}$		plat dec		dyn/var
	Gauss	GMM	Gauss	GMM	Gauss	GMM
0	1.50	1.16	0.63	0.70	0.83	0.67
1	1.83	1.17	0.58	0.73	0.88	0.82
2	1.55	1.90	0.53	0.77	0.67	1.18
mean	1.63	1.41	0.58	0.73	0.79	0.89

Table 5: Exp6 K=30:

\beta_{c}

, plateau coverage, and dynamic-to-variational ratio across seeds.

\beta_{c}^{\mathrm{var}}

is the analytic reverse-KL optimum;

\mathrm{dyn}/\mathrm{var}=\beta_{c}^{\mathrm{dyn}}/\beta_{c}^{\mathrm{var}}<1

here because the variational bound is an upper bound in well-separated geometries.

Figure 7: Exp6 K=30: Latent geometry and coverage in the crowded regime. Top : VAE latent space projections (2D, seed 0) for Gaussian (left) and GMM (right) priors, coloured by true mode identity. The Gaussian prior compresses $K=30$ modes into an overlapping envelope ( $d_{\min}/(2\sigma_{0})=0.79$ ), while the GMM prior — shown here as an illustrative example of a mode-separating prior — maintains separated clusters ( $d_{\min}/(2\sigma_{0})=1.60$ ). Bottom : Coverage vs. $\beta$ across 3 independent seeds (mean $\pm$ $1\sigma$ ; faint lines are individual seeds). Solid lines = latent coverage, dashed = decoded coverage, blue = Gaussian prior, orange = GMM prior. Vertical dotted lines mark each prior’s $\beta_{c}^{\mathrm{latent}}$ . The collapse positions are close for both priors (Main Line 1 holds in well-separated regimes), but the decoded coverage plateau is consistently higher for the GMM prior across all seeds (Main Line 2). Latent projections are shown for seed 0; the per-seed collapse direction (GMM decoded plateau $>$ Gaussian) is consistent across all three seeds (Fig. 12 ), and under seeds where the GMM geometry drifts near the overlap threshold the advantage shrinks accordingly — reinforcing that geometry, not the prior label, determines the outcome (Discussion).

Seed	$\beta_{c}^{\mathrm{lat}}$		$\beta_{c}^{\mathrm{dec}}$		$\sigma_{\mathrm{lat}}$		gap
	Gauss	GMM	Gauss	GMM	Gauss	GMM	ratio
0	0.224	0.00316	0.01	$\leq 10^{-6}$	0.958	0.027	$71\times$
1	0.224	$\leq 0.01$	0.014	$\leq 0.01$	0.955	0.027	$\geq 22\times$
2	0.014	$\leq 0.01$	0.01	$\leq 0.01$	1.14	0.030	$\geq 1.4\times$

Table 6: Exp7:

\beta_{c}

across seeds (20 ns MD). A 60 ns re-run with the same priors gives identical Gaussian-prior

\beta_{c}

(0.224 for seeds 0/1) and GMM-prior

\beta_{c}\leq 0.01

for all seeds (see cross-MD robustness paragraph).

Figure 8: Exp7: Latent geometry and coverage for alanine dipeptide. Top left : Real MD data in $(\phi,\psi)$ torsion space, coloured by KDE basin assignment (beta/C5, C7eq, alpha_R). Top middle : Gaussian prior VAE latent space (seed 0) — the three basins overlap ( $d_{\min}/(2\sigma_{0})=0.81$ , cls_acc $=0.875$ ). Top right : GMM prior VAE latent space (seed 0) — the three basins form clearly separated clusters ( $d_{\min}/(2\sigma_{0})=1.03$ , cls_acc $=0.919$ ). Bottom : Latent coverage vs. $\beta$ for Gaussian (blue) and GMM (orange) priors (3 seeds, mean $\pm$ $1\sigma$ ). The Gaussian prior collapses at $\beta_{c}=0.224$ , while the GMM prior collapses at $\beta_{c}\approx 3.2\times 10^{-3}$ (grid-limited, bisect refines to exact value). The $71\times$ gap arises because the Gaussian prior’s latent geometry enters the mode-overlap regime ( $d_{\min}/(2\sigma_{0})=0.81$ ). Latent projections are from seed 0: the Gaussian prior systematically packs the three basins into an overlapping envelope, while the GMM prior is shown as an illustrative example of a mode-separating prior, not a universal guarantee — under seeds where its geometry drifts near the overlap threshold the gap vanishes (Discussion), reinforcing that geometry, not the prior label, determines the outcome. Per-seed curves for all three seeds are in Fig. 12 .

Figure 9: Bisect experiment on exp7 GMM prior, seed 0. 16-point

\beta

grid from

10^{-6}

10

. Latent coverage (blue) collapses to 0.33 for

\beta\leq 2\times 10^{-3}

, recovers to 1.0 at

\beta\geq 10^{-2}

, giving

\beta_{c}=3.16\times 10^{-3}

. Dashed red line marks the crossover scale

\beta_{\mathrm{cross}}=2\lambda\sigma_{0}^{2}=1.4\times 10^{-3}

from ( 3.2 ); the transition sits just above it.

Figure 10: Hard vs. soft latent coverage along the reverse-KL fine-tuning scan for Exp7 seed 0. Left : Gaussian prior (overlap regime,

d_{\min}/\sigma_{0}=1.61

). The hard metric (solid circles) collapses to

1/3

\beta=0.1

and saturates at

1.0

above

\beta=5

; the soft metric (open squares) reveals

\sim 23\%

residual mass on the two hard-“lost” modes and approaches its

0.95

plateau only asymptotically. The hard collapse boundary (

\beta_{c}^{\mathrm{hard}}=0.224

) is therefore

\gtrsim 2.2\times

larger than the soft boundary (

\beta_{c}^{\mathrm{soft}}<0.1

). Right : GMM prior (separated,

d_{\min}/\sigma_{0}=2.06

). Hard coverage is

1.0

across the full scan; soft coverage is flat at

\sim 0.94

, i.e. no collapse–recovery transition occurs (the bisect boundary is

3.16\times 10^{-3}

). The dotted line marks the

0.5

recovery threshold.

	Gaussian prior ( $d_{\min}/\sigma_{0}=1.61$ )		GMM prior ( $d_{\min}/\sigma_{0}=2.06$ )
$\beta$	hard	soft	hard	soft
$0.1$	0.333	0.560	1.000	0.847
$0.5$	0.667	0.652	1.000	0.929
$1$	0.667	0.717	1.000	0.935
$5$	1.000	0.850	1.000	0.934
$10$	1.000	0.892	1.000	0.937

Table 7: Per-

\beta

hard vs. soft latent mode coverage for Exp7 seed 0 along the reverse-KL fine-tuning scan (same runs as Figure 10 ; values from column 6 of the curve arrays in results/exp7_soft_s0_* ).

\beta_{c}

is log-interpolated at the

0.5

recovery threshold;

<\!0.1

marks metrics that stay above threshold across the whole scan (below the scan floor). The GMM hard boundary is below the scan floor and the dedicated bisect run locates it at

\beta_{c}=3.16\times 10^{-3}

71\times

gap to the Gaussian

0.224

). “pre-FT” is coverage before fine-tuning; “plateau” is the mean of the two highest-

\beta

points (

\beta=5000,10000

). The soft metric never reaches

1.0

because the Gaussian assignment kernel has finite bandwidth overlap with neighbouring centres.

Basin	20 ns frame frac.	60 ns frame frac.
C7eq (dominant)	43.5%	43.4%
$\alpha_{R}$ (secondary)	33.8%	35.2%
$\beta$ /C5 (rarest)	15.7%	14.4%

Table 8: Basin frame-fraction comparison: 20 ns vs. 60 ns MD. TorusKDE finds the same three basins at matching

(\phi,\psi)

positions in both trajectories (peak coordinates and KDE masses in Appendix A , Table 10 ); only the occupancies shift mildly. Despite the

3\times

longer trajectory, the Gaussian-prior VAE produces identical

\beta_{c}

in both, confirming that

\beta_{c}

depends on latent geometry, not on MD sampling length.

Figure 11: The bridge mechanism:

\mathrm{dyn}/\mathrm{var}

vs.

d_{\min}/(2\sigma_{0})

across all experiments. Clean data points (green) show a decreasing trend: well-separated modes give

\mathrm{dyn}/\mathrm{var}\approx 10

–

40\times

. The Exp7 bisect point (orange star, boundary-exact) gives

\approx 83\times

d_{\min}/(2\sigma_{0})\approx 1.03

. Artifact points (red X, K=3 geometric saturation / scan floor) are excluded from the trend. Overlap regimes (

d_{\min}/(2\sigma_{0})<1

) show much larger amplification, explaining the

71\times

\beta_{c}

gap in Exp7’s Gaussian prior.

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

minimum KDE mass	$5\%$	$2\%$	$1\%$	$0.5\%$
20 ns ( $n=40{,}000$ )	3	3	3	3
60 ns ( $n=60{,}626$ )	3	3	3	3

Appendix

Table 9: Number of TorusKDE basins detected as a function of the minimum-mass threshold.

K=3

in both trajectories at every threshold down to

0.5\%

Basin	20 ns	60 ns	20 ns	60 ns	20 ns	60 ns
	$(\phi,\psi)$ peak		KDE mass		frame fraction
C7eq (dominant)	$(-69.3,152.1)$	$(-71.1,152.1)$	44.52%	44.59%	43.46%	43.39%
$\alpha_{R}$ (secondary)	$(-72.9,-17.1)$	$(-72.9,-17.1)$	38.87%	40.20%	33.82%	35.16%
$\beta$ /C5 (rarest)	$(-146.7,157.5)$	$(-146.7,157.5)$	16.61%	15.21%	15.71%	14.40%
unassigned ( $>0.8$ rad from any peak)	—		—		7.01%	7.05%

Appendix

Table 10: Physical basins of the two MD trajectories. All coordinates in degrees. “KDE” = integrated TorusKDE mass; “frame” = fraction of raw frames within the

0.8

rad labelling cap. The same three peak locations are found in both trajectories (maximal peak shift

1.8^{\circ}

); the rarest basin is the same

\beta

/C5 extended conformation at

(-146.7^{\circ},+157.5^{\circ})

in both, only

8.1^{\circ}

from the canonical C5 reference

(-150.1^{\circ},+150.1^{\circ})

MD	seed	prior	$\sigma_{\mathrm{lat}}$	$d_{\min}/\sigma$	cls	$\beta_{c}^{\mathrm{lat}}$	$\beta_{c}^{\mathrm{dec}}$	plat lat	plat dec
20 ns	0	Gaussian	0.958	1.61	0.875	0.224	$\leq 0.01$	1.00	1.00
20 ns	0	GMM	0.027	2.06	0.919	$\leq 0.01$	$\leq 0.01$	1.00	1.00
20 ns	1	Gaussian	0.955	1.56	0.875	0.224	0.014	1.00	0.67
20 ns	1	GMM	0.027	1.82	0.911	$\leq 0.01$	$\leq 0.01$	1.00	1.00
20 ns	2	Gaussian	1.140	0.99	0.898	0.014	$\leq 0.01$	1.00	1.00
20 ns	2	GMM	0.030	1.71	0.917	$\leq 0.01$	$\leq 0.01$	1.00	1.00

Appendix

Table 11: Complete Exp7 record, 3 seeds

\times

2 priors

\times

2 MD lengths. cls = nearest-mean basin classification accuracy of the latent codes;

\beta_{c}^{\mathrm{lat}}

\beta_{c}^{\mathrm{dec}}

= latent/decoded collapse boundaries (

0.5

coverage threshold, log-interpolated when exact); plat lat /plat dec = mean coverage at the two highest-

\beta

scan points. Source: results/exp7{,_s1,s2}/{gaussian,gmm}.json and results/exp7_60ns_s{0,1,2}{gaussian,gmm}.json .

Figure 12: Per-seed fine-tuning coverage curves for Exp6 (K=30, top row) and Exp7 (alanine, bottom row). Solid lines with markers show latent coverage; dashed lines show decoded coverage. Blue circles = Gaussian prior, red squares = GMM prior. The horizontal line marks the 0.5 coverage threshold used to define

\beta_{c}

. The Gaussian/GMM direction never flips across any seed; the Gaussian seed-2 anomaly (wider latent,

d_{\min}/\sigma\approx 1.0

) appears under both 20 ns and 60 ns MD (Table 11 ), confirming it is VAE-initialisation-driven.

	empirical $\beta_{c}$			numeric theory $\beta_{c}$
$\lambda$	reverse	forward	fwd/rev	reverse	forward	fwd/rev
0.1	0.122	$<10^{-6}$	—	0.0493	0.0612	1.24
1	0.914	0.157	0.17	0.493	0.612	1.24
10	11.34	31.06	2.74	4.93	6.12	1.24
100	85.05	299.6	3.52	49.25	61.19	1.24

Appendix

Table 12: Reverse- and forward-KL collapse boundaries on the alanine torus at four reward temperatures. Empirical columns carry the bisect flag; the

\lambda=0.1

forward boundary is censored (“below”): coverage is still

0.67

at the scan floor

10^{-6}

. The theory predicts a fixed fwd/rev ratio

1.24

at every

\lambda

, whereas the empirical ratio is non-monotone (censored, then

0.17

, then

2.7

–

3.5

). At

\lambda=1

, forward-directed fine-tuning retains only

0.33

target-basin coverage at the reverse collapse boundary ( cov_fwd_at_rev_bc ). Source: results/remote_bundle_v2/boundary_l{0.1,1.0,10.0,100.0}.json .

dim	boundary	$n$	$p_{\lambda}$	$p_{d}$	$p_{K}$	$R^{2}$
1	empirical reverse	48	1.013	$2.033$	$2.311$	0.994
1	empirical forward	48	0.777	$-0.788$	$-0.448$	0.802
1	numeric theory	44	1.000	$2.484$	$2.180$	0.995
2	empirical reverse	18	1.016	$2.739$	$0.833$	0.992
2	empirical forward	18	0.679	$1.424$	$-1.711$	0.545
2	numeric theory	18	1.000	$2.764$	$0.886$	0.999

Appendix

Table 13: OLS scaling exponents for empirical reverse-KL, empirical forward-KL, and numeric-theory boundaries, by ambient dimension.

n

= number of resolved configurations (of

64

in 1D,

18

in 2D/3D). Reverse-KL exponents track the theory closely with

p_{\lambda}\approx 1

and

R^{2}\geq 0.85

. Forward-KL fits are poor (

R^{2}=0.55

–

0.80

) with

p_{\lambda}\leq 0.79

and negative

p_{K}

, i.e. the forward boundary is censored/non-monotone over much of the grid and is not described by a clean power law. Source: results/remote_bundle_v2/scaling_by_dim.json .

$K$	$\sigma$	$d/(2\sigma)$	mode	saddle	valley	MC	sad/mode
5	0.25	10.0	0.418	251.82	145.74	0.452	603.1
10	0.25	10.0	1.119	143.80	61.78	1.257	128.6
20	0.25	10.0	0.831	225.94	130.65	0.854	271.9
5	0.50	5.0	0.221	15.82	11.40	0.263	71.6
10	0.50	5.0	0.366	8.37	8.35	0.399	22.9
20	0.50	5.0	0.387	13.55	7.95	0.423	35.0

Appendix

Table 14: 2D width sweep at

t=0.02

(

d=5

fixed; 40k-step seed-0 models): per-region score MSE, density-weighted MC MSE, and saddle/mode ratio. The gap is maximal for narrow modes (

\sigma/d=0.05

d/(2\sigma)=10

) and closes as modes broaden; the MC MSE stays within

1.4\times

of the mode MSE throughout. Source: results/expA_learn/2d_K_sig_s0.json , field score_mse.t0.02 .

dim	$K$	$d$	mode	saddle	valley	MC	sad/mode	cover
2	5	3	0.072	0.048	0.864	0.104	0.67	1.00
2	5	5	0.098	0.301	1.184	0.127	3.08	1.00
2	10	3	0.107	0.055	0.672	0.118	0.51	1.00
2	10	5	0.199	0.240	0.790	0.222	1.20	1.00
2	20	3	0.117	0.132	0.335	0.121	1.13	1.00
2	20	5	0.165	0.358	0.582	0.191	2.18	1.00

Appendix

Table 15: Lattice packs

2

–

5

t=0.02

\sigma=1

: per-region score MSE, saddle/mode ratio, and

\mathrm{cover}_{0.5}

. The

5

D models failed to learn (coverage

0.00

–

0.60

), and their saddle estimates rest on

4

–

30

evaluation points each (Table 16 ); treat the

5

D block as estimator diagnostics, not dimension evidence. Source: results/expA_learn/{2,3,4,5}d_pack_*_s0.json .

configuration	$n_{\mathrm{mode}}$	$n_{\mathrm{saddle}}$	$n_{\mathrm{valley}}$	cover
2D $K{=}5$ , $\sigma{=}0.25$	170	3087	6743	1.00
2D $K{=}5$ , $\sigma{=}2.0$	3338	186	6476	1.00
4D $K{=}5$ , $d{=}3$	307	25	9668	1.00
4D $K{=}20$ , $d{=}5$	79	209	9712	0.85
5D $K{=}5$ , $d{=}3$	93	5	9902	0.60
5D $K{=}10$ , $d{=}3$	35	4	9961	0.30

Appendix

Table 16: Evaluation-point counts per region (of

10{,}000

uniform points) for the extreme configurations. Mode and saddle strips empty out with dimension and

K

; 5D saddle MSEs are means over

\leq 30

points from models that did not learn. Source: score_mse.n_{mode,saddle,valley} fields.

configuration	$t$	mode	saddle	valley	MC	sad/mode
2D $K{=}5$ , $\sigma{=}0.25$	0.02	0.418	251.82	145.74	0.452	603
	0.05	0.070	123.72	127.77	0.076	1773
	0.10	0.048	31.25	40.36	0.045	648
5D $K{=}20$ , $d{=}5$ (cover 0.15)	0.02	0.114	0.314	1.670	0.147	2.8
	0.05	0.031	0.129	1.716	0.051	4.2
	0.10	0.018	0.081	1.461	0.025	4.6

Appendix

Table 17: Regional score MSE at all three diffusion times for the two extreme configurations. The saddle/mode gap persists across

t

; density-weighted MC MSE remains an order of magnitude below the saddle MSE even at

600\times

mode error.

$K$	$\sigma$	$\mathrm{cov}_{q}$	res. $q/r$	$e_{\rm ref}^{\leq.05}$	$e_{q}^{\leq.05}$	$\times_{\leq.05}$	$\times_{(.05,.2)}$	grid/path
5	0.25	0.20	1.23	10.15	149.1	14.7	84.0	24.8
10	0.25	0.40	1.07	32.15	53.8	1.7	5.0	4.5
20	0.25	0.25	1.00	46.52	67.9	1.5	3.7	4.9
5	0.50	0.40	1.10	3.94	6.1	1.5	9.8	4.0
10	0.50	0.20	1.08	5.04	5.8	1.1	3.5	1.7
20	0.50	0.35	1.01	8.74	10.4	1.2	1.5	1.6

Appendix

Table 18: Controlled-trajectory test, 2D width sweep. Columns:

\beta{=}0.3\beta_{c}

endpoint coverage; saddle-residence ratio

q/r

; reference-score saddle error along reference paths and collapsing paths, time-averaged over

t\leq 0.05

; their ratio at

t\leq 0.05

and

t\in(0.05,0.2)

; and grid

t{=}0.02

saddle MSE divided by the matched-window path error. Reference coverage is

1.00

in every row. Source: results/expP_paths/ .

Figure 13: Controlled PF-ODE trajectories from the shared $x_{T}$ batch (200 RK2 steps, 1024 saved trajectories; script plot_expP_paths.py ). Dotted grey circles are mode cells ( $1.5\sigma$ ), dashed orange circles saddle strips ( $0.25\,\hat{d}$ ), black crosses mode centres, and the green star the RL target basin. (a) $K{=}5,\sigma{=}0.25$ , reference: paths settle at all five modes (grey: log-density of all states; blue: 48 example trajectories). (b) Same $x_{T}$ , $\beta=0.3\beta_{c}$ : endpoints funnel onto the target basin and the centre mode and the three other modes are abandoned (cover $0.20$ ); corridor crossings at intermediate times are visibly similar to (a). (c) $\sigma=2$ , collapsing policy: the $1.5\sigma$ mode cells (radius $3$ ) swallow the saddle midpoints, which is why saddle residence is $\sim 0.02$ . (d) Saddle residence vs. diffusion time (trajectory flows left $\to$ right): the $\sigma=0.25$ reference leaves the strips after $t\approx 0.1$ while its collapsing policy lingers at $0.1$ – $0.2$ late — the single outlier configuration; for $\sigma=2$ both curves stay near zero. (e) Reference-score saddle MSE encountered along the same paths: only the $\sigma=0.25$ collapsing run (upper red dashed) separates strongly from its reference at $t\lesssim 0.2$ (the $84\times$ window); the two $\sigma=2$ curves coincide. (f) Endpoint nearest-mode fractions for $K{=}5,\sigma{=}0.25$ : the reference spreads uniformly ( $0.18$ – $0.24$ per mode) while the collapsed policy funnels $74\%$ of endpoints onto the target basin $(-5,0)$ and $\approx 17\%$ onto the centre, leaving the other modes nearly empty. Collapse is thus reward-target funneling, not saddle-rerouting.

Figure 14: Measured collapse boundary versus PF-ODE step count

N

in all eight geometries (Experiment Q). Orange: exact tilted GMM with analytic scores (solver-only budget); blue: fully learned pipeline; green: learned policy with exact-score KL anchor; dashed: grid-numeric

\beta_{c}^{\rm var}

. Every curve is exactly flat from

N{=}25

400

(zero discretisation contribution), the solver-only boundary coincides with

\beta_{c}^{\rm var}

wherever modes are separated, and neither learned pipeline reaches it.

Figure 15: Left: boundary recovered under each error budget at

N{=}400

versus geometry — analytic scores (orange) recover

\beta_{c}^{\rm var}

(dashed) except in overlap; the exact-anchor intervention (green) tracks the fully learned pipeline (blue), not the analytic one. Right: the path-integrated KL-anchor gap and reference score MSE at

t{=}.02

grow with separation, i.e. anti -correlated with the amplification.

Figure 16: Raw coverage curves (

N{=}400

) behind the boundary estimates: blue learned pipeline, orange analytic scores, green exact anchor; dotted line at the

0.5

coverage threshold. In the overlapping packs (bottom row) all pipelines start with non-trivial coverage at

\beta\!\to\!0

, the saturated-coverage regime of Open Problem 3.

cell	$w_{\rm kl}$	$d_{\min}/(2\sigma_{\rm lat})$	$\beta_{c}^{\rm var}$	$\beta_{c}^{\rm emp}$	emp/var
w0.03	0.03	2.08	3.21	1.37	0.43
w0.1	0.10	1.55	1.99	0.50	0.25
w0.3	0.30	1.14	1.17	0.85	0.73
w1.0	1.00	1.12	1.20	0.67	0.56
w3.0	3.00	0.65	0.77	0.96	1.24
w1 h64	1.00	1.16	1.32	0.74	0.56

Appendix

Table 19: Experiment R: all seven cells use the SAME Gaussian prior. The KL-weight intervention moves latent geometry monotonically from

2.08

0.65

and the variational boundary from

3.21

0.77

Figure 17: Experiment R mediation test. Filled circles: measured

\beta_{c}

under a fixed Gaussian prior with geometry moved by KL weight; crosses: each cell’s variational boundary; the dotted line joins them; stars: the Exp6 Gaussian and GMM endpoints. The Exp6 Gaussian star sits on the

w_{\rm kl}{=}1

intervention cell.

Figure 18: Within-training ELBO trajectories under the fixed Gaussian prior (Experiment R): geometry ratio (left) and latent blob width (centre) saturate at KL-weight-dependent levels; right shows the reconstruction and weighted-KL loss terms.

prior	$\sigma_{\rm lat}$	$d_{\min}/(2\sigma_{\rm lat})$	placement	$\beta_{c}^{\rm var}$	$\beta_{c}^{\rm emp}$	emp/var	$\alpha_{\rm emp}$	plateau
gaussian	$0.36{\pm}.01$	$1.19{\pm}.05$	—	$1.16{\pm}.10$	$0.71{\pm}.02$	$0.61{\pm}.08$	$2.7{\pm}.2$	1.00
gmm	$0.039{\pm}.001$	$5.2{\pm}1.6$	$0.01d$	$1.47{\pm}.23$	$1.27{\pm}.19$	$0.87{\pm}.10$	$430{\pm}70$	1.00
vamp	$0.34{\pm}.01$	$1.73{\pm}.23$	$0.27d$	$5.9{\pm}1.6$	$4.0{\pm}1.7$	$0.67{\pm}.18$	$18{\pm}9$	1.00
gmvae	$0.75{\pm}.01$	$1.13{\pm}.04$	$0.67d$	$4.5{\pm}.6$	$4.5{\pm}1.4$	$0.97{\pm}.21$	$4.0{\pm}1.2$	$0.95$ – $1.00$

Appendix

Table 20: Experiment V, mean

{\pm}

std over three seeds. Placement is the mean greedy component-to-mode distance in units of

d

(fraction matched is 10/10 for every learned mixture).

\alpha_{\rm emp}=\beta_{c}^{\rm emp}/(2\sigma_{\rm lat}^{2})

; plateau is decoded coverage at the two largest

\beta

values.

Figure 19: Experiment V on the mediation plane (stars: 3-seed mean

{\pm}

std; hollow circles: the Exp R fixed-Gaussian KL-weight arm; large stars: Exp6 endpoints). Vamp/GMVAE cells occupy their own geometries rather than the GMM corner.

Figure 20: Experiment V collapse curves normalised by each cell’s own

\beta_{c}^{\rm var}

(mean

{\pm}

std band, three seeds). All four prior families cross coverage

0.5

within a factor

0.5

–

1.2

of their geometry-predicted boundary.

Figure 21: Experiment V decoded plateau coverage. No prior family loses data-space modes at large

\beta

	$g$	1	2	4	10
$K{=}3$ ( $j{=}1$ )	$\alpha^{\rm var}$	sat	4.1	19.8	128
	$\alpha^{\rm emp}$	0.11	0.46	6.5	14.5
$K{=}5$ ( $j{=}2$ )	$\alpha^{\rm var}$	1.9	18.2	78.3	494
	$\alpha^{\rm emp}$	$0.91{\pm}.21$	3.5	$18.7{\pm}6.1$	55.9
$K{=}10$ ( $j{=}5$ )	$\alpha^{\rm var}$	10.5	67.7	283	1772
	$\alpha^{\rm emp}$	4.5	17.0	71.1	445

Appendix

Table 21: Experiment W:

\alpha^{\rm var}

and

\alpha^{\rm emp}

at four separations (“sat”: ball metric saturated at

\beta=0

\beta_{c}^{\rm var}=0

censored). Extra-seed values for

K{=}5

are exact at

g=2

(3.51 all seeds) and spread at

g=1

(

0.62,1.06,1.06

) and

g=4

(

15.1,15.1,25.7

Figure 22: Experiment W:

\alpha=\beta_{c}/\beta_{\rm cross}

against

g=d_{\min}/(2\sigma_{0})

. Blue squares: exact variational optimum; red circles: learned pipeline (faint diamonds: extra

K=5

seeds); hollow triangles: ball-saturated / below-grid censors; black dashed: dominant-order saddle envelope; dotted:

\alpha=1

and

d_{\min}/\sigma_{0}=4

. Right panel: Exp Q 2D/3D packs against the

K=5

envelope.

Figure 23: Experiment X: non-target-basin retention versus target enrichment (decoded), three basins, mean over three seeds (

{\pm}

std band for soft curves). Solid: soft retention; dashed: hard retention. The dotted line marks

3\times

Figure 24: Experiment X, C5 rare basin. Left: hard retention at

3\times

enrichment (hollow marker: seed censored below

3\times

, plotted at its

2.91\times

maximum, where retention is

1.0

). Right: soft retention AUC across the sweep for all three basins.

arm	$\alpha$	$\sigma_{\rm floor}$	$d_{\min}/(2\sigma_{\rm lat})$	$\sigma_{\rm lat}$	matched Gaussian cell
width	1.0	0.35	2.15	0.48	— (above image edge $2.08$ )
width	1.0	0.50	2.02	0.67	$w_{\rm kl}{=}0.03$ ( $g{=}2.08$ )
width	1.0	0.70	1.93	0.93	$w_{\rm kl}{=}0.03$
width	1.0	1.00	1.88	1.32	$w_{\rm kl}{=}0.03$
width	1.0	1.30	1.79	1.69	$w_{\rm kl}{=}0.03$
contract	0.9	0.50	1.49	0.40	$w_{\rm kl}{=}0.1$ ( $g{=}1.55$ )

Appendix

Table 22: Experiment Y intervention calibration (seed 0, full VAE budget). The width floor moves achieved geometry through the upper part of the Gaussian-ELBO image; adding centre contraction reaches the middle image geometries. Centre separation grows as components widen, so the width-only arm saturates near

g\approx 1.8

Figure 25: Experiment Y reverse mediation. Left: measured

\beta_{c}

versus achieved geometry

g

for the Gaussian KL-weight arm (circles) and the GMM width-floor (squares) and centre-contraction (diamonds) arms; crosses mark each group’s own-geometry

\beta_{c}^{\rm var}

; dotted lines join nearest-geometry cross-prior pairs. Right: the same data as emp/var ratios. The inset reports the prior-label coefficient of

\log\beta_{c}^{\rm emp}\sim\log g+\mathbb{1}_{\rm GMM}

(a direct prior effect would move the coefficient away from

1\times

Latent Space Geometry

See all 29 papers

Sep 26, 2026

Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancing

Post-hoc Bayesian fine-tuning places Gaussians around trained low-rank adapters, yet a calibrated posterior does not by itself yield a useful generalization certificate. Such a posterior admits an informative PAC-Bayes certificate only when both the loss of its sampled predictors and its KL divergence from an admissible prior are small. In this research, we characterize this certification frontier for Gaussian LoRA posteriors and separate three interventions: changing the prior, changing the stochastic predictor, and changing only how its complexity is counted. First, an exact isotropic KL envelope eliminates the prior scale and yields a width threshold that excludes posterior widths before sampling, while a zero-KL floor identifies targets that no complexity reduction can reach at a measured risk bound. Second, we minimize KL in closed form over the full

GL(r)

symmetry of the low-rank factors, leaving every sampled adapter product unchanged, and derive the noncentral objective required when the prior center is trained on an independent split. On a small-pool RoBERTa audit of 567 configurations, the recorded 64-draw summaries imply a certificate floor of

0.7298

even with zero KL and Chernoff accounting, so reducing complexity alone cannot certify these posteriors at the recorded budgets. For a stable posterior in a controlled Gaussian-factor task, Chernoff accounting certifies risk below

0.1

on 20 of 20 datasets with 1024 draws, whereas Hoeffding certifies none. On deliberately deformed synthetic rank-four factors, matrix balancing reduces KL by

29.3\%

on average beyond scalar balancing without changing any prediction. Numerical split-prior scenarios make the remaining risk and complexity budgets explicit.

Joyanta Jyoti Mondal, Ibne Farabi Shihab

Department of Computer and Information Sciences, University of Delaware, USA · Department of Computer Science, Iowa State University, USA

Bayesian LoRA PAC-Bayesian Generalization Bounds Low-Rank Adaptation

Ask PDF

Sep 22, 2026

The Gaussian Is Enough: Flow-Matching Priors Do Not Help When Fine-Tuning Large Behavior Models

Modern robot imitation learning increasingly relies on generative policies based on diffusion or flow-matching models, which generate actions by transforming samples from a prior distribution. A key question is whether the choice of prior matters. Replacing the standard Gaussian with a closer-to-target, non-Gaussian prior has been shown to substantially improve performance when training from scratch. A natural next step is to ask whether these gains transfer to fine-tuning pretrained Large Behavior Models (LBMs) such as LBM 1.0,

π_{0.5}

, and GR00T~N1.5, where one might expect even larger gains. Surprisingly, we find that this is not the case, except possibly at very low fine-tuning data fractions. Across over 100K simulation rollouts spanning all three aforementioned LBMs on 40+ tasks in two simulation platforms, and 1250 hardware rollouts on five bimanual manipulation tasks, non-Gaussian priors that are demonstrably closer to the target yield statistically indistinguishable or worse fine-tuning performance than a standard Gaussian prior. Diagnostic analyses suggest why: fine-tuned imitation learning policies converge to similar action predictions across priors, despite their fine-tuned encoder embeddings diverging substantially from the pretrained embeddings and each other. A learning-rate ablation further confirms that encoder training is the dominant factor in fine-tuning performance, substantially outweighing the effect of prior choice. We conclude with concrete directions for future research on when and why learned priors might still matter in fine-tuning. Project page: https://cxu-tri.github.io/non_gaussian_FT/

Chen Xu, Rishi Shah, Hadas Kress-Gazit +2

Toyota Research Institute · Woven by Toyota · Cornell University

Robot Imitation Learning Diffusion Model Fine-Tuning Robot Foundation Models Flow Matching

Ask PDF

May 4, 2026

Binary Rewards and Reinforcement Learning: Fundamental Challenges

Reinforcement learning with verifiable rewards (RLVR) has become a standard approach for improving reasoning in language models, yet models trained with RLVR often suffer from diversity collapse: while single-sample accuracy improves, multi-sample coverage degrades, sometimes falling below the base model. We provide a structural account of this phenomenon grounded in the properties of binary rewards. Binary rewards create a fundamental degeneracy for policy gradient methods: the set of distributions maximizing expected reward is infinite, with no distinguished element. KL-control resolves this degeneracy by selecting, in the limit

β\to 0

, the filtered model

p_*:=a(\cdot\mid\mathcal{Y}_1)

-- the base model conditioned on validity -- which is the unique fully valid distribution closest to the base model in KL divergence. This selection operates through a nontrivial asymmetry: the tilted distribution

p_{[β]}\propto a(y)\,e^{v(y)/β}

converges to

p_*

in forward KL as

β\to 0

, yet

p_*

cannot serve as a direct optimization target because

\mathrm{KL}(q\,\|\,p_*)

is infinite for any full-support policy

q

. We develop explicit formulas relating the hyperparameter

β

to the more interpretable target validity rate

μ

. Under model misspecification -- the typical practical regime -- the pressure to decrease

β

drives the optimizer toward highly concentrated distributions over a small number of valid outputs, collapsing toward ever fewer as

β

decreases, rather than toward the filtered model. We illustrate this mechanism on a toy autoregressive experiment and discuss how alternative divergences that target

p_*

directly -- as pursued empirically by \citet{kruszewski_whatever_2026} -- avoid this failure mode by rewarding coverage of

p_*

's support rather than concentration on high-validity outputs.

Marc Dymetman

∗Scientific Consultant

Reinforcement Learning KL-Regularized RL RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Reward Design for RL Behavioral Diversity in RL

Ask PDF

Abstract

Latent Space Geometry

Explore similar work

Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancing

The Gaussian Is Enough: Flow-Matching Priors Do Not Help When Fine-Tuning Large Behavior Models

Binary Rewards and Reinforcement Learning: Fundamental Challenges