Surprisal-Rényi Free Energy

Shion Matsumoto; Raul Castillo; Benjamin Prada; Ankur Arjun Mali

stat.MLMar 3, 2026

Surprisal-Rényi Free Energy

Authors: Shion Matsumoto, Raul Castillo, Benjamin Prada, Ankur Arjun Mali

Organizations: Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, Florida, USA

Abstract

The forward and reverse Kullback-Leibler (KL) divergences arise as limiting objectives in learning and inference yet induce markedly different inductive biases that cannot be explained at the level of expectations alone. In this work, we introduce the Surprisal-Rényi Free Energy (SRFE), a log-moment-based functional of the likelihood ratio that lies outside the class of $f$ -divergences. We show that SRFE recovers forward and reverse KL divergences as singular endpoint limits and derive local expansions around both limits in which the variance of the log-likelihood ratio appears as a first-order correction. This reveals an explicit mean-variance tradeoff governing departures from KL-dominated regimes. We further establish a Gibbs-type variational characterization of SRFE as the unique minimizer of a weighted sum of KL divergences and prove that SRFE directly controls large deviations of excess code-length via Chernoff-type bounds, yielding a precise Minimum Description Length interpretation. Together, these results identify SRFE as a variance- and tail-sensitive free-energy functional that clarifies the geometric and large-deviation structure underlying forward and reverse KL limits, without unifying or subsuming distinct learning frameworks.

Figures & tables

Figure 1: Gaussian $Q$ that minimizes forward KL (blue) and reverse KL (red) with a mixture of Gaussians $P$ with means $\mu_{1},\mu_{2}$ where $\mu_{2}\geq\mu_{1}$ and variance $\sigma_{1}^{2}=\sigma_{2}^{2}=1$ . Gaussians are equally weighted.

Panel of Figure 2 — Figure 2: SRFE updates are driven by the score $\nabla_{\theta}\log q_{\theta}$ evaluated under the escort $r_{\tau}$ (which downweights regions where $q_{\theta}$ is small when $\tau\in(0,1)$ ), whereas CR updates are driven by samples from $q_{\theta}$ with an explicit likelihood-ratio weight $u^{\tau}$ that can amplify variance when $q_{\theta}\ll p$ .

Figure 3: Experiment 1 - $\tau\in\{0.3,0.5,0.7,0.9\}$ tends to spread out, matching Forward KL performance; $\tau\in\{0.1\}$ and Reverse KL hone in on fewer modes.

Figure 4: Experiment 2 - $\tau$ -sweep for $\tau$ with optimal mode coverage, entropy error, effective sample size, and test log-likelihood (in red).

Figure 5: Experiment 3 - Fixed $\tau=0.99$ schedule is unstable when compared to the other schedules

Figure 6: Experiment 4 - Entropy error, effective sample size, and test log-likelihood worsen as outlier contamination grows. Mode coverage improves.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: SRFE as a KL-regularized variational projection onto the escort (Chernoff) distribution $r_{\tau}$ .

Target Distribution
Component	Configuration
Type	3-component Gaussian mixture
Means	$\mu_{1}=(-3,0)$ , $\mu_{2}=(3,0)$ , $\mu_{3}=(0,4)$
Covariance	$\Sigma=0.5\cdot I_{2}$ (shared)
Weights	$w=[0.3,0.3,0.4]$
Variational Family

Appendix

Table 1: Experimental Configuration

Method	Mode Coverage (out of 3) $\uparrow$	Entropy Error $\downarrow$	Test LogLik $\uparrow$	ESS $\uparrow$
Forward KL $(\tau\rightarrow 1)$	3	1.2232	-4.4727	2141.2888
SRFE $(\tau=0.9)$	3	1.2984	-4.5134	2045.3491
SRFE $(\tau=0.7)$	3	1.1252	-4.4556	2130.7683
SRFE $(\tau=0.5)$	3	1.1209	-4.4815	2094.5110
SRFE $(\tau=0.3)$	3	1.0225	-4.5039	1853.8958
SRFE $(\tau=0.1)$	1	1.0986	-17.5261	3091.4211

Appendix

Table 2: Experiment 1 – Performance comparison of divergence measures on multimodal VI task

Method	Avg. Mode Coverage $\uparrow$	Avg. Entropy Error $\downarrow$	Avg. ESS $\uparrow$	Avg. Test LogLik $\uparrow$
SRFE $(\tau=0.1)$	1	1.0929	7685.0161	-17.1385
SRFE $(\tau=0.2)$	1	1.0738	6173.0793	-16.7109
SRFE $(\tau=0.3)$	3	0.9969	1817.4306	-4.5164
SRFE $(\tau=0.4)$	3	1.0979	2062.4072	-4.4815
SRFE $(\tau=0.5)$	3	1.0871	2069.2320	-4.4624
SRFE $(\tau=0.6)$	3	1.1537	2119.8005	-4.4656

Appendix

Table 3: Experiment 2 – Performance comparison of SRFE at different

\tau

values (avg. of metrics)

Method	Entropy Error Std.	ESS Std.	Test LogLik Std.
SRFE $(\tau=0.1)$	0.0067	3254.3556	0.1807
SRFE $(\tau=0.2)$	0.0132	3868.7299	0.2623
SRFE $(\tau=0.3)$	0.0375	133.3366	0.0315
SRFE $(\tau=0.4)$	0.0229	45.0580	0.0134
SRFE $(\tau=0.5)$	0.0094	11.2182	0.0041
SRFE $(\tau=0.6)$	0.0111	51.8966	0.0108

Appendix

Table 4: Experiment 2 – Performance comparison of SRFE at different

\tau

values (std. of metrics)

Method	Entropy Error $\downarrow$	Mode Coverage $\uparrow$	ESS $\uparrow$	Test LogLik $\uparrow$	Final Loss
Fixed $\tau=0.5$	1.1721	3	2146.8315	-4.4581	1.7322
Fixed $\tau=0.99$ (Forward KL)	1.3121	3	2060.4182	-4.5003	0.0000
Fixed $\tau=0.01$ (Reverse KL)	0.4594	2	8.6457	-6.7723	2.6208
Annealing ( $0.3\rightarrow 0.9$ )	1.1561	3	2064.5593	-4.4414	1.0876
Reverse Annealing ( $0.9\rightarrow 0.3$ )	1.0088	3	1893.3969	-4.5421	2.0154
Stepwise ( $0.3\rightarrow 0.5\rightarrow 0.7\rightarrow 0.9$ )	1.2391	3	2093.3459	-4.4936	1.5042

Appendix

Table 5: Experiment 3 – Performance comparison of fixed and annealing scheduling strategies

Method	Outlier Weight	Entropy Error $\downarrow$	Mode Coverage $\uparrow$	ESS $\uparrow$	Test LogLik $\uparrow$
SRFE $(\tau=0.01)$	0	0.4511	2	42.1605	-7.1755
SRFE $(\tau=0.5)$	0	1.1326	3	2143.2493	-4.4626
SRFE $(\tau=0.99)$	0	1.4338	3	1351.3932	-4.6495
SRFE $(\tau=0.01)$	0.1	20.6654	3	4.8961	-23.0920
SRFE $(\tau=0.5)$	0.1	21.0623	3	34.1576	-23.4870
SRFE $(\tau=0.99)$	0.1	6.6339	3	24.1573	-9.2847

Appendix

Table 6: Experiment 4 – Performance comparison of SRFE at different

\tau

values and outlier contamination.

Large Deviation Principles

See all 8 papers

Exponential families from a single KL identity
Apr 30, 2026Marc DymetmanInformation Geometry Exponential Family
Scientific Consultant
Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning
Sep 14, 2026Emmy Blumenthal, Nikolas Claussen, Benjamin Eysenbach +4
Joseph Henry Laboratories of Physics, Princeton University, Princeton, New Jersey 08544, USA · Princeton Center for Theoretical Science, Princeton University, Princeton, New Jersey 08544, USA · Department of Computer Science, Princeton University, Princeton, New Jersey 08544, USA
Universal Bounds for Self-Normalized Kullback-Leibler Estimation
Oct 9, 2026Buu Phan, Lester Mackey, David Alvarez-MelisKullback-Leibler Divergence Importance Sampling
University of Toronto · Microsoft Research · Harvard University

Apr 30, 2026J/K move · Enter open · S save

Exponential families from a single KL identity

Marc Dymetman

Scientific Consultant

Information Geometry Exponential Family Kullback-Leibler Divergence

Open paper AskPDF arXiv

Exponential families encompass the distributions central to modern machine learning -- softmax, Gaussians, and Boltzmann distributions -- and underlie the theory of variational inference, entropy-regularized reinforcement learning, and RLHF. We isolate a simple identity for exponential families that expresses the KL difference

\mathrm{KL}(q \| p_{λ_2}) - \mathrm{KL}(q \| p_{λ_1})

in terms of the log-partition function

A(λ)

and the moment

μ_q

. Remarkably, this identity together with the single fact that

\mathrm{KL} \geq 0

(with equality iff

p = q

) suffices, by direct substitution and rearrangement, to derive a cluster of results that are classically obtained by separate, heavier arguments: a generalized three-point identity for arbitrary reference distributions, Pythagorean theorems for I-projections and reverse I-projections, convexity of the log-partition function, identification of its Legendre dual in KL terms, the Gibbs variational principle, and the explicit optimizer in KL-regularized reward maximization, including the exponential tilting formula underlying entropy-regularized control and RLHF. Beyond these purely algebraic consequences, standard analytic arguments recover the gradient formula for the log-partition function, the Bregman representation of within-family KL divergence, and the surjectivity of the moment map. The note is self-contained.

Abstract

Large Deviation Principles

Explore similar work

Exponential families from a single KL identity

Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning

Universal Bounds for Self-Normalized Kullback-Leibler Estimation