Functional Bregman Divergences

Follow this topic Ask about this topic

Functional Bregman divergences measure the gap between a convex functional at two objects and its first-order approximation, extending familiar vector divergences to functions, distributions, and model outputs. Approaches include Hilbert-space formulations with Riesz representers and kernel mean embeddings, as well as connections to Legendre and Fenchel–Young duality. The geometry supports applications such as kernel-based divergence estimation, online calibration with proper losses, decision-tree impurity design, alignment objectives, and optimization preconditioning. A central challenge is balancing useful asymmetry and task-specific geometry against tractable, statistically reliable estimation and performance that transfers across settings.

Latest papers

10 papers3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

The Bayesian Learning Rule Beyond KL Geometry
Oct 22, 2026Sophia Sklaviadis, Wu Lin, Andre Martins +3Bayesian Inference Variational Inference
Instituto de Telecomunicações and Instituto Superior Técnico, Universidade de Lisboa, Portugal · University of Central Florida, USA · RIKEN Center for Advanced Intelligence Project, Japan +1
How Bregman Divergences Shape Shampoo
Oct 6, 2026Bing Liu, Wenjie Zhou, Chengcheng Zhao +4Deep Learning Optimization Functional Bregman Divergences
Zhejiang University · University of the Chinese Academy of Sciences · Peking University +2
Bregman Consensus
Sep 28, 2026Andrei N. SoklakovFunctional Bregman Divergences Multi-Agent Consensus
Role-Aware Neural Convex Divergence Heads for Asymmetric Representation Learning
Jul 2, 2026He Huang, Lu Shen, Yunfeng Huang +1Representation Learning Functional Bregman Divergences
School of Mathematics and Statistics, Chongqing Technology and Business University, Chongqing, 400067, China · School of Food Science, Chongqing Technology and Business University, Chongqing,2026 400067, China · Faculty of Electrical EngineeringUniversity, andDortmund,InformationGermanyTechnology, TU Dortmund +1
A Bregman Perspective on Classification and Regression Trees
Jun 12, 2026Mathias BourelFunctional Bregman Divergences Decision Tree Learning
IESTA, Facultad de Ciencias Económicas y de Administración, Universidad de la República, Uruguay · IRL-2030, Instituto Franco-Uruguayo de Matemática e Interacciones (IFUMI)
Calibeating for general proper losses: A Bregman divergence approach
May 17, 2026Maximilian Fichtl, Cristóbal Guzmán, Nishant A. MehtaFunctional Bregman Divergences Proper Scoring Rules
Independent researcher · Institute for Mathematical and Computational Engineering, Faculty of Mathematics and School of Engineering, Pontificia Universidad Católica de Chile · Department of Computer Science, University of Victoria
BSO: Safety Alignment Is Density Ratio Matching
May 12, 2026Tien-Phat Nguyen, Truong Nguyen, Thin Nguyen +3Functional Bregman Divergences Direct Preference Optimization
Hanoi University of Science and Technology Hanoi, Vietnam · Deakin University Burwood, VIC 3125, Australia · Max Planck Research School for Intelligent Systems +2
Generalized Distributional Alignment Games for Unbiased Answer-Level Fine-Tuning
May 4, 2026Mehryar Mohri, Jon Schneider, Yutao ZhongFine-Tuning LLM Alignment
Google Research & CIMS · New York, NY 10011 · Google Research
Generalising maximum mean discrepancy: kernelised functional Bregman divergences
Apr 27, 2026Russell Tsuchida, Frank NielsenKernel Mean Embeddings Functional Bregman Divergences
Department of Data Science and AI, Monash University, Australia. · Sony Computer Science Laboratories, Inc. Tokyo, Japan.
Quadratic polarity and polar Fenchel-Young divergences from the canonical Legendre polarity
Mar 5, 2026Frank Nielsen, Basile Plus-Gourdon, Mahito SugiyamaInformation Geometry Functional Bregman Divergences
Sony Computer Science Laboratories Inc. Tokyo, Japan · École Normale Supérieure Paris-Saclay, France · National Institute for Informatics Tokyo, Japan

Oct 22, 2026J/K move · Enter open · S save

The Bayesian Learning Rule Beyond KL Geometry

Sophia Sklaviadis, Wu Lin, Andre Martins, Mario Figueiredo+2

Instituto de Telecomunicações and Instituto Superior Técnico, Universidade de Lisboa, Portugal · University of Central Florida, USA · RIKEN Center for Advanced Intelligence Project, Japan+1

Bayesian Inference Variational Inference Information Geometry Functional Bregman Divergences

Open paper AskPDF arXiv

The Bayesian learning rule (BLR) exploits the relation between Bayesian updating and the Kullback-Leibler divergence (KLD) to derive efficient additive updates for exponential family variational posteriors. The KL geometry is powerful but restrictive: combined with posterior families with parameter-dependent support it can lead to undefined objectives. We ask which non-KL geometries preserve the additive BLR computation. Previous works that use

f

-divergences suggest that this is only possible with the KLD. Here, we show that this is not true. We show that additive information-geometric optimization algorithms can be derived for the class of mixed-space Bregman or Fenchel-Young discrepancies. These discrepancies induce generalized maximum-entropy families, dual coordinates, and generalized-BLR canonical gradient updates, recovering the classical BLR. The generalized BLR leads to new algorithms that cannot be derived from the KL-based BLR. For the Tsallis negentropy with quadratic statistics, the generalized BLR yields bounded-support and heavy-tailed

q

-Gaussian posteriors and an easy Newton-like implementation. Experiments up to CIFAR-100 illustrate controllable support and tail behavior and competitive performance.