stat.MLNov 3, 2026

A Unified Framework for Training-Dataset Aggregation and Exact Machine Unlearning via Gibbs Probability Measures

Authors: Yaiza Bermudez, Samir M. Perlaza, Iñaki Esnaola

Organizations: Centre Inria d’Université Côte d’Azur, INRIA, 06902 Sophia Antipolis, France · College of Computing, Université Mohamed VI Polytechnique (UM6P), Ben Guerir, 43150, Morocco · Department of Electrical and Computer Engineering at Princeton University, Princeton, NJ 08544 USA · GAATI Mathematics Laboratory at the University of French Polynesia, 98702 Faaa, French Polynesia · 

Abstract

In this paper, three operations on Gibbs probability measures are studied: renormalization, normalized log-linear combination, and nesting. The resulting measures are shown to be Gibbs probability measures with explicit expressions for their objective functions, reference measures, and regularization factors. These measures are further characterized as unique solutions to optimization problems involving linear combinations of the original objective functions, subject to either relative entropy regularization or relative entropy constraints. When the reference measures are probability measures, geometric interpretations of these three operations are established involving Pythagorean identities and exponential geodesics.A central result is the equivalence between nesting and normalized log-linear combinations of Gibbs probability measures. These results are then applied to supervised machine learning with Gibbs algorithms, which are represented by Gibbs probability measures. In this context, either through nesting or log-linear combinations, Gibbs algorithms trained on local datasets are used to form new Gibbs algorithms trained upon the aggregation of all local training datasets. Interestingly, the parameters of such operations, determine the weights of each training data point on the aggregated dataset. %Operations on local Gibbs probability measures are shown to control the contributions of training data. This observation is leveraged for achieving: (a) exact machine unlearning in centralized, decentralized, and federated learning systems; and (b) centralized-equivalent performance, in the sense of producing algorithms trained on the aggregation of all local datasets in decentralized and federated learning .

Figures & tables

Explore similar work

CardsList
  1. Machine Unlearning for Gibbs Supervised Learning Algorithms

    Sep 24, 2026Yaiza Bermudez, Samir M. Perlaza, Iñaki EsnaolaMachine Unlearning

  2. Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms

    Apr 22, 2026Yaiza Bermudez, Samir M. Perlaza, Iñaki EsnaolaDecentralized LearningStatistical Learning Theory

  3. Machine Unlearning via Information Theoretic Regularization

    Feb 8, 2025Shizhou Xu, Thomas StrohmerMachine Unlearning