A Unified Framework for Training-Dataset Aggregation and Exact Machine Unlearning via Gibbs Probability Measures
Authors: Yaiza Bermudez, Samir M. Perlaza, Iñaki Esnaola
Organizations: Centre Inria d’Université Côte d’Azur, INRIA, 06902 Sophia Antipolis, France · College of Computing, Université Mohamed VI Polytechnique (UM6P), Ben Guerir, 43150, Morocco · Department of Electrical and Computer Engineering at Princeton University, Princeton, NJ 08544 USA · GAATI Mathematics Laboratory at the University of French Polynesia, 98702 Faaa, French Polynesia · School of Electrical and Electronic Engineering, University of Sheffield, Sheffield S1 3JD, U.K.
In this paper, three operations on Gibbs probability measures are studied: renormalization, normalized log-linear combination, and nesting. The resulting measures are shown to be Gibbs probability measures with explicit expressions for their objective functions, reference measures, and regularization factors. These measures are further characterized as unique solutions to optimization problems involving linear combinations of the original objective functions, subject to either relative entropy regularization or relative entropy constraints. When the reference measures are probability measures, geometric interpretations of these three operations are established involving Pythagorean identities and exponential geodesics.A central result is the equivalence between nesting and normalized log-linear combinations of Gibbs probability measures. These results are then applied to supervised machine learning with Gibbs algorithms, which are represented by Gibbs probability measures. In this context, either through nesting or log-linear combinations, Gibbs algorithms trained on local datasets are used to form new Gibbs algorithms trained upon the aggregation of all local training datasets. Interestingly, the parameters of such operations, determine the weights of each training data point on the aggregated dataset. %Operations on local Gibbs probability measures are shown to control the contributions of training data. This observation is leveraged for achieving: (a) exact machine unlearning in centralized, decentralized, and federated learning systems; and (b) centralized-equivalent performance, in the sense of producing algorithms trained on the aggregation of all local datasets in decentralized and federated learning .
Figures & tables
Fig. 1: Geometric representation of the (α,Q,Q1) -renormalization of PY∣X=x(h,Q,λ) in ( 6 ). The original Gibbs probability measure PY∣X=x(h,Q,λ) lies on the boundary of GQ,γ , whereas the renormalized probability measure PY∣X=x(h,Q1,αλ) in ( 17 ) lies on the boundary of GQ1,η1 . The reals γ and η1 are defined in ( 63 ) and ( 64 ), respectively. The purple arrow represents the mapping from PY∣X=x(h,Q,λ) to PY∣X=x(h,Q1,αλ) , which is induced by the renormalization operation.
Fig. 2: Geometric representation of the (α,Q,Q) -renormalization of PY∣X=x(h,Q,λ) in ( 6 ). The solid purple curve represents the exponential geodesic generated by the measure PY∣X=x(h,Q,αλ) in ( 17 ) as α varies. The panels show (a) γ<η1 , with GQ,γ⊆GQ,η1 , and (b) η1<γ , with GQ,η1⊆GQ,γ . The sets GQ,γ and GQ,η1 are of the form in ( 62 ), with γ and η1 in ( 63 ) and ( 64 ), respectively.
Fig. 3: Geometric representation of the (α,Q,Q1) -renormalization of PY∣X=x(h,Q,λ) in ( 6 ), with Q1=Q . The solid purple curve represents the exponential geodesic generated by the measure PY∣X=x(h,Q1,αλ) in ( 17 ) when α varies. The panels show (a) η1<η2 , with GQ1,η1⊆GQ1,η2 , and (b) η1>η2 , with GQ1,η2⊆GQ1,η1 . The sets GQ,γ , GQ1,η1 , and GQ1,η2 are of the form in ( 62 ), with γ , η1 , and η2 defined in ( 63 ), ( 64 ), and ( 68 ), respectively.
Fig. 4: Geometric representation of the normalized log-linear combination PY∣X=x(hβ,Q,1) in ( 39 ), with Q1=Q2=Q and β=(λ1α,λ21−α) , where 0<α<1 . The solid purple curve represents the exponential segment obtained by varying α over [0,1] . Its endpoints are PY∣X=x2(h2,Q,λ2) at α=0 and PY∣X=x1(h1,Q,λ1) at α=1 , and it passes through the normalized log-linear combination at the chosen value of α . The neighborhoods GQ,γ1 , GQ,γ2 , and GQ,γ0 are defined by ( 62 ), with γ1 and γ2 given in ( 61 ) and γ0 given in ( 69 ). Under the illustrated ordering γ1<γ2 , the panels show (a) γ0<γ1<γ2 , (b) γ1<γ0<γ2 , and (c) γ1<γ2<γ0 .
Fig. 5: Geometric representation of the normalized log-linear combination PY∣X=x(hβ,Q,1) in ( 39 ), with β=(α/λ1,(1−α)/λ2) and 0<α<1 . The solid purple curve represents the exponential geodesic obtained by varying α over [0,1] . Its endpoints are PY∣X=x2(h2,Q,λ2) at α=0 and PY∣X=x1(h1,Q,λ1) at α=1 . The original Gibbs probability measures PY∣X=x1(h1,Q1,λ1) and PY∣X=x2(h2,Q2,λ2) are represented separately on the boundaries of GQ1,γ1 and GQ2,γ2 , respectively, and do not, in general, belong to the exponential geodesic. The sets GQ,η1 , GQ,η2 , and GQ,γ0 contain the two endpoints and the normalized log-linear combination, respectively, with η1 , η2 , and γ0 defined in ( 72 ), ( 73 ), and ( 69 ).
Fig. 6: Geometric representation of nested Gibbs probability measures illustrating (a) GQ1,γ1⊆GQ1,η and (b) GQ1,η⊆GQ1,γ1 . The dotted neighborhood GQ2,γ2 has reference measure Q2 in both cases.
Fig. 7: Pythagorean interpretation of Lemma 14 .
Fig. 8: Geometric representation of recursively nested Gibbs probability measures illustrating (a) GQ1,γ1⊆GQ1,η and (b) GQ1,η⊆GQ1,γ1 .
Fig. 9: Pythagorean interpretation of Lemma 15 .
Fig. 10: Geometric representation of exact unlearning under the assumption in ( 172 ), with γ1<η . The neighborhoods GQ,γ1 , GQ,η , and GPΘ∣Z=z0(Ln0,Q,λ0),γ2 are defined as in ( 62 ), with γ1 , γ2 , and η in ( 175 ), ( 176 ), and ( 177 ), respectively. The Gibbs probability measure in ( 167 ) and the nested Gibbs probability measure in ( 170 ) coincide and satisfy ( 178 ).
Fig. 11: Nested structure induced by the forward–backward peer-to-peer communication protocol: for all k∈{1,2,…,K−1} , the Gibbs probability measure produced by client k becomes the reference measure used by client k+1 .
Fig. 12: Geometric representation of recursively nested Gibbs probability measures for K=4 , under the assumption in ( 190 ). The neighborhoods GQk,γk , with k∈{1,2,…,4} , and GQ1,η are defined as in ( 62 ), with γk and η in ( 194 ) and ( 196 ), respectively. The points Qk+1 represent the Gibbs probability measures in ( 198 ). The final point Q5 in ( 199 ) coincides with the Gibbs probability measure in ( 192 ) obtained by training upon the aggregated dataset z0 .
Fig. 13: One-shot client–server communication system. For all k∈{1,2,…,K} , the Gibbs algorithm of client k , PΘ∣Z=zk(Lnk,Qk,λk) , is defined in ( 211 ), and the probability measure representing the aggregation at the server, SK , is defined in ( 212 ).
Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.
Sep 24, 2026·Yaiza Bermudez, Samir M. Perlaza, Iñaki EsnaolaMachine Unlearning
Centre Inria d’Universit´e Cˆote d’Azur, INRIA, Sophia Antipolis, France. · Laboratoire GAATI, Universit´e de la Polyn´esie franc¸aise, Fa‘a‘¯a, French Polynesia. · ECE Dept. Princeton University, Princeton, 08544 NJ, USA. +1
Centre Inria d'Université Côte d'Azur, INRIA, Sophia Antipolis, France. · Laboratoire GAATI, Université de la Polynésie française, Fa'a'ā, French Polynesia. · ECE Dept. Princeton University, Princeton, 08544 NJ, USA. +1
Department of Mathematics University of California Davis Davis, CA 95616-5270, USA · Department of Mathematics Center of Data Science and Artificial Intelligence Research University of California Davis Davis, CA 95616-5270, USA