Lifted inference algorithms enable scalable probabilistic inference even for large object domains by leveraging the indistinguishability of objects in a probability distribution. An essential prerequisite for constructing a lifted representation is to identify commutative factors, i.e., functions whose output values are invariant under permutations of a subset of their input values, in a potential-based factorisation. In practice, however, parameters learned from data inevitably deviate even if associated objects are indistinguishable, causing their corresponding factors to be only approximately commutative instead of being exactly commutative. We address this problem by introducing the concept of ε-commutativity, a relaxation of commutativity where output values are only approximately invariant under permutations of input values. Specifically, we show how ε-commutativity can be exploited for lifted model construction, downstream probabilistic inference, and prove strict bounds on the induced approximation error, thereby ensuring the practical applicability of lifted model construction while maintaining highly accurate query results. These theoretical guarantees are confirmed empirically, demonstrating comparable query accuracy at lower runtime.
Figures & tables
Figure 1: (a) An fg modelling the interplay between the competences of two employees and the revenue of the company they work for. (b) The function definition of ϕ3 , where φ1,…,φ6∈R≥0 with φi=φj for all i=j .
Table 1: (a) An exemplary factor ϕ(ComA,ComB,Rev) with ε -commutative arguments {ComA,ComB} provided that φ3 and φ3′ as well as φ4 and φ4′ are ε -equivalent. (b) A compressed representation of ϕ(ComA,ComB,Rev) from (a) using a crv to represent the ε -commutative arguments {ComA,ComB} .
Figure 2: Downstream lifted inference performance, averaged over all generated instances and queries. (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ). (b) Distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p .
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=1ε -commutative factors with ε=0.001 .
Figure 4: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=1ε -commutative factors with ε=0.01 .
Figure 5: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=1ε -commutative factors with ε=0.1 .
Figure 6: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=3ε -commutative factors with ε=0.001 .
Figure 7: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=3ε -commutative factors with ε=0.01 .
Figure 8: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=3ε -commutative factors with ε=0.1 .
Figure 9: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=7ε -commutative factors with ε=0.001 .
Figure 10: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=7ε -commutative factors with ε=0.01 .
Figure 11: (a) Run time of lve on the compressed model returned by acp and its ε -relaxed variant ( acp ±ε ), and (b) distribution of the per-query quotient p′/p between the approximate result p′ and the exact result p for input fg containing k=7ε -commutative factors with ε=0.1 .
Figure 12: (a) Speedup for online inference run times (i.e., ratio of the run time of lve on the compressed model returned by acp and the run time of lve on the compressed model returned by the ε -relaxed variant of acp ). (b) Offline run times of acp and its ε -relaxed variant ( acp ±ε ).
Figure 13: Compression ratio ∣M′∣/∣M∣ between the compressed model M′ and the original model M for input fg containing k=1ε -commutative factors with (a) ε=0.001 and (b) ε=0.1 .
Figure 14: Compression ratio ∣M′∣/∣M∣ between the compressed model M′ and the original model M for input fg containing k=3ε -commutative factors with (a) ε=0.001 and (b) ε=0.1 .
Figure 15: Compression ratio ∣M′∣/∣M∣ between the compressed model M′ and the original model M for input fg containing k=7ε -commutative factors with (a) ε=0.001 and (b) ε=0.1 .
Figure 16: Compression ratio ∣M′∣/∣M∣ between the compressed model M′ and the original model M broken down per instance class, averaged over the number of commutative factors k and the tolerance ε .
Exploiting the indistinguishability of objects in a probabilistic graphical model such as a factor graph is key to lifted probabilistic inference algorithms and allows for tractable probabilistic inference problems with respect to domain sizes. A central building block for the exploitation of indistinguishable objects in factor graphs is the identification of commutative factors, i.e., factors whose output values are invariant under permutations of input values assigned to a subset of their arguments. In this paper, we revisit the theoretical foundations underlying the state-of-the-art algorithm to detect commutative factors. Specifically, we show that in its current form, the state-of-the-art algorithm relies on a central theorem that is mistakenly regarded as a sufficient condition to identify commutative factors, while it actually only implies necessary condition. Consequently, the state of the art might, as we show in this paper, deliver incorrect results. To fix the flaws currently present in the state of the art, we prove a slightly modified version of the aforementioned theorem, which serves as a necessary condition to identify commutative factors. Moreover, we present a corrected version of the state-of-the-art algorithm, which keeps its efficiency while ensuring correctness and introduce a complementary algorithm with tighter worst-case bounds.
Malte Luttermann, Ralf Möller, Marcel Gehrke
Institute for Humanities-Centered Artificial Intelligence, University of Hamburg, Germany
Lifted inference exploits indistinguishabilities in probabilistic graphical models by using a representative for indistinguishable objects, thereby speeding up query answering while maintaining exact answers. In this article, we show how lifting can be applied to efficiently compute causal effects in relational domains. More specifically, we introduce parametric causal factor graphs (PCFGs) to incorporate causal knowledge in lifted models and give a formal semantics of interventions therein. We further present the Lifted Causal Inference (LCI) algorithm to compute causal effects on a lifted level, thereby drastically speeding up causal inference compared to propositional inference, e.g., in causal Bayesian networks. In addition, we present partially directed parametric causal factor graphs (PD-PCFGs) as a generalisation of PCFGs to handle partial causal knowledge and extend LCI to perform lifted causal inference in a PD-PCFG, thereby extending the applicability of lifted causal inference to a broader range of models requiring less prior knowledge about causal relationships.
Malte Luttermann, Tanya Braun, Ralf Möller +1
Institute for Humanities-Centered Artificial Intelligence, University of Hamburg, Hamburg, Germany. · German Research Center for Artificial Intelligence (DFKI), Lübeck, Germany. · Data Science Group, University of Münster, Münster, Germany.
Reconciling the tension between inductive learning and deductive reasoning in first-order relational domains is a longstanding challenge in AI. We study the problem of answering queries in a first-order relational probabilistic logic through a joint effort of learning and reasoning, without ever constructing an explicit model. Traditional lifted inference assumes access to a complete model and exploits symmetry to evaluate probabilistic queries; however, learning such models from partial, noisy observations is intractable in general. We reconcile these two challenges through implicit learning to reason and first-order relational probabilistic inference techniques. More specifically, we merge incomplete first-order axioms with independently sampled, partially observed examples into a bounded-degree fragment of the sum-of-squares (SOS) hierarchy in polynomial time. Our algorithm performs two lifts simultaneously: (i) grounding-lift, where renaming-equivalent ground moments share one variable, collapsing the domain of individuals; and (ii) world-lift, where all pseudo-models (partial world assignments) are enforced in parallel, producing a global bound that holds across all worlds consistent with the learned constraints. These innovations yield the first polynomial-time framework that implicitly learns a first-order probabilistic logic and performs lifted inference over both individuals and worlds.
Luise Ge, Brendan Juba, Kris Nilsson +1
Computer Science & Engineering, Washington University in St. Louis, St. Louis, Missouri, USA