Probabilistic Graphical Models

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 22

Sep 22, 2026stat.ML

A Practical Guide on Graphical Model Validation

This manuscript formalizes the most popular model validation tools used in general insurance actuarial modeling. These include graphical tools like calibration plots, actual-vs-expected plots, lift charts, Murphy diagrams, as well as classical statistical tools such as Bregman losses, deviance losses, elementary losses, Murphy's decomposition and Gini scores. Particular emphasis is placed on whether calibration and discrimination are studied under a policy-weighted or an exposure-weighted population measure. This distinction is crucial in ensuring that premium schemes are calibrated on the correct scale.

Mario V. Wüthrich

Department of Mathematics, ETH Zurich

Actuarial Ratemaking Probabilistic Graphical Models Validation Bregman Divergences

Sep 11, 2026cs.LG

Local Robustness Quantification for Naive Bayes Classifiers and Generative Forests: a General Approach

We provide methods for calculating the robustness of the predictions of two types of generative classifiers whose underlying distribution is a Probabilistic Graphical Model (PGM): naive Bayes classifiers and generative forests (a probabilistic extension of random forests). Following the paradigm of robustness quantification, we define the robustness of a prediction as the extent to which the distribution of the classifier can be perturbed without changing this prediction. We consider perturbations obtained by varying the local models of the PGMs within general neighborhoods and focus in particular on epsilon-contamination, total variation distance and chi-squared divergence balls. We test our methods on benchmark datasets, demonstrate that the robustness value of a prediction serves as an indicator for its trustworthiness and compare our approach with other such indicators.

Adrián Detavernier, Jasper De Bock

Foundations Lab for imprecise probabilities Ghent University Ghent, Belgium

Robustness Verification Random Forest Probabilistic Graphical Models Kullback-Leibler Divergence Generative Models

Sep 7, 2026stat.ME

Bayesian Matrix-Valued Graphs for Context-Dependent Multivariate Relationships

Many scientific graphs attach several variables to each node, so a single scalar edge weight cannot describe direction-dependent interactions. We model each edge by a symmetric positive-definite (SPD) matrix and infer a posterior over matrix-valued graph geometries, which we call the Bayesian matrix-valued graph (BMVG). We ask how these interactions reconfigure across contexts: how large the change is and which multivariate directions strengthen or weaken. The geodesic distance induced by the affine-invariant Riemannian metric (AIRM) quantifies deformation magnitude and generalized eigenvalues resolve its signed directions.Against fused graphical lasso, Bayesian multiple-GGM, and common principal components, BMVG is competitive on global precision recovery while retaining identifiable matrix-valued edge structure and accurately recovering edge-level deformation directions. In controlled known-truth experiments, it resolves structural change with increasing sample size, including orientation changes that leave ordinary eigenvalues unchanged. In one year of Bay Area weather data, the geometry of 12-hour change reconfigures spatial coupling about as much as whole seasons differ. In TCGA-BRCA, estrogen-receptor (ER)-associated reconfiguration concentrates on specific gene-module pairs and persists under graph-scaffold sparsification and removal of subgroup mean differences. These results establish posterior matrix-valued edge geometry as a unified framework for quantifying and interpreting context-dependent multivariate reconfiguration.

Papri Dey

Probabilistic Graphical Models Bayesian Principal Component Analysis Deformable Cnn

Jul 30, 2026stat.ML

Accelerated Random-Sweep Gibbs Sampling for Gaussian Graphical Models via Dual Normal Factor Graphs

We study the convergence properties of the random-sweep Gibbs sampler for Gaussian graphical models with a thin-membrane prior. We demonstrate that the convergence rate of the Gibbs sampler is significantly accelerated in the dual model, which is obtained by applying the Fourier transform to the local factors of the normal factor graph representing the original model. In both domains, we derive the exact convergence rates for homogeneous

k

-regular graphs. We prove that, for all homogeneous models whose graphical representations contain cycles, the convergence rate in the dual domain is universal and independent of the underlying graph topology. Moreover, we show that the effective convergence rate in the dual domain is governed by the algebraic connectivity of the graph, providing an additional acceleration without increasing the computational complexity per sweep. We further establish an explicit algebraic relation between the covariance structures of the primal and dual models, enabling marginal statistics of the primal model to be recovered directly from those of the dual model. Finally, numerical experiments on several graph families confirm our theoretical results and demonstrate substantial improvements in the convergence rates in various settings.

Borna Khodabandeh, Mehdi Molkaraie

Department of Statistics, University of Oxford, Oxford OX1 3LB, UK · Department of Signal Theory and Communications, UPC, 08034 Barcelona, Spain · Department of Statistical Sciences, University of Toronto, ON M5G 1X6, Canada

Probabilistic Graphical Models Gibbs Factor Graphs Convergence Acceleration

Jul 26, 2026stat.ME

A Characterization of the Orthocomplement of the Tangent Space of Semiparametric Markov Models

Graphical models are ubiquitous in social and empirical science as they are intuitive and easy to use. These models belong to the broader class of Markov models, defined using solely conditional independence (CI) restrictions. In order to estimate finite-dimensional target parameters in such models efficiently, semi-parametric theory provides a principled framework for constructing regular and asymptotically linear estimators via influence functions (IFs). These estimators are asymptotically normal and root-

n

consistent. Characterizing the class of all influence functions for a target parameter is crucial for statistically efficient inference in these models. For models that are Markov relative to directed acyclic graphs (DAGs), the orthogonal complement of the tangent space is known, implying that for any target the class of all influence functions can be derived once an influence function is obtained. On the other hand, for Markov models not equivalent to a DAG model -- such as ordinary Markov models associated with undirected graphs, chain graphs, or acyclic directed mixed graphs -- the orthogonal complement has not been characterized, impeding semi-parametric inference in these models. We derive closed form expressions for the orthogonal complement of the tangent space for general Markov models and illustrate our results by characterizing the class of influence functions for the conditional mean parameter in several graphical models.

Trung Phung, Ilya Shpitser

Computer Science Department, Johns Hopkins University, Baltimore, Maryland, USA

Probabilistic Graphical Models Influence Function Tangent Space Conditional Independence Markov Complementarity

Jul 24, 2026stat.ML

Graph-Based Correlation Matrix Generation: A Convex Optimization Approach

This work addresses the generation of theoretical correlation matrices with prescribed sparsity patterns associated to graph structures. We propose a novel convex optimization framework in which an initial matrix is projected onto an elliptope under a positive semidefiniteness constraint. Several numerical schemes are implemented and compared. The problem falls within the broader class of matrix completion, where off-diagonal entries corresponding to absent edges are fixed to zero and diagonal entries are fixed to one. Beyond this structural constraint, the approach offers greater flexibility than existing methods by allowing control over the mean of the off-diagonal entry distribution, enabling the generation of correlation matrices that better reflect realistic data. This procedure is not designed to yield a uniform distribution over the feasible set; rather, it provides a principled and tunable way to construct correlation matrices suitable for benchmarking statistical methods for graphical model inference. Theoretical guarantees on the existence of solutions are established, both in the general setting and under the additional mean constraint. Simulation studies illustrate the properties of the generated matrices with respect to graph structure. The methodology is applied to two real-world datasets from neuroscience and finance, and a comparison with GAN-based correlation matrix generation is provided.

Ali Fakhar, K{é}vin Polisano, Ir{è}ne Gannaz +1

UGA · Univ. Grenoble Alpes, CNRS, Grenoble INP, Inria, LJK, F-38000 Grenoble, France · G-SCOP\_GROG, G-SCOP, Grenoble INP, UGA +2

Full-Rank Correlation Matrices Matrix Probabilistic Graphical Models Convex Optimization Spurious Correlations Sparsity

Jul 21, 2026cs.LG

Parallel Noising in Neural Markov Logic Networks

Neural Markov Logic Networks (NMLNs) are a flexible neurosymbolic relational model. Previous work has shown that, although NMLNs achieve strong performance as generative models for small relational structures, they underperform diffusion-based generative graph models on larger structures. In this paper, we strengthen NMLNs along two main dimensions: (i) we increase the expressive capacity of their potential functions using graph neural networks, and (ii) we develop a new training and inference algorithm inspired by parallel-tempering Markov chain Monte Carlo methods, which we name parallel noising. Together, these enhancements enable NMLNs to attain strong performance in graph generation relative to general diffusion-based generative graph models. Furthermore, they allow NMLNs to match the performance of specialized text-based recurrent models when generating small molecular structures.

Peter Jung, Giuseppe Marra, Ondrej Kuzelka

Czech Technical University, Prague, Czech Republic · Department of Computer Science, KU Leuven, Leuven, Belgium

Graph Neural Networks Probabilistic Graphical Models Markov Neuro-Symbolic Framework

Jul 20, 2026stat.ML

Mixing-Free and Signal-Optimal Learning of Gaussian Graphical Models from Glauber Dynamics

Gaussian graphical model selection is usually studied under independent sampling, but in many applications the data arise as a single trajectory of a dependent stochastic process. We study exact recovery of the graph from one trajectory of random-scan Gaussian Glauber dynamics. Existing techniques for this problem either inherit the mixing time of the chain, which can be super-polynomial in the dimension

p

without strong assumptions, or are suboptimal in the minimum normalized edge strength

κ

. We propose two algorithms that are mixing-free and attain the

κ^{-2}

dependence of the information-theoretic lower bounds. Both instantiate a shared dueling-neighborhood search meta-algorithm with a local statistic built directly from the update sequence. For every fixed precision matrix and deterministic initialization, the first algorithm fits a least-squares regression at the updates of each node and has pointwise recovery horizon

\widetilde O(pd^{2}/κ^{2})

, where

d

is the maximum degree. Its horizon depends logarithmically on a local conditioning quantity and on the initialization potential. The second algorithm is based on counting occurences of a specific update pattern and requires

\widetilde O(pd^{4}/κ^{2})

updates, with no dependence on any condition number. The central technical challenge is that both statistics are built from dependent, non-stationary observations. Our analysis tackles this by demonstrating how to extract fresh Gaussian innovations from the update sequence, which yields mixing-free control of appropriate quantities. Neither the algorithms nor their analyses invoke stationarity, a spectral gap, or mixing conditions.

Vignesh Tirukkonda, Gautam Dasarathy

Arizona State University

Probabilistic Graphical Models Optimal Learner Stochastic Processes Efficient Sequence Mixers

Jul 9, 2026cs.LG

Learning $\mathsf{AC}^0$ under Locally Sampleable Graphical Models

The problem of learning constant-depth circuits holds profound implications for computational learning theory. In a seminal result, by introducing the low-degree algorithm, Linial, Mansour, and Nisan (J. ACM 1993) presented a quasipolynomial-time learner for

\mathsf{AC}^0

under the uniform distribution. However, obtaining comparable learning guarantees for broader classes of correlated distributions has remained a longstanding challenge. Recently, Chandrasekaran, Gaitonde, Moitra, and Vasilyan (arXiv 2026) extended these guarantees to Gibbs distributions on bounded-degree graphical models with both strong spatial mixing and polynomial growth. In this paper, we give a quasipolynomial-time learner for

\mathsf{AC}^0

under graphical models that admit efficient local samplers, circumventing the polynomial-growth requirement in prior work. The key ingredient is a new low-degree approximation for Gibbs distributions, established by simulating and suitably truncating the classical Glauber dynamics. As applications, this framework yields learners for two-spin systems, including the hard-core model and Ising model, on arbitrary bounded-degree graphs, in regimes approaching their respective sampling thresholds.

Weiming Feng, Xiongxin Yang, Yixiao Yu +1

School of Computing and Data Science, The University of Hong Kong. · Department of Computer Science, University of California, Santa Barbara. · State Key Laboratory for Novel Software Technology, New Cornerstone Science Laboratory, Nanjing University.

Learnability Quantum Learning Probabilistic Graphical Models

Jul 1, 2026stat.ML

Characterizing and Identifying Separable Graphical Models

We study a broad class of graphical models whose independencies correspond to vertex separation in mixed graphs with directed, undirected, and bidirected edges, that are capable of encoding independence structures arising from feedback, latent and selection mechanisms. In particular, we introduce separable graphs, in which each missing edge implies the existence of a separating set for its endpoints, and essentially separable graphs, those graphs separation equivalent to a separable graph. We show that these models include many existing graph families used to define graphical models an provide several characterizations of separable graphs and essentially separable graphs. We also provide multiple characterizations of separation equivalence for separable graphs. One is a graphical characterization in terms of ordinary graph properties, extending earlier results for specific subfamilies Another is a separational characterization depending only on graph separation properties. Finally, we provide a canonical representation for the equivalence classes of essentially separable graphs and develop an algorithm that, under suitable assumptions, identifies the equivalence class of any essentially separable graph.

Christopher Meek, Kayvan Sadeghi

Department of Statistics, University of Washington · Department of Statistical Science, University College London

Probabilistic Graphical Models Graph Representations

Jun 30, 2026cs.LG

Learning Gaussian Graphical Models from a Glauber Trajectory Without Mixing

We study the task of learning the structure of a

d

-sparse Gaussian graphical model on

n

variables from a single trajectory of Glauber dynamics. Beyond algorithmic considerations, many applications present temporally correlated observations rather than i.i.d.\ samples. In the classical i.i.d.\ setting, under comparably general sparsity and minimum edge-strength assumptions, sublinear-in-

n

sample guarantees are known, but achieving them in polynomial-time remains open. Motivated in part by this gap, we give a polynomial-time algorithm that recovers the conditional-independence graph from a single Glauber trajectory, with a trajectory-length guarantee that does not depend on the mixing time. Technically, our algorithm has three components. First, we estimate the conditional variances and rescale the trajectory to reduce to the unit-diagonal case, without changing the underlying graph. Second, we design a local edge test that extracts adjacency information from short update windows by isolating pairwise influence. Third, we aggregate these local statistics using a robust median-based estimator, and prove accuracy despite temporal dependence arising from a single trajectory.

Eric Shen, Tony Wu, Mahbod Majid +1

MIT

Probabilistic Graphical Models Conditional Independence Temporal Dependence Optimal Sample Complexity

Jun 26, 2026cs.AI

Lifted Causal Inference

Lifted inference exploits indistinguishabilities in probabilistic graphical models by using a representative for indistinguishable objects, thereby speeding up query answering while maintaining exact answers. In this article, we show how lifting can be applied to efficiently compute causal effects in relational domains. More specifically, we introduce parametric causal factor graphs (PCFGs) to incorporate causal knowledge in lifted models and give a formal semantics of interventions therein. We further present the Lifted Causal Inference (LCI) algorithm to compute causal effects on a lifted level, thereby drastically speeding up causal inference compared to propositional inference, e.g., in causal Bayesian networks. In addition, we present partially directed parametric causal factor graphs (PD-PCFGs) as a generalisation of PCFGs to handle partial causal knowledge and extend LCI to perform lifted causal inference in a PD-PCFG, thereby extending the applicability of lifted causal inference to a broader range of models requiring less prior knowledge about causal relationships.

Malte Luttermann, Tanya Braun, Ralf Möller +1

Institute for Humanities-Centered Artificial Intelligence, University of Hamburg, Hamburg, Germany. · German Research Center for Artificial Intelligence (DFKI), Lübeck, Germany. · Data Science Group, University of Münster, Münster, Germany.

Causal Graph Causal Inferences Probabilistic Inference Probabilistic Graphical Models Deduction

Jun 14, 2026cs.SE

Graphical-Probabilistic Modeling of Generative Flows in LLM-Native Software Systems

Engineering LLM-native software remains a challenging and immature field. Current practice is largely exploratory, relying on experimentation and heuristic techniques such as prompting and context engineering. These, however, are low-level and lack the principled structure needed to support design-level reasoning or analysis. In contrast, traditional software engineering leverages modularity and abstraction to communicate and analyze system behavior. To bring similar rigor to LLM-native development, we propose methods for documenting generative flows and for stating properties of LLM-based software designs. Such methods must account for the stochastic, prompt-dependent behavior of large language models while remaining expressive enough to capture emergent phenomena. Our initial approach is based on graphical probabilistic models, tailored to capture phenomena characteristic of LLM-native systems. This framework -- what we term Generation Networks -- aims to provide a foundation for principled reasoning about generative interactions and system-level properties in LLM-centric software architectures.

Víctor A. Braberman, Flavia Bonomo-Braberman

Departamento de Computación, FCEN, Universidad de Buenos Aires / ICC, UBA-CONICET Buenos Aires, Argentina

Large Language Model Generation Generative Flow Networks Software Probabilistic Graphical Models

Jun 3, 2026cs.LG

In-Context Graphical Inference

Marginal inference in discrete graphical models forces a choice between exactness and scalability: exact algorithms are intractable for high-treewidth graphs, while iterative approximations (Belief Propagation, variational methods) sacrifice convergence guarantees on frustrated topologies. We argue that this dichotomy stems from a mismatched inductive bias: iterative methods abandon the sequential elimination structure that makes exact inference correct. We introduce In-Context Graphical Inference (ICG-I), an autoregressive Graph Transformer that restores this structure by mimicking Variable Elimination with learned, Tensor- Train-compressed intermediate factors, paired with a Dirichlet output layer and Weighted Conformal Prediction for calibrated, distribution-free coverage guarantees under topological shift. We prove that TT compression errors propagate at most lincarly through the autoregressive chain, that the Dirichlet-Multinomial loss is a proper scoring rule, and that WCP maintains coverage with a quantifiable degradation under estimated density ratios. We conducted intensive experiments to evaluate ICG-I and achieved state-of-the-art performance across all benchmarks. ICG-I reduces MAE from 0.041 (best baseline) to 0.020 on standard instances and achieves 0.048 on N=500 frustrated spin glasses where BP diverges entirely.

Zehua Cheng, Wei Dai, Jiahao Sun

Department of Computer Science, University of Oxford, Oxford, United Kingdom · FLock.io, London, United Kingdom

Probabilistic Graphical Models Probabilistic Inference In-Context Learning In-Context Spatial Photonic Ising Machines Autoregressive Transformers+1

May 28, 2026cs.LG

Destruction is a General Strategy to Learn Generation; Diffusion's Strength is to Take it Seriously; Exploration is the Future

I present diffusion models as part of a family of machine learning techniques that withhold information from a model's input and train it to guess the withheld information. I argue that diffusion's destroying approach to withholding is more flexible than typical hand-crafted information withholding techniques, providing a rich training playground that could be advantageous in some settings, notably data-scarce ones. I then address subtle issues that may arise when porting reinforcement learning techniques to the diffusion context, and wonder how such exploration problems could be addressed in more diffusion-native ways. I do not have definitive answers, but I do point my fingers in directions I deem interesting. A tutorial follows this thesis, expanding on the destroy-then-generate perspective. A novel kind of probabilistic graphical models is introduced to facilitate the tutorial's exposition.

Pierre-André Noël

ServiceNow AI Research

Diffusion Models Probabilistic Graphical Models Exploration Perspective

May 26, 2026cs.AI

On the Detection of Commutative Factors in Factor Graphs: Necessary and Sufficient Conditions

Exploiting the indistinguishability of objects in a probabilistic graphical model such as a factor graph is key to lifted probabilistic inference algorithms and allows for tractable probabilistic inference problems with respect to domain sizes. A central building block for the exploitation of indistinguishable objects in factor graphs is the identification of commutative factors, i.e., factors whose output values are invariant under permutations of input values assigned to a subset of their arguments. In this paper, we revisit the theoretical foundations underlying the state-of-the-art algorithm to detect commutative factors. Specifically, we show that in its current form, the state-of-the-art algorithm relies on a central theorem that is mistakenly regarded as a sufficient condition to identify commutative factors, while it actually only implies necessary condition. Consequently, the state of the art might, as we show in this paper, deliver incorrect results. To fix the flaws currently present in the state of the art, we prove a slightly modified version of the aforementioned theorem, which serves as a necessary condition to identify commutative factors. Moreover, we present a corrected version of the state-of-the-art algorithm, which keeps its efficiency while ensuring correctness and introduce a complementary algorithm with tighter worst-case bounds.

Malte Luttermann, Ralf Möller, Marcel Gehrke

Institute for Humanities-Centered Artificial Intelligence, University of Hamburg, Germany

Factor Graphs Probabilistic Graphical Models Factor Probabilistic Inference Indistinguishability Sufficiency

May 22, 2026cs.LG

Private Adaptive Covariance Estimation via Gaussian Graphical Models

We propose PACE-GGM, a data-adaptive differentially private method for covariance estimation that concentrates its privacy budget on the most informative entries of the empirical covariance matrix, rather than perturbing all entries. This applies in the natural setting where the modeler supplies separate bounds for each variable, so that individual entries can be measured with less noise than the full matrix. In each round, our method selects a poorly approximated entry, measures it using the Gaussian mechanism, and then reconstructs a full covariance matrix using a maximum-entropy reconstruction objective, leading to a Gaussian graphical model structure. Experiments on diverse real-world datasets demonstrate consistent improvements in estimation error with respect to the Gaussian mechanism and other baselines, particularly in high-dimensional and low-to-moderate privacy regimes.

Cecilia Ferrando, Miguel Fuentes, Brett Mullins +2

Manning College of Information and Computer Sciences University of Massachusetts Amherst

Covariance Empirical Bayes Gaussian Primitives Probabilistic Graphical Models

May 15, 2026stat.ML

Testing properties of trees in graphical models with covariance queries

We consider the problem of testing properties of graphs underlying high-dimensional graphical models. We adopt the model of covariance queries introduced by Lugosi, Truszkowski, Velona, and Zwiernik (2021). We study the case when the underlying graph is a tree. The main results of the paper show that, while reconstructing the entire tree may be costly, certain global structural properties can be tested efficiently. In particular, we design randomized tests for global structural properties that use a sub-quadratic number of queries. We develop testing procedures for several fundamental properties, including the number of leaves, the maximum degree, the typical distance, and the diameter of the tree. For each property, we obtain explicit query complexity bounds that depend on the target threshold and tolerance parameters.

Sofiya Burova, Francisco Calvillo, Gábor Lugosi +1

IMTECH and Departament de Matem`atiques, Universitat Polit`nica de Catalunya, Barcelona, Spain and Department of Economics and Business, Pompeu Fabra University, Barcelona, Spain. · LPSM, Sorbonne Universit´e, 4 Place Jussieu, 75005 Paris, France · ICREA, Pg. Llu´ıs Companys 23, 08010 Barcelona, Spain; Department of Economics and Business, Pompeu Fabra University, Barcelona, Spain; Barcelona School of Economics. +1

Probabilistic Graphical Models Trees Covariance Sample Complexity Queries

May 6, 2026stat.ML

Proximal Projection for Doubly Sparse Regularized Models

Regularization is often used in high-dimensional regression settings to generate a sparse model, which can save tremendous computing resources and identify predictors that are most strongly associated with the response. When the predictors can be represented by a Gaussian graphical model, the structure of the predictor graph can be exploited during regularization. Our proposed model exploits this underlying predictor graph structure by decomposing the estimated coefficient vector into a sum of latent variables that correspond to the sum of each node contribution to the coefficient vector. Regularization is then performed on the latent variables rather than on the coefficient vector directly. We use a penalty function that permits a clear user-defined trade-off between the L1 and L2 penalties and propose a novel proximal projection during optimization. Further, our implementation computes the projection operator for the intersection of selected groups, which conserves more computing resources compared to predictor duplication methods, especially for high-dimensional data. Through simulation, we evaluate the performance of our approach under different graph structures and node counts, and present results on real-world data. Results suggest that our method exhibits stable performance relative to other singly or doubly sparse graphical regression models.

Jia Wei He, R. Ayesha Ali, Gerarda Darlington

Department of Mathematics & Statistics , University of Guelph

Probabilistic Graphical Models Proximal Operators Regularization Latent Variable

Apr 26, 2026stat.ML

Probabilistic Graphical Model using Graph Neural Networks for Bayesian Inversion of Discrete Structural Component States

The health condition of components in civil infrastructures can be described by various discrete states according to their performance degradation. Inferring these states from measurable responses is typically an ill-posed inverse problem. Although Bayesian methods are well-suited to tackle such problems, computing the posterior probability density function (PDF) presents challenges. The likelihood function cannot be analytically formulated due to the unclear relationship between discrete states and structural responses, and the high-dimensional state parameters resulting from numerous components severely complicates the computation of the marginal likelihood function. To address these challenges, this study proposes a novel Bayesian inversion paradigm for discrete variables based on Probabilistic Graphical Models (PGMs). The Markov networks are employed as modeling tools, with model parameters learned from data and structural topology prior. It has been proved that inferring this PGM produces the same probabilistic estimation as the posterior PDF derived from Bayesian inference, which effectively solves the above challenges. The inference is accomplished by Graph Neural Networks (GNNs), and a graph property-based GNN training strategy is developed to enable accurate inference across varying graph scales, thereby significantly reducing the computational overhead in high-dimensional problems. Both synthetic and experimental data are used to validate the proposed framework

Teng Li, Stephen Wu, Yong Huang +2

Key Lab of Smart Prevention and Mitigation of Civil Engineering Disasters of the Ministry of Industry and Information Technology, Harbin Institute of Technology, Harbin 150090, China · Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, School of Civil Engineering, Harbin, Institute of Technology, Harbin 150090, China · The Institute of Statistical Mathematics, Research Organization of Information and Systems, 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan +2

Probabilistic Graphical Models Graph Neural Networks State-Tracking

Jul 16, 2025cs.LG

Learning Task Mixtures from Task Affinities: A Probabilistic Graphical Model for Supervised Fine-Tuning

Supervised fine-tuning performance for large language models depends strongly on how training budget is distributed across a heterogeneous set of tasks. In practice, mixtures are often fixed using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant sources. We introduce TaskPGM, a framework for learning continuous task mixtures via an energy-based model over tasks. Tasks form the nodes of a Markov random field: unary potentials capture per-task utility, and pairwise potentials encode inter-task relationships using behavioral divergences computed from predictive distributions of single-task fine-tuned models (e.g., Jensen--Shannon divergence and pointwise mutual information). Optimizing this objective yields mixtures that balance coverage against redundancy. We show that the resulting set function is weakly submodular under budget constraints, enabling approximation guarantees for discrete selection variants. Across multiple model families (LLaMA-7B, Qwen2-7B) and evaluation suites (BIG-Bench Hard), TaskPGM improves over standard mixing strategies and provides interpretable structure over task interactions.

Prateek Chanda, Saral Sureka, Parth Pratim Chatterjee +3

IIT Bombay · IBM Research · Red Hat AI Innovation +1

Multi--Task Learning Mixtures Kullback-Leibler Divergence Probabilistic Graphical Models

Oct 3, 2023cs.LG

Delta-AI: Local objectives for amortized inference in sparse graphical models

We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call

Δ

-amortized inference (

Δ

-AI). Our approach is based on the observation that when the sampling of variables in a PGM is seen as a sequence of actions taken by an agent, sparsity of the PGM enables local credit assignment in the agent's policy learning objective. This yields a local constraint that can be turned into a local loss in the style of generative flow networks (GFlowNets) that enables off-policy training but avoids the need to instantiate all the random variables for each parameter update, thus speeding up training considerably. The

Δ

-AI objective matches the conditional distribution of a variable given its Markov blanket in a tractable learned sampler, which has the structure of a Bayesian network, with the same conditional distribution under the target PGM. As such, the trained sampler recovers marginals and conditional distributions of interest and enables inference of partial subsets of variables. We illustrate

Δ

-AI's effectiveness for sampling from synthetic PGMs and training latent variable models with sparse factor structure.

Jean-Pierre Falet, Hae Beom Lee, Esmeralda S. Whitammer +6

Mila – Qu´ebec AI Institute, Universit´e de Montr´eal · Mila – Québec AI Institute, Université de Montréal

Probabilistic Inference Probabilistic Graphical Models Sparsity Singular Learning Theory Latent Variable Generative Flow Networks+3