Bayesian Inference

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 300% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 219

Apr 30, 2026cs.CV

Context as Prior: Bayesian-Inspired Intent Inference for Non-Speaking Agents with a Household Cat Testbed

Many agents in real-world environments cannot reliably communicate their goals through language, including household pets, pre-verbal infants, and other non-speaking embodied agents. In such settings, intent must be inferred from incomplete behavioral observations in context-rich environments. This creates a core ambiguity: observable behavior is often noisy or underspecified, while context provides strong prior information but can also induce brittle shortcut predictions if used naively. We present CatSignal, a Bayesian-inspired probabilistic framework for multimodal intent inference that models spatial context as a prior-like constraint and behavioral observations as evidence. Rather than treating context as an ordinary input feature, our method uses a context-gated Product-of-Experts formulation to compute posterior-like intent distributions from context, pose dynamics, and acoustic cues. We instantiate this formulation in a household cat setting as a focused proof-of-concept for intent inference in non-speaking agents. Under Leave-One-Video-Out evaluation on a multimodal domestic cat dataset, the proposed prior-guided fusion achieves the best overall accuracy of 77.72%, outperforming feature concatenation (71.83%) and stronger late-fusion baselines. More importantly, it substantially reduces context-driven shortcut failures in ambiguous cases. While simpler fusion strategies remain competitive in Macro-F1 and selective prediction, the proposed model provides the strongest overall accuracy and the best suppression of context-based shortcut collapse.

Wenqian Zhang, Zehao Wang

University of California, Riverside, Riverside, CA

Bayesian Inference Intent Classification Animal Behavior Analysis

Apr 30, 2026stat.ML

Bayesian X-Learner: Calibrated Posterior Inference for Heterogeneous Treatment Effects under Heavy-Tailed Outcomes

Conditional Average Treatment Effect (CATE) estimation in practice demands three properties simultaneously: heterogeneous effects

τ(x)

, calibrated uncertainty over them, and robustness to the heavy tails that contaminate real outcome data. Meta-learners (Künzel et al., 2019) give (i); causal forests and BART give (i)-(ii) with Gaussian-tail assumptions; no widely used tool gives all three. We present Bayesian X-Learner, an X-Learner built on cross-fitted doubly robust pseudo-outcomes (Kennedy, 2020) with a full MCMC posterior over

τ(x)

via a Welsch redescending pseudo-likelihood. On Hill's IHDP benchmark the default configuration attains mean

\sqrt{\varepsilon_{\mathrm{PEHE}}} = 0.56

on 5 replications (lowest mean; differences from S-/T-/X-learners, full-config Causal BART, and a causal forest baseline are not significant at

α=0.05

, and rank ordering is unstable at 10 replications -- IHDP comparisons are competitive rather than dominant). On contaminated "whale" DGPs with up to 20-25% tail density, a one-flag extension (contamination_severity) that selects a Huber-

δ

nuisance loss per Huber's minimax-

δ

relation recovers RMSE

\approx 0.13

with tight credible intervals (single-cross-fit 30-seed coverage 83% [Wilson 66%, 93%] at 20% density; modular-Bayes pooling with Bayesian-bootstrap nuisance draws restores nominal 95% coverage).

Eichi Uehara

Independent researcher

Robust Regression Causal Effect Estimation Conditional Average Treatment Effect Estimation Bayesian Inference Heavy-Tailed Distributions Individualized Treatment Effect Estimation+1

Apr 29, 2026stat.ML

Laplace Approximation for Bayesian Tensor Network Kernel Machines

Uncertainty estimation is essential for robust decision-making in the presence of ambiguous or out-of-distribution inputs. Gaussian Processes (GPs) are classical kernel-based models that offer principled uncertainty quantification and perform well on small- to medium-scale datasets. Alternatively, formulating the weight space learning problem under tensor network assumptions yields scalable tensor network kernel machines. However, these assumptions break Gaussianity, complicating standard probabilistic inference. This raises a fundamental question: how can tensor network kernel machines provide principled uncertainty estimates? We propose a novel Bayesian Tensor Network Kernel Machine (LA-TNKM) that employs a (linearized) Laplace approximation for Bayesian inference. A comprehensive set of numerical experiments shows that the proposed method consistently matches or surpasses Gaussian Processes and Bayesian Neural Networks (BNNs) across diverse UCI regression benchmarks, highlighting both its effectiveness and practical relevance.

Albert Saiapin, Kim Batselier

1Delft Center for Systems and Control, Delft University of Technology

Uncertainty Quantification Bayesian Inference Kernel Methods

Apr 28, 2026cs.LG

Teacher Forcing as Generalized Bayes: Optimization Geometry Mismatch in Switching Surrogates for Chaotic Dynamics

Identity teacher forcing (ITF) enables stable training of deterministic recurrent surrogates for chaotic dynamical systems and has been highly effective for dynamical systems reconstruction (DSR) with recurrent neural networks (RNNs), including interpretable almost-linear RNNs (AL-RNNs). However, as an intervention-based prediction loss (and thus a generalized Bayes update), teacher forcing need not match the free-running model's marginal likelihood geometry. We compare the objective-induced curvatures of ITF and marginal likelihood in a probabilistic switching augmentation of AL-RNNs, estimating ambiguity-aware observed information via Louis' identity. In the switching setting studied here, conditioning on a single forced regime path (as ITF does) inflates curvature, while marginal likelihood curvature is reduced by a missing-information correction when multiple switching explanations remain plausible. In Lorenz-63 experiments, windowed evidence fine-tuning improves held-out evidence but can degrade dynamical quantities of interest (QoIs) relative to ITF-pretrained models.

Andre Herz, Daniel Durstewitz, Georgia Koppe

Interdisciplinary Center for Scientific Computing, Heidelberg University, Germany · Dept. of Theoretical Neuroscience, Central Institute of Mental Health (CIMH), Mannheim, Germany · Hector Institute for AI in Psychiatry and Dept. of Psychiatry and Psychotherapy, CIMH, Mannheim, Germany +1

Neural Surrogate Modeling Dynamical Systems Bayesian Inference Recurrent Neural Networks Chaotic Time-Series Prediction

Apr 28, 2026stat.ML

Occam's Razor is Only as Sharp as Your ELBO

The marginal likelihood, also known as the evidence, is regarded as a mathematical embodiment of Occam's razor, enabling model selection that avoids overfitting. The evidence lower bound (ELBO) objective from variational inference has also been used for similar purposes. Prior work has shown that restricting the approximate posterior family via a mean-field approximation can lead the ELBO to underfit. In this paper, we show how ELBO-based hyperparameter learning in a simple over-parameterized regression model can also produce overfitting, depending on the assumed rank of the covariance matrix in a Gaussian approximate posterior. Surprisingly, among only the underfit and overfit options, Bayesian model selection via the evidence itself sometimes prefers the overfit version, while the ELBO does not. Bayesian practitioners hoping to scale to large models should be cautious about how reduced-rank assumptions needed for tractability may impact the potential for model selection.

Ethan Harvey, Michael C. Hughes

Tufts University

Model Selection Bayesian Inference Variational Inference Hyperparameter Optimization

Apr 28, 2026stat.AP

Adaptive Meta-Learning Stochastic Gradient Hamiltonian Monte Carlo Simulation for Bayesian Updating of Structural Dynamic Models

In the last few decades, Markov chain Monte Carlo (MCMC) methods have been widely applied to Bayesian updating of structural dynamic models in the field of structural health monitoring. Recently, several MCMC algorithms have been developed that incorporate neural networks to enhance their performance for specific Bayesian model updating problems. However, a common challenge with these approaches lies in the fact that the embedded neural networks often necessitate retraining when faced with new tasks, a process that is time-consuming and significantly undermines the competitiveness of these methods. This paper introduces a newly developed adaptive meta-learning stochastic gradient Hamiltonian Monte Carlo (AM-SGHMC) algorithm. The idea behind AM-SGHMC is to optimize the sampling strategy by training adaptive neural networks, and due to the adaptive design of the network inputs and outputs, the trained sampler can be directly applied to various Bayesian updating problems of the same type of structure without further training, thereby achieving meta-learning. Additionally, practical issues for the feasibility of the AM-SGHMC algorithm for structural dynamic model updating are addressed, and two examples involving Bayesian updating of multi-story building models with different model fidelity are used to demonstrate the effectiveness and generalization ability of the proposed method.

Xianghao Meng, James L. Beck, Yong Huang +1

Key Lab of Smart Prevention and Mitigation of Civil Engineering Disasters of the Ministry of Industry and Information Technology, Harbin Institute of Technology, Harbin, China · Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, Harbin Institute of Technology, Harbin, China · Division of Engineering and Applied Science, California Institute of Technology, CA, USA

Markov Chain Monte Carlo Meta-Learning Bayesian Inference Structural Health Monitoring

Apr 25, 2026cs.LG

Unstable Rankings in Bayesian Deep Learning Evaluation

Standard evaluations of Bayesian deep learning methods assume that metric estimates are reliable, but we show this assumption fails under data scarcity. Method rankings are not only unreliable at small

n

, but also dataset-dependent in ways that point estimates cannot reveal: the same method comparison yields

P(\mathrm{MCD} \prec \mathrm{Ensemble}) = 1.000

at

n = 50

on one dataset and remains below

0.95

even at

n = 500

on another. Across the datasets we consider, no universal sample size threshold exists, which is precisely why dataset-specific posterior inference is necessary. To address this, we use a Bayesian hierarchical model with method-specific variances to treat evaluation metrics as random variables across data realizations, and we use a predictive Minimum Detectable Difference curve to assess whether an observed gap would be detectable at a given training size. Across six Bayesian deep learning methods and five regression datasets, our results show that uncertainty-aware evaluation is necessary in low-data settings, because current evidence for method superiority and predictive detectability at the same training size can diverge substantially. Our framework provides practitioners with principled tools to determine whether their evaluation data is sufficient before drawing conclusions about method superiority.

Qishi Zhan, Minxuan Hu, Guansu Wang +2

Department of Mathematical and Statistical Sciences Marquette University, USA · Cornell University, USA · The University of Melbourne, Australia +2

Bayesian Neural Networks Statistical Power Analysis Bayesian Inference

Apr 22, 2026cs.LG

Fast Bayesian equipment condition monitoring via simulation based inference: applications to heat exchanger health

Accurate condition monitoring of industrial equipment requires inferring latent degradation parameters from indirect sensor measurements under uncertainty. While traditional Bayesian methods like Markov Chain Monte Carlo (MCMC) provide rigorous uncertainty quantification, their heavy computational bottlenecks render them impractical for real-time process control. To overcome this limitation, we propose an AI-driven framework utilizing Simulation-Based Inference (SBI) powered by amortized neural posterior estimation to diagnose complex failure modes in heat exchangers. By training neural density estimators on a simulated dataset, our approach learns a direct, likelihood-free mapping from thermal-fluid observations to the full posterior distribution of degradation parameters. We benchmark this framework against an MCMC baseline across various synthetic fouling and leakage scenarios, including challenging low-probability, sparse-event failures. The results show that SBI achieves comparable diagnostic accuracy and reliable uncertainty quantification, while accelerating inference time by a factor of82

\times

compared to traditional sampling. The amortized nature of the neural network enables near-instantaneous inference, establishing SBI as a highly scalable, real-time alternative for probabilistic fault diagnosis and digital twin realization in complex engineering systems.

Peter Collett, Alexander Johannes Stasik, Simone Casolo +1

Cognite AS, Oslo, Norway · Department of Data Science, Norwegian University of Life Sciences, Ås, Norway · Department for Mathematics and Cybernetics, SINTEF Digital, Oslo, Norway

Simulation-Based Inference Efficient Inference Amortized Inference Bayesian Inference Predictive Maintenance

Apr 22, 2026cs.LG

Tokenised Flow Matching for Hierarchical Simulation Based Inference

The cost of simulator evaluations is a key practical bottleneck for Simulation Based Inference (SBI). In hierarchical settings with shared global parameters and exchangeable site-level parameters and observations, this structure can be exploited to improve simulation efficiency. Existing hierarchical SBI approaches factorise the posterior yet still simulate across multiple sites per training sample; We instead explore likelihood factorisation (LF) to train from single-site simulations. In LF sampling we learn a per-site neural surrogate of the simulator and then assemble synthetic multi-site observations to amortise inference for the full hierarchical posterior. Building on this, we propose Tokenised Flow Matching for Posterior Estimation (TFMPE), a tokenised flow matching approach that supports function-valued observations through likelihood factorisation. To enable systematic evaluation, we introduce a benchmark for hierarchical SBI. We validate TFMPE on this benchmark and on realistic infectious disease and computational fluid dynamics models, finding well-calibrated posteriors while reducing computational cost.

Giovanni Charles, Cosmo Santoni, Seth Flaxman +1

Imperial College London, London, United Kingdom · University of Oxford, Oxford, United Kingdom

Flow Matching Simulation-Based Inference Bayesian Inference Hierarchical Bayesian Modeling

Apr 22, 2026stat.ML

On Bayesian Softmax-Gated Mixture-of-Experts Models

Mixture-of-experts models provide a flexible framework for learning complex probabilistic input-output relationships by combining multiple expert models through an input-dependent gating mechanism. These models have become increasingly prominent in modern machine learning, yet their theoretical properties in the Bayesian framework remain largely unexplored. In this paper, we study Bayesian mixture-of-experts models, focusing on the ubiquitous softmax-based gating mechanism. Specifically, we investigate the asymptotic behavior of the posterior distribution for three fundamental statistical tasks: density estimation, parameter estimation, and model selection. First, we establish posterior contraction rates for density estimation, both in the regimes with a fixed, known number of experts and with a random learnable number of experts. We then analyze parameter estimation and derive convergence guarantees based on tailored Voronoi-type losses, which account for the complex identifiability structure of mixture-of-experts models. Finally, we propose and analyze two complementary strategies for selecting the number of experts. Taken together, these results provide one of the first systematic theoretical analyses of Bayesian mixture-of-experts models with softmax gating, and yield several theory-grounded insights for practical model design.

Nicola Bariletto, Huy Nguyen, Nhat Ho +1

The University of Texas at Austin

Bayesian Mixture Models Bayesian Inference Mixture-of-Experts Models Density Estimation

Apr 21, 2026cs.LG

MoBayes: A Modular Bayesian Framework for Separating Reasoning from Language in Conversational Clinical Decision Support

Large language models (LLMs) are increasingly used for conversational clinical decision support, yet they conflate next token prediction with probabilistic decision making. We argue that this conflation reflects an architectural limitation: such systems lack explicit posterior tracking, controllable abstention thresholds, and auditable reasoning chains. We introduce MoBayes, a Modular Bayesian dialogue framework that separates reasoning from language. The LLM acts only as a language interface, parsing patient conversation into structured observations, while a Bayesian module performs probabilistic inference over these observations to update posteriors, select follow-up questions via expected-information-gain and determine when to stop or defer through calibrated decision thresholds. This design enables explicit posterior tracking, controllable selective decision-making, and replaceable population-specific statistical backends without retraining the language model. Across empirical and LLM-generated knowledge bases, MoBayes outperforms standalone frontier LLM doctors, including matched model-family comparisons where inexpensive sensor models paired with MoBayes exceed larger autonomous models at lower cost. The advantage persists under adversarial patient communication styles and across varying diagnostic scenarios. These results suggest that reliable conversational clinical decision support systems should separate probabilistic reasoning from language generation rather than scaling model size alone. Code is available at https://anonymous.4open.science/r/MoBayes/

Yusuf Kesmen, Fay Elhassan, Jiayi Ma +7

LiGHT, EPFL · University of Bern · CLAN, Aarhus University

Selective Prediction Clinical Decision Support Bayesian Inference Clinical Reasoning Medical LLMs

Apr 18, 2026cs.AI

Local Inconsistency Resolution: The Interplay between Attention and Control in Probabilistic Models

We present a generic algorithm for learning and approximate inference with an intuitive epistemic interpretation: iteratively focus on a subset of the model and resolve inconsistencies using the parameters under control. This framework, which we call Local Inconsistency Resolution (LIR) is built upon Probabilistic Dependency Graphs (PDGs), which provide a flexible representational foundation capable of capturing inconsistent beliefs. We show how LIR unifies and generalizes a wide variety of important algorithms in the literature, including the Expectation-Maximization (EM) algorithm, belief propagation, adversarial training, GANs, and GFlowNets. In the last case, LIR actually suggests a more natural loss, which we demonstrate improves GFlowNet convergence. Each method can be recovered as a specific instance of LIR by choosing a procedure to direct focus (attention and control). We implement this algorithm for discrete PDGs and study its properties on synthetically generated PDGs, comparing its behavior to the global optimization semantics of the full PDG.

Oliver E. Richardson, Mandana Samiei, Mehran Shakerinava +4

Université de Montréal · School of Computer Science, McGill University · Institut Polytechnique de Paris +1

Bayesian Inference Probabilistic Graphical Models

Apr 9, 2026astro-ph.HE

High-dimensional inference for the $γ$ -ray sky with differentiable programming

We motivate the use of differentiable probabilistic programming techniques in order to account for the large model-space inherent to astrophysical

γ

-ray analyses. Targeting the longstanding Galactic Center

γ

-ray Excess (GCE) puzzle, we construct differentiable forward model and likelihood that make liberal use of GPU acceleration and vectorization in order to simultaneously account for a continuum of possible spatial morphologies consistent with the GCE emission in a fully probabilistic manner. Our setup allows for efficient inference over the large model space using variational methods. Beyond application to

γ

-ray data, a goal of this work is to showcase how differentiable probabilistic programming can be used as a tool to enable flexible analyses of astrophysical datasets.

Siddharth Mishra-Sharma, Tracy R. Slatyer, Yitian Sun +1

Faculty of Computing and Data Sciences, Boston University, Boston, MA 02215, USA · 2The NSF AI Institute for Artificial Intelligence and Fundamental Interactions · Center for Theoretical Physics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA +2

Bayesian Inference High-Energy Physics

Mar 22, 2026stat.ML

Time-adaptive infinite-dimensional Gaussian process regression on manifolds

This paper proposes a new formulation of functional Gaussian Process regression on manifolds, based on an Empirical Bayes approach, in the spatiotemporal random field context. We apply the machinery of tight Gaussian measures in separable Hilbert spaces, exploiting the invariance property of covariance kernels under the group of isometries of the manifold. The identification via characteristic function of these measures with the infinite product of one-dimensional Gaussian measures is then obtained, in terms of the eigenfunctions of the Laplace-Beltrami operator on the manifold. The involved time-varying angular spectrum constitutes the key tool for dimension reduction in the implementation of this regression approach, adopting a suitable truncation scheme depending on the functional sample size. The simulation study and synthetic data application illustrate the performance of the proposed functional regression predictor.

MD Ruiz-Medina, AE Madrid, A Torres-Signes +1

Empirical Bayes Bayesian Inference Gaussian Processes Gaussian Process Regression

Mar 15, 2026cs.LG

Deep probabilistic model synthesis enables unified modeling of whole-brain neural activity across individual subjects

Many disciplines need quantitative models that synthesize experimental data across multiple instances of the same general system. For example, neuroscientists must combine data from the brains of many individual animals to understand the species' brain in general. However, typical machine learning models treat one system instance at a time. Here we introduce a machine learning framework, deep probabilistic model synthesis (DPMS), that leverages system properties auxiliary to the model to combine data across system instances. DPMS specifically uses variational inference to learn a conditional prior distribution and instance-specific posterior distributions over model parameters that respectively tie together the system instances and capture their unique structure. DPMS can synthesize a wide variety of model classes, such as those for regression, classification, and dimensionality reduction, and we demonstrate its ability to improve upon single-instance models on synthetic data and whole-brain neural activity data from larval zebrafish.

William E. Bishop, Luuk W. Hesselink, Bernhard Englitz +2

Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA · Computational Neuroscience Lab, Donders Center for Neuroscience, Radboud University, Nijmegen, The Netherlands · Departments of Neurobiology, Engineering Sciences and Applied Mathematics, and Physics and Astronomy, Northwestern University, Evanston, IL +1

Bayesian Inference Hierarchical Bayesian Modeling

Feb 25, 2026cs.LG

Calibrated Test-Time Guidance for Bayesian Inference

Test-time guidance is a widely used mechanism for steering pretrained diffusion models toward outcomes specified by a reward function. Existing approaches, however, focus on maximizing reward rather than sampling from the true Bayesian posterior, leading to miscalibrated inference. In this work, we show that common test-time guidance methods do not recover the correct posterior distribution and identify the structural approximations responsible for this failure. We then propose consistent alternative estimators that enable calibrated sampling from the Bayesian posterior. We significantly outperform previous methods on a set of Bayesian inference tasks, and set a new state-of-the-art PSNR in black hole image reconstruction.

Daniel Geyfman, Felix Draxler, Jan Groeneveld +3

Department of Computer Science, University of California, Irvine, CA, USA · Seoul National University, Seoul, South Korea · Chan-Zuckerberg Initiative, Redwood City, CA, USA

Bayesian Inverse Problems Bayesian Inference Posterior Sampling Diffusion Posterior Sampling Inference-Time Guidance

Feb 16, 2026stat.ML

Universal priors: solving empirical Bayes via Bayesian inference and pretraining

We theoretically justify the recent empirical finding of [Teh et al., 2025] that a transformer pretrained on synthetically generated data achieves strong performance on empirical Bayes (EB) problems. We take an indirect approach to this question: rather than analyzing the model architecture or training dynamics, we ask why a pretrained Bayes estimator, trained under a prespecified training distribution, can adapt to arbitrary test distributions. Focusing on Poisson EB problems, we identify the existence of universal priors such that training under these priors yields a near-optimal regret bound of

\widetilde{O}(\frac{1}{n})

uniformly over all test distributions. Our analysis leverages the classical phenomenon of posterior contraction in Bayesian statistics, showing that the pretrained transformer adapts to unknown test distributions precisely through posterior contraction. This perspective also explains the phenomenon of length generalization, in which the test sequence length exceeds the training length, as the model performs Bayesian inference using a generalized posterior.

Nick Cannella, Anzo Teh, Yanjun Han +1

Courant Institute of Mathematical Sciences, NYU, New York City, NY · Department of EECS, MIT, Cambridge, MA · Courant Institute of Mathematical Sciences and Center for Data Science, NYU, New York City, NY

Synthetic Data Pretraining Empirical Bayes Bayesian Inference

Jan 12, 2026stat.ML

Neural Architectures for Amortized Bayesian Inference: Statistical Foundations and Empirical Assessments

Since the turn of the century, approximate Bayesian inference has steadily evolved as new computational techniques have been incorporated to handle increasingly complex, large-scale predictive problems. The recent success of deep neural networks and foundation models has now given rise to a new paradigm in statistical modeling, in which Bayesian inference can be amortized through large-scale learned predictors. In amortized inference, substantial computation is required at the beginning to train a neural network, but it can subsequently produce approximate posteriors or predictions at much lower computational cost across a wide range of tasks. While the typical Bayesian inference procedures are computationally expensive due to repeated likelihood calculations and Monte Carlo steps for each new dataset, amortized inference provides a much lower computational cost at deployment. Despite the growing popularity of amortized inference, its statistical interpretation and position within Bayesian inference remain poorly explored. In this paper, we present a statistical perspective on several major neural architectures, including feedforward networks, Deep Sets, and Transformers, and examine how they naturally support amortized Bayesian inference. We explore how these models perform structured approximation and also probabilistic reasoning in ways that yield controlled generalization error throughout a wide range of deployment scenarios, and how these properties can be harnessed for Bayesian computation. Via simulation studies, we evaluate the accuracy, robustness, and uncertainty quantification of amortized inference across varying sample sizes, varying noise distributional families, varying sparsity levels, and multimodality, highlighting its strengths and limitations.

Roy Shivam Ram Shreshtth, Arnab Hazra, Gourab Mukherjee

Department of Statistics and Data Science, Indian Institute of Technology Kanpur, Kanpur, India 208016 · Marshall School of Business, University of Southern California, Los Angeles, United States 90089-0809

Uncertainty Quantification Amortized Inference Bayesian Inference

Dec 28, 2025stat.ML

JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference

We consider problems of parameter estimation where design variables can be actively optimized to maximize information gain. To this end, we introduce JADAI, a framework that jointly amortizes Bayesian adaptive design and inference by training a policy, a history network, and an inference network end-to-end. The networks minimize a generic loss that aggregates incremental reductions in posterior error along experimental sequences. Inference networks are instantiated with diffusion-based posterior estimators that can approximate high-dimensional and multimodal posteriors at every experimental step. Across standard adaptive design benchmarks, JADAI achieves superior or competitive performance.

Niels Bracher, Lars Kühmichel, Desi R. Ivanova +3

1Rensselaer Polytechnic Institute, USA · 2TU Dortmund University, Germany · University of Oxford, UK.

Parameter Estimation Amortized Inference Bayesian Inference Bayesian Experimental Design

Dec 5, 2025cs.CV

Ideal Observer for Segmentation of Dead Leaves Images

The visible parts of a scene are determined by occlusion among overlapping surfaces. Here we consider "dead leaves" models, which replicate this by independently sampling objects ("leaves") with position, shape, color, and texture and layering them until the image is covered. Building on prior theory, we present a self-contained framework that rigorously defines the dead leaves model and derives an analytical Bayesian ideal observer for partitioning finite pixel sets. The longest part of the paper spans the derivation of the prior probability, which elevates the observer beyond pixel-similarity methods by incorporating geometric information. These computations are practical only for small pixel sets (up to 9-10 pixels). We emphasize accessibility through step-by-step derivations, extensive visualizations, and examples. We empirically evaluate three tractable observers (prior-only, likelihood-only, and the full ideal observer), plus a random baseline on 108 dead leaves image datasets varying in texture intensity, leaf size, and image size. Likelihood-only performance falls with increasing texture intensity and image size. Prior-only performance falls with decreasing leaf size and increasing maximal image dimension. All model-based observers strongly outperform the random baseline, and the ideal observer consistently outperforms the others by combining both information sources. The model provides a principled upper bound on segmentation performance for limited pixel sets, enabling comparisons with human observers and algorithms.

Swantje Mahncke, Malte Ott, Lars C. Reining +1

Centre for Cognitive Science, Institute of Psychology, TU Darmstadt, Germany · Center for Mind, Brain and Behavior (CMBB), Universities of Marburg, Giessen, and Darmstadt, Germany · Department of Mathematics, TU Darmstadt, Germany

Image Segmentation Occlusion Reasoning Bayesian Inference

Dec 2, 2025gr-qc

Flexible Gravitational-Wave Parameter Estimation with Transformers

Gravitational-wave data analysis relies on accurate and efficient methods to extract physical information from noisy detector signals, yet the increasing rate and complexity of observations represent a growing challenge. Deep learning provides a powerful alternative to traditional inference, but existing neural models typically lack the flexibility to handle variations in data analysis settings. Such variations accommodate imperfect observations or are required for specialized tests, and could include changes in detector configurations, overall frequency ranges, or localized cuts. We introduce a flexible transformer-based architecture paired with a training strategy that enables adaptation to diverse analysis settings at inference time. Applied to parameter estimation, we demonstrate that a single flexible model, called Dingo-T1, can (i) analyze 48 gravitational-wave events from the third LIGO-Virgo-KAGRA Observing Run under a wide range of analysis configurations, (ii) enable systematic studies of how detector and frequency configurations impact inferred posteriors, and (iii) perform inspiral-merger-ringdown consistency tests probing general relativity. Dingo-T1 also improves median sample efficiency on real events from a baseline of 1.4% to 4.2%. Our approach thus demonstrates flexible and scalable inference with a principled framework for handling missing or incomplete data, key capabilities for current and next-generation observatories.

Annalena Kofler, Maximilian Dax, Stephen R. Green +6

Max Planck Institute for Intelligent Systems, Max-Planck-Ring 4, 72076 Tübingen, Germany · ELLIS Institute Tübingen, Maria-von-Linden-Straße 2, 72076 Tübingen, Germany · Tübingen AI Center, Maria-von-Linden-Straße 1, 72076 Tübingen, Germany +4

Transformer Inference Gravitational-Wave Astronomy Bayesian Inference Inverse Problems

Nov 18, 2025cs.CL

Graded strength of comparative illusions is explained by Bayesian inference

Like visual processing, language processing is susceptible to illusions in which people systematically misperceive stimuli. In one such case--the comparative illusion (CI), e.g., More students have been to Russia than I have--comprehenders tend to judge the sentence as acceptable despite its underlying nonsensical comparison. Prior research has argued that this phenomenon can be explained as Bayesian inference over a noisy channel: the posterior probability of an interpretation of a sentence is proportional to both the prior probability of that interpretation and the likelihood of corruption into the observed (CI) sentence. Initial behavioral work has supported this claim by evaluating a narrow set of alternative interpretations of CI sentences and showing that comprehenders favor interpretations that are more likely to have been corrupted into the illusory sentence. In this study, we replicate and go substantially beyond this earlier work by directly predicting the strength of illusion with a quantitative model of the posterior probability of plausible interpretations, which we derive through a novel synthesis of statistical language models with human behavioral data. Our model explains not only the fine gradations in the strength of CI effects, but also a previously unexplained effect caused by pronominal vs. full noun phrase than-clause subjects. These findings support a noisy-channel theory of sentence comprehension by demonstrating that the theory makes novel predictions about the comparative illusion that bear out empirically. This outcome joins related evidence of noisy channel processing in both illusory and non-illusory contexts to support noisy channel inference as a unified computational-level theory of diverse language processing phenomena.

Yuhan Zhang, Erxiao Wang, Cory Shain

Margaret Jacks1DepartmentHall, Buildingof Linguistics,460 Rm.Stanford127, Stanford,UniversityCA ,94305-2150

Bayesian Inference Computational Psycholinguistics

Oct 20, 2025astro-ph.EP

Estimating Orbital Parameters of Direct Imaging Exoplanet Using Neural Network

In this work, we propose a flow-matching Markov chain Monte Carlo (FM-MCMC) algorithm for estimating the orbital parameters of exoplanetary systems, especially for those only one exoplanet is involved. Compared to traditional methods that rely on random sampling within the Bayesian framework, our approach first leverages flow matching posterior estimation (FMPE) to efficiently constrain the prior range of physical parameters, and then employs MCMC to accurately infer the posterior distribution. For example, in the orbital parameter inference of beta Pictoris b, our model achieved a substantial speed-up while maintaining comparable accuracy-running 77.8 times faster than Parallel Tempered MCMC (PTMCMC) and 365.4 times faster than nested sampling. Moreover, our FM-MCMC method also attained the highest average log-likelihood among all approaches, demonstrating its superior sampling efficiency and accuracy. This highlights the scalability and efficiency of our approach, making it well-suited for processing the massive datasets expected from future exoplanet surveys. Beyond astrophysics, our methodology establishes a versatile paradigm for synergizing deep generative models with traditional sampling, which can be adopted to tackle complex inference problems in other fields, such as cosmology, biomedical imaging, and particle physics.

Bo Liang, Hanlin Song, Chang Liu +9

Center for Gravitational Wave Experiment, National Microgravity Laboratory, Institute of Mechanics, Chinese Academy of Sciences, Beijing 100190, China · Taiji Laboratory for Gravitational Wave Universe (Beijing/Hangzhou), University of Chinese Academy of Sciences (UCAS), Beijing 100049, China · National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China +6

Markov Chain Monte Carlo Simulation-Based Inference Bayesian Inference

Oct 11, 2025stat.ML

Neural Variational Cut Posteriors without Upstream Data

In many applications, one must propagate parameter uncertainty from an earlier (upstream) analysis, available as samples, to subsequent (downstream) analyses without feedback. This problem is called cutting feedback or cut-Bayes, and the cut-posterior, the optimal posterior preserving information-flow constraints, is well characterized. However, sampling from it (e.g., via nested MCMC) is computationally intensive, while existing variational inference methods for cut-Bayes require access to upstream data and model, often unavailable. We propose a modular and provably accurate cut-Bayes approach requiring no access to upstream data or model. We leverage the characterization of the cut-posterior as the minimizer of the expected downstream conditional Kullback-Leibler divergence over the upstream posterior, replacing the expectation with the sample average over upstream draws. Our method, NeVI-Cut (neural variational inference for cut-Bayes), employs conditional normalizing flows as the variational family for downstream parameters. We provide fixed-data convergence rates of NeVI-Cut in terms of the richness of neural architecture and complexity of the cut-posterior. We establish, to our knowledge, first results on uniform Kullback-Leibler approximation rates of conditional distributions by common flow classes, yielding widely applicable fixed-data error rates for variational flows. A stochastic algorithm implements NeVI-Cut efficiently, and we demonstrate its speed and accuracy on multiple applications.

Jiafang Song, Sandipan Pramanik, Abhirup Datta

Department of Biostatistics, Johns Hopkins University, USA

Bayesian Inference Variational Inference Normalizing Flows

Aug 5, 2025eess.SP

Robust Sparse Bayesian Learning Based on Minimum Error Entropy for Noisy High-Dimensional Brain Activity Decoding

Objective: Sparse Bayesian learning provides an effective framework to solve high-dimensional problems in brain signal decoding. However, conventional likelihoods regarding data distributions, such as Gaussian or Bernoulli, are potentially inadequate for handling the noisy recordings of brain activity. Hence, this work aims to formulate a robust sparse Bayesian learning framework to address noisy high-dimensional brain activity decoding. Methods: Motivated by the commendable robustness of the minimum error entropy learning criterion for addressing non-Gaussian signals, this study reformulated the sparse Bayesian learning framework under a generalized Bayesian paradigm, in which the model parameter is regulated with the minimum error entropy loss rather than a conventional likelihood function. Results: Our developed SBL-MEE algorithm was evaluated with two real-world brain decoding tasks of regression and classification scenarios, respectively. Experimental results demonstrated that our approach not only realizes superior brain decoding performance than existing methods, but also presents more physiologically interpretable decoder patterns. Conclusion: Although minimum error entropy is not constructed from an arbitrary probabilistic distribution, it is effective to establish noise-robust inference in sparse Bayesian learning method. Significance: This work provides a powerful tool to improve brain activity decoding capability, particularly regarding the noisy high-dimensional setting, thus promoting biomedical engineering applications such as brain-computer interface.

Yuanhao Li, Badong Chen, Wenjun Bai +2

Center for Advanced Intelligence Project, RIKEN, Tokyo 103-0027, Japan · Department of Computational Brain Imaging, ATR Neural Information Analysis Laboratories, Kyoto 619-0237, Japan · Institute of Artificial Intelligence and Robotics (IAIR), Xi’an Jiaotong University, Xi’an 710049, China +1

Neural Decoding Bayesian Inference

Aug 4, 2025cs.CV

A Morse-Bott Framework for Blind Inverse Problems: Local Recovery Guarantees and the Failure of the MAP

Maximum A Posteriori (MAP) estimation is a cornerstone framework for blind inverse problems, where an image and a forward operator are jointly estimated as the maximizers of a posterior distribution. In applications such as blind deblurring, this principle is used to recover sharp images from degraded observations. In this paper, we analyze the recovery guarantees of MAP-based methods by adopting a \emph{Morse--Bott framework}. We model the image potential as a Morse--Bott function, where natural images are modeled as residing locally on a critical submanifold. This means that while the potential is locally flat along the

natural'' directions of the image manifold, it is strictly convex in the directions normal to it. We demonstrate that this Morse--Bott hypothesis aligns with the structural properties of state-of-the-art learned priors, a finding we validate through an experimental analysis of the potential landscape and its Hessian spectrum. Our theoretical results show that, in a neighborhood of the ground-truth image and operator, the posterior admits local minimizers that are stable both with respect to initialization (gradient descents converge to the same minimizer) and to small perturbations of the data (solutions vary smoothly with the observations). This local stability potentially provides a theoretical justification for the empirical success of well designed gradient-based optimization in these settings. However, we also demonstrate that this local stability is a \textbf{local} property: the

blurry trap'', well-known for sparse priors in blind deconvolution, persists even with state-of-the-art learned priors. Our findings demonstrate that the failure of MAP in blind deconvolution is not a limitation of prior quality, but an intrinsic characteristic of the landscape. We conclude that successful recovery depends on strategic initialization around favorable local minima.

Minh-Hai Nguyen, Edouard Pauwels, Pierre Weiss

IRIT, Toulouse University, Toulouse, France · Toulouse School of Economics, Universit´e Toulouse Capitole et IUF, Toulouse, France

Image Deblurring Bayesian Inference Image Inverse Problems

Jul 31, 2025stat.ML

Formal Bayesian Transfer Learning via the Total Risk Prior

Existing methods for transfer learning struggle to deal with situations where the source datasets are limited and not guaranteed to be well-aligned with the target dataset. A typical strategy is to use the empirical loss minimizer on the source data as a prior mean for the target parameters. Our key conceptual contribution is to use a risk minimizer conditional on source parameters instead. This allows us to construct a single joint prior distribution for all parameters from the source datasets as well as the target dataset. As a consequence, we benefit from full Bayesian uncertainty quantification and can perform model averaging via Gibbs sampling over indicator variables governing the inclusion of each source dataset. We show how a particular instantiation of our prior leads to a Bayesian Lasso in a transformed coordinate system and discuss computational techniques to scale our approach to moderately sized datasets. We discuss connections between the Maximum a Posteriori estimate associated with our approach and the recently proposed Trans-Lasso method and demonstrate that the MAP estimator MSE-dominates the Trans-Lasso in the normal means setting when there is no regularization on the source datasets. Finally, we perform numerical experiments finding that full Bayesian inference provides superior predictive performance relative to Trans-Lasso on a genetics application, especially when the source data are limited.

Nathan Wycoff, Ali Arab, Lisa O. Singh

Department of Mathematics and Statistics, University of Massachusetts Amherst · Department of Mathematics and Statistics, Georgetown University and · Department of Computer Science, Georgetown University

Bayesian Inference Transfer Learning

Jul 15, 2025stat.ML

LLMs are Bayesian, In Expectation, Not in Realization

Bayesian accounts of in-context learning face a direct objection: exact posterior predictives for exchangeable data are invariant to task-preserving order, yet transformers change next-token probabilities when the same examples are serialized differently. We show this objection targets a structural invariant rather than the quantity scoring online prediction. For any Bayesian reference, excess prequential code length is exactly cumulative predictive KL. For unordered support sets that must be serialized, the expected regret of a single admissible ordering decomposes into that of the order-averaged predictor plus an order-averaging gain. Exchangeability violations are therefore not binary refutations; they are priced by log loss. We instantiate the theory with KT/Dirichlet finite-alphabet prediction and coarsened Bayesian linear-regression (BLR) predictive distributions. On Qwen2.5-7B/14B, floored candidate distributions at support

256

have one-step excess code lengths of

0.020/0.011

bits for Bernoulli and

0.039/0.022

bits for four-way categorical prediction, with candidate mass above

0.999

; coarsened BLR continuations increasingly match the posterior-predictive digit distribution as support grows. A frequentist plug-in baseline sharpens the reading: the predictive distributions sit closer to the Bayesian posterior predictive than to the maximum-likelihood plug-in, by a margin largest at small support, where the plug-in is degenerate, and vanishing as the references converge. Position interventions and a from-scratch ablation localize order sensitivity to the positional encoding, activation patching tests causal use of decoded sufficient statistics, and permutation mixtures quantify the downstream log-loss cost of arbitrary orderings. Transformers need not realize exchangeable posterior predictives for every serialization to be Bayes-competitive prequential predictors.

Leon Chlon, Fatima Sheaib, Zein Khamis +3

Hassana Labs · Harvard University

Bayesian Inference In-Context Learning

Jul 11, 2025stat.ML

The Bayesian Approach to Continual Learning: An Overview

Continual learning is an online paradigm where a learner continually accumulates knowledge from different tasks encountered over sequential time steps. Importantly, the learner is required to extend and update its knowledge without forgetting about the learning experience acquired from the past, and while avoiding the need to retrain from scratch. Given its sequential nature and its resemblance to the way humans think, continual learning offers an opportunity to address several challenges which currently stand in the way of widening the range of applicability of deep models to further real-world problems. The continual need to update the learner with data arriving sequentially strikes inherent congruence between continual learning and Bayesian inference which provides a principal platform to keep updating the prior beliefs of a model given new data, without completely forgetting the knowledge acquired from the old data. This survey inspects different settings of Bayesian continual learning, namely task-incremental learning and class-incremental learning. We begin by discussing definitions of continual learning along with its Bayesian setting, as well as the links with related fields, such as domain adaptation, transfer learning and meta-learning. Afterwards, we introduce a taxonomy offering a comprehensive categorization of algorithms belonging to the Bayesian continual learning paradigm. Meanwhile, we analyze the state-of-the-art while zooming in on some of the most prominent Bayesian continual learning algorithms to date. Furthermore, we shed some light on links between continual learning and developmental psychology, and correspondingly introduce analogies between both fields. We follow that with a discussion of current challenges, and finally conclude with potential areas for future research on Bayesian continual learning.

Tameem Adel

NPL, Maxwell Centre, University of Cambridge, JJ Thomson Avenue, Cambridge, CB3 OHE, United Kingdom.

Class-Incremental Learning Continual Learning Bayesian Inference Incremental Learning

Jun 27, 2025stat.ML

Bayesian Invariance Modeling of Multi-Environment Data

Invariant prediction [Peters et al., 2016] analyzes feature/outcome data from multiple environments to identify invariant features - those with a stable predictive relationship to the outcome. Such features support generalization to new environments and help reveal causal mechanisms. Previous methods have primarily tackled this problem through hypothesis testing or regularized optimization. Here we develop Bayesian Invariant Prediction (BIP), a probabilistic model for invariant prediction. BIP encodes the indices of invariant features as a latent variable and recover them by posterior inference. Under the assumptions of Peters et al. [2016], the BIP posterior targets the true invariant features. We prove that the posterior is consistent and that greater environment heterogeneity leads to faster posterior contraction. To handle many features, we design an efficient variational approximation called VI-BIP. In simulations and real data, we find that BIP and VI-BIP are more accurate and scalable than existing methods for invariant prediction.

Luhuan Wu, Mingzhang Yin, Yixin Wang +2

Department of Applied Mathematics and Statistics, Johns Hopkins University. · Warrington College of Business, University of Florida. · Department of Statistics, University of Michigan. +2

Bayesian Inference