Generalized Hierarchical Bayesian Segmentation with Irregular Designs, Multi-Sequence Hierarchies, and Grouped/Latent-Group Designs

Omid Shams Solari

cs.LGMar 16, 2026

Generalized Hierarchical Bayesian Segmentation with Irregular Designs, Multi-Sequence Hierarchies, and Grouped/Latent-Group Designs

Authors: Omid Shams Solari

Organizations: sAIm Labs

Abstract

Bayesian change-point and segmentation models provide uncertainty-aware piecewise-constant representations of ordered data, but exact inference is often limited to narrow likelihood classes, single sequences, or index-uniform designs. We present BayesBreak, a modular offline Bayesian segmentation framework that separates local block scoring from global inference: each candidate block supplies a marginal likelihood and any needed moment numerators, while a dynamic program combines these scores to compute posteriors over segment counts, boundaries, and latent signals. For weighted exponential-family likelihoods with conjugate priors, block evidences and posterior moments are available in closed form from cumulative sufficient statistics, enabling exact sum-product inference for $p(y\mid k)$ , $p(k\mid y)$ , boundary marginals, and Bayes regression curves. We distinguish these summaries from the joint MAP segmentation, recovered by a separate max-sum recursion. BayesBreak supports design-aware partition priors for irregular observations, exact pooling across replicates with shared boundaries, and latent-template mixtures with exact EM updates. For non-conjugate GLM blocks, the same DP layer can use deterministic local approximations such as Laplace, variational methods, EP, or quadrature. We prove a posterior-odds stability bound: uniform per-block log-evidence error $\varepsilon$ perturbs $k$ -odds and boundary-odds by at most $(k+k')\varepsilon$ and $2k\varepsilon$ . Validation includes synthetic recovery, calibration, and scaling experiments, plus four real-data illustrations: well-log geology, array-CGH copy number, equity-return volatility, and CpG-atlas methylation.

Figures & tables

Symbol	Meaning
$n$	Number of observations on the ordered design.
$x_{i}$	Design point / location of observation $y_{i}$ .
$y_{i}$	Observation at location $x_{i}$ (single-sequence case).
$w_{i}$	Deterministic exposure, trial-count, or precision weight when required by the observation model. Unless otherwise stated, point-observation models use $w_{i}\equiv 1$ .
$k$ ; $k_{\max}$	Number of segments; maximum segment count considered by the DP.
$t=(t_{0},\dots,t_{k})$	Boundary vector, with $0=t_{0}<t_{1}<\cdots<t_{k}=n$ .

Table 1

Panel of Figure 1 — Figure 1: Plate model for a single conjugate EF block. Shaded nodes are observed; unshaded are latent; point nodes are fixed hyperparameters/exposures. Marginalizing $\theta$ gives $A^{(0)}_{ij}$ .

Panel of Figure 2 — Figure 2: Plate model for multi-subject pooling under shared boundaries. The boundary vector $t$ is shared across subjects; each subject has its own segment parameters $\theta^{(s)}_{q}$ and exposures $w^{(s)}_{i}$ .

Panel of Figure 3 — Figure 3: Plate model with known group memberships $z_{s}$ . Each group has its own segmentation template $t^{(g)}$ ; subjects within a group share that template but retain subject-specific segment parameters $\theta^{(g,s)}_{q}$ .

Panel of Figure 4 — Figure 4: Plate model for the latent-group template mixture. Unshaded $z_{s}$ are latent; EM alternates between exact E-step responsibilities $p(z_{s}=g\mid y^{(s)},\pi,\tau)$ and M-step max-sum updates of the templates $\tau_{g}$ .

Family	Prior	Posterior	Block evidence $A^{(0)}_{ij}$	Mean $\mathbb{E}[m(\theta)\|(i,j]]$
Gaussian $(\mu,\sigma^{2}/w)$	$\mathcal{N}(\nu,\rho^{2})$	$\mathcal{N}(\nu_{B},\rho_{B}^{2})$	closed-form normal ratio	$(\rho^{2}S+\sigma^{2}\nu)/\kappa$
Poisson $(\lambda w)$	$\mathrm{Gamma}(a_{0},b_{0})$	$\mathrm{Gamma}(a_{0}+C,b_{0}+W)$	$\propto b_{0}^{a_{0}}\Gamma(a_{0}+C)/(b_{0}+W)^{a_{0}+C}$	$(a_{0}+C)/(b_{0}+W)$
Binomial $(m,p)$	$\mathrm{Beta}(a_{0},b_{0})$	$\mathrm{Beta}(a_{0}+C,b_{0}+M-C)$	$B(a_{0}+C,b_{0}+M-C)/B(a_{0},b_{0})$	$(a_{0}+C)/(a_{0}+b_{0}+M)$
NegBin $(r,p)$	$\mathrm{Beta}(a_{0},b_{0})$	$\mathrm{Beta}(a_{0}+N,b_{0}+C)$	$B(a_{0}+N,b_{0}+C)/B(a_{0},b_{0})$	$(a_{0}+N)/(a_{0}+b_{0}+N+C)$
Beta-obs $(\mu,\phi)$	$\mathrm{Beta}(a_{0},b_{0})$	(1D quadrature)	Gauss–Legendre nodes	from quadrature

Table 2: Block families derived in Section 11 . Base-measure factor

\exp\{H_{ij}\}

is implicit in each row. All families share the ratio-of-normalizers structure required by Theorem 5.1 ; the Beta-obs row is non-conjugate and uses 1D quadrature but remains compatible with the DP layer.

Panel of Figure 5 — Figure 5: Plate model for a non-conjugate block with Pólya–Gamma augmentation. Conditional on $\omega_{t}$ , the block is quadratic in $\theta$ with a Gaussian posterior (Proposition 12.5 ); the same template applies to Laplace and variational routines with the $\omega_{t}$ node omitted.

Evaluation objective	Metric
Boundary recovery	Precision / Recall / F1 at tolerance $\tau$
Signal recovery	MSE between estimated and true latent signal
Posterior calibration	Calibration curve / ECE for $p(b_{i}=1\mid y)$
Model fit quality	$-\log p(y)$ (marginal evidence) or predictive NLL
Runtime	Wall-clock time vs. $n$ and $k_{\max}$

Table 3: Metrics used in the synthetic experiments. When ground-truth boundaries are available, boundary-level metrics use a small tolerance window (here

\tau=2

indices) so that near-misses are not counted as complete failures.

Figure 6: Single-sequence Gaussian example. Top: marginal posterior probability that each interior index $i\in\{1,\dots,n-1\}$ is a changepoint, with dashed vertical lines marking the true changepoints. Bottom: observed data, true latent mean, exported joint MAP segmentation, and 90% segment-level posterior intervals.

Quantity	Value
Selected $k$ ( k_ml_ )	4
Posterior mean $\mathbb{E}[k]$	4.742
MAP $k$	4
$\log p(y)$	-18.031

Table 4: Posterior summary for the single-sequence Gaussian example in Figure 6 . The table separates the displayed exported segmentation from posterior summaries of the segment count.

Figure 7: Likelihood-family showcase. The first three panels use closed-form conjugate block integrals (Gaussian, Poisson, and Binomial), while the final panel uses the one-dimensional Beta-response quadrature routine. In each case the fitted BayesBreak signal is close to the true piecewise-constant latent parameter.

Family	n	$k^{\star}$	$\hat{k}$	F1@ $\tau$	MAE	MSE	$-\log p(y)/n$
Gaussian	120	3	3	0.968	0.00	0.0023	0.191
Poisson	120	3	4	0.777	0.20	0.4501	2.162
Binomial	120	3	3	0.960	0.06	0.0004	2.108
Beta-valued	120	3	3	1.000	0.00	0.0001	2.620

Table 5: Available synthetic summary across the bundled family-specific examples. The final row corresponds to the Beta-response quadrature example rather than to a conjugate closed-form model.

Figure 8: Calibration of marginal boundary posterior probabilities under synthetic Gaussian data. The dashed diagonal denotes perfect calibration; the orange curve is the empirical frequency within bins of predicted probability.

Figure 9: Latent-group pooling on synthetic Gaussian sequences. Left: posterior responsibilities for each sequence and latent group (sequences sorted for display). Right: group-specific marginal boundary probabilities under the fitted latent templates.

Method	$\max\|\Delta\log A^{0}\|$	time (s)	F1@ $\tau$	$\hat{k}$
quadrature	0.000	0.108	0.286	6
laplace	4.081	0.066	0.286	6
jj	4.076	0.087	0.500	3
ep	17.654	0.077	0.500	7
pg_vb	4.076	0.090	0.500	3

Table 6: Approximation trade-offs for a non-conjugate Bernoulli/logistic block model. Quadrature is used as the local reference block integral; the remaining methods trade block-level fidelity against computational cost.

Figure 10: Runtime scaling for the Gaussian implementation as a function of series length $n$ for two values of $k_{\max}$ . Points are empirical means over repeated runs; error bars denote one standard deviation.

n	$k_{\max}$	mean (s)	std (s)
50	20	0.0718	0.0010
100	20	0.1530	0.0024
200	20	0.3200	0.0049
400	20	0.6681	0.0029

Table 7: Numerical runtime values corresponding to Figure 10 . The bundled table reports the

k_{\max}=20

configuration.

Figure 11: Well-log segmentation (placeholder). Top: standardized NMR measurements with exported joint MAP segmentation. Bottom: marginal boundary posterior probabilities. The plot is produced by the pipeline in Appendix A.12.1 ; the version archived here is a placeholder pending the finalized run.

Configuration	$\widehat{k}$	MAP evidence	ECE (boundary)	Runtime (s)
Index-uniform prior, $g\equiv 1$	—	—	—	—
Length-aware prior, $g(\ell)\propto\ell$	—	—	—	—

Table 8: Well-log placeholder results. Entries will be filled by the pipeline in Appendix A.12.1 .

Figure 12: Array-CGH multi-subject pooling (placeholder). Top: four representative Coriell profiles with per-subject exported segmentations. Bottom: pooled marginal boundary posterior. Figure will be regenerated by the pipeline in Appendix A.12.2 .

Strategy	Boundary F1	Boundary MAE	Pooled log-evidence	Runtime (s)
Independent per-subject (no pooling)	—	—	—	—
Shared boundaries, subject-specific $\mu$	—	—	—	—

Table 9: Array-CGH placeholder results comparing independent per-subject segmentation to pooled shared-boundary inference. Ground-truth boundaries from Snijders et al. (2001) annotations.

Figure 13: S&P 500 volatility regimes (placeholder). Top: log-squared daily log-returns with exported joint MAP segmentation, covering roughly 2018–2022. Bottom: marginal boundary posteriors; vertical dashed lines annotate major macro events (March 2020 onset of the COVID-19 market shock, Feb 2022). Figure will be regenerated by the pipeline in Appendix A.12.3 .

Block model	$\widehat{k}$	Log evidence	Visual alignment with known macro events
Gaussian on $\log r_{t}^{2}$	—	—	—
Poisson on threshold crossings	—	—	—

Table 10: S&P 500 volatility-regime placeholder results. The “visual alignment” column is a qualitative description in lieu of ground-truth boundaries.

Figure 14: DNA-methylation segmentation on a CpG-atlas region (placeholder). Top:

\beta

-values along a selected chromosome-arm region, colored by local precision

\phi_{t}

(log-scale). Bottom: marginal boundary posterior on the same region. Figure will be regenerated by the pipeline in Appendix A.12.4 .

Region / cell type	$\widehat{k}$	Held-out log-predictive	Boundary F1 vs. atlas
Region A, cell type 1	—	—	—
Region A, cell type 2	—	—	—

Table 11: CpG-atlas methylation placeholder results. Held-out log-predictive is computed on a withheld subset of CpGs within the same region; boundary F1 is computed against the atlas-provided block annotations.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Theme	Representative papers	Annotation relative to BayesBreak
Foundational product partitions	Barry and Hartigan (1992) ; Barry and Hartigan (1993)	Introduced the product-partition view of Bayesian changepoint analysis. BayesBreak adopts exactly this segment-factorization principle, but repackages it around a reusable block-evidence interface that is intended to work across multiple observation families and prediction tasks.
Exact offline DP	Fearnhead (2006) ; Hutter (2007)	These are the closest conceptual ancestors of the exact offline core. Fearnhead (2006) emphasizes exact posterior recursion for changepoints, while Hutter (2007) highlights Bayesian curve summaries and evidence-based model comparison. BayesBreak systematizes both inside a single modular framework and makes the distinction between sum-product summaries and joint MAP segmentations explicit.
Segment dependence and richer local models	Fearnhead and Liu (2011) ; Ruggieri and Lawrence (2014)	These papers show how offline Bayesian segmentation can be enriched beyond i.i.d. block means, for example by allowing dependence across neighboring segments or by fitting piecewise polynomial regression. They motivate the claim that the real bottleneck is the local block integral, not the DP itself.
Frequentist exact partitioning	Auger and Lawrence (1989) ; Jackson et al. (2005) ; Killick et al. (2012)	These methods are critical baselines for optimization-based segmentation. They solve related partitioning problems efficiently but do not natively provide posterior probabilities over boundary locations or segment counts. BayesBreak differs mainly in inferential target rather than in the use of dynamic programming per se.
Penalized and multiscale segmentation	Tibshirani et al. (2005) ; Bleakley and Vert (2011) ; Frick et al. (2014) ; Fryzlewicz (2014)	These approaches offer strong algorithmic or confidence guarantees, especially in large-scale or multisequence settings. They are natural comparators for future empirical work, but they optimize penalized objectives or detection rules rather than the posterior quantities emphasized here.
Online and trans-dimensional Bayes	Adams and MacKay (2007) ; Green (1995) ; Denison et al. (1998)	These references represent two alternative Bayesian regimes: streaming detection and fully flexible posterior simulation over model dimension. BayesBreak instead targets exact offline inference when block marginalization is feasible, and deterministic approximations when only the block integral is intractable.

Appendix

Table 12: Annotated literature review.

Change-Point Detection

Change-point detection identifies when the statistical behavior or underlying regime of a sequence changes.

See all 83 papers

Oct 14, 2026

Time-partitioning Gaussian Process Mixture Model

Nonstationarity and heterogeneous dynamics are common in financial and macroeconomic time series. Volatility alternates between calm and turbulent regimes, interest rates follow monetary-policy cycles, and major events trigger persistent structural shifts. We propose the Time-partitioning Gaussian Process Mixture Model (TPGPMM), a Bayesian nonparametric framework for regime identification and nonstationary prediction that infers such regimes and their within-regime dependence from data, without Markovian assumptions. It combines (i) an adaptive Pitman-Yor process prior on ordered compositions, mapped to block lengths with contiguity enforced by construction, and (ii) regime-specific Gaussian processes for temporal dependence, yielding a nonstationary predictive distribution. By inferring its discount parameter, the prior adapts to homogeneous and highly heterogeneous regime lengths. Unlike sequential or Markov-switching methods, it flags persistent structural shifts as change points rather than reacting to transient shocks. A split-merge Metropolis-within-Gibbs sampler with exact Gibbs updates of regime boundaries jointly learns change points and hyperparameters in batch, quantifying uncertainty over the partition and within-regime dynamics. With an RBF kernel and a fixed, data-informed noise level, TPGPMM produces a sparse, confident segmentation. On simulated data it recovers the true regimes, and on the CBOE Volatility Index and the 10-year US Treasury yield it identifies regimes aligned with documented market and monetary-policy events. For prediction, variants estimating the noise level fit better than change-point baselines and fit and forecast better than the main specification, at the cost of a less sparse or less event-aligned segmentation; with a regime mean, they fit better than a fitted stationary Gaussian process and have lower forecast RMSE except at the shortest horizons.

Guilherme Piantino, Hedibert F. Lopes

Insper Institute of Education and Research, São Paulo

Time Series Modeling Non-Stationary Time Series Change-Point Detection Gaussian Processes Bayesian Inference

Ask PDF

Oct 18, 2024

Predictive variational inference: Learn the predictively optimal posterior distribution

Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.

Jinlin Lai, Antonio Linero, Yuling Yao

College of Information and Computer Sciences, University of Massachusetts Amherst · Department of Statistics and Data Sciences, University of Texas, Austin

Model Misspecification Variational Inference

Ask PDF

Sep 30, 2026

Amortized Bayesian Inference on Multilevel Models of Arbitrary Structure

We develop a general method for amortized Bayesian inference on multilevel models of arbitrary structure. Given a generative model specified as a directed acyclic graph, our method automatically derives valid factorizations of the joint posterior and matching neural network architectures. The key steps, graph expansion and graph inversion, yield an inverse graph that determines how inference networks are stacked and conditioned, producing factorizations that amortize over the number of groups and the number of observations within each group. Unlike approaches that simplify the dependency structure to speed up learning or inference, our method preserves all conditional independence and exchangeability assumptions of the generative model. Across three case studies, it closely matches gold-standard samplers on models with more than 6,500 parameters while reducing inference to a near-instant forward pass once trained.

Daniel Habermann, Andreas Bulling, Stefan T. Radev +1

Department of Statistics TU Dortmund University, Germany · Institute for Visualisation and Interactive Systems University of Stuttgart, Germany · Department of Cognitive Science Rensselaer Polytechnic Institute, USA

Amortized Inference Hierarchical Bayesian Modeling Probabilistic Graphical Models

Ask PDF

Abstract

Change-Point Detection

Explore similar work

Time-partitioning Gaussian Process Mixture Model

Predictive variational inference: Learn the predictively optimal posterior distribution

Amortized Bayesian Inference on Multilevel Models of Arbitrary Structure