Time-partitioning Gaussian Process Mixture Model

Guilherme Piantino; Hedibert F. Lopes

stat.MLOct 14, 2026

Time-partitioning Gaussian Process Mixture Model

Authors: Guilherme Piantino, Hedibert F. Lopes

Organizations: Insper Institute of Education and Research, São Paulo

Abstract

Nonstationarity and heterogeneous dynamics are common in financial and macroeconomic time series. Volatility alternates between calm and turbulent regimes, interest rates follow monetary-policy cycles, and major events trigger persistent structural shifts. We propose the Time-partitioning Gaussian Process Mixture Model (TPGPMM), a Bayesian nonparametric framework for regime identification and nonstationary prediction that infers such regimes and their within-regime dependence from data, without Markovian assumptions. It combines (i) an adaptive Pitman-Yor process prior on ordered compositions, mapped to block lengths with contiguity enforced by construction, and (ii) regime-specific Gaussian processes for temporal dependence, yielding a nonstationary predictive distribution. By inferring its discount parameter, the prior adapts to homogeneous and highly heterogeneous regime lengths. Unlike sequential or Markov-switching methods, it flags persistent structural shifts as change points rather than reacting to transient shocks. A split-merge Metropolis-within-Gibbs sampler with exact Gibbs updates of regime boundaries jointly learns change points and hyperparameters in batch, quantifying uncertainty over the partition and within-regime dynamics. With an RBF kernel and a fixed, data-informed noise level, TPGPMM produces a sparse, confident segmentation. On simulated data it recovers the true regimes, and on the CBOE Volatility Index and the 10-year US Treasury yield it identifies regimes aligned with documented market and monetary-policy events. For prediction, variants estimating the noise level fit better than change-point baselines and fit and forecast better than the main specification, at the cost of a less sparse or less event-aligned segmentation; with a regime mean, they fit better than a fitted stationary Gaussian process and have lower forecast RMSE except at the shortest horizons.

Figures & tables

Figure 1: Graphical representation of the TPGPMM. Dashed blue boxes denote conditional independence across regimes, separating the global partition (

\rho

) from local parameters (

\boldsymbol{\phi}_{k}

). Within each regime, the gray lines connecting latent nodes (

f

) illustrate the non-Markovian GP covariance structure.

Scenario	Method	$\hat{K}$ (mean $\pm$ SD)	Precision	Recall	F1	CP_MAE
homogeneous	TPGPMM	10.3 $\pm$ 0.7	0.968	0.998	0.981	0.21 $\pm$ 1.41
homogeneous	TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	10.3 $\pm$ 0.6	0.973	0.998	0.984	0.20 $\pm$ 1.41
homogeneous	TPGPMM (Matérn-3/2, $\sigma_{\mathrm{n}}$ estimated)	10.0 $\pm$ 0.2	0.996	0.996	0.996	0.40 $\pm$ 1.98
homogeneous	TPGPMM (Matérn-3/2, regime mean, $\sigma_{\mathrm{n}}$ estimated)	10.0 $\pm$ 0.2	0.996	0.996	0.996	0.40 $\pm$ 1.98
homogeneous	BOCPD	11.1 $\pm$ 1.3	0.844	0.938	0.885	4.98 $\pm$ 5.56
homogeneous	HDP-SLDS	15.5 $\pm$ 4.7	0.563	0.833	0.658	8.56 $\pm$ 8.33

Table 1: Simulation-study change-point recovery summary across

50

replications per scenario, for TPGPMM, its three alternatives with estimated noise and the baselines. SD denotes standard deviation.

Figure 2: Distribution of the posterior mean of

\sigma

across

50

replicate seeds in the homogeneous and power-law scenarios, for TPGPMM and its three alternatives with estimated noise (one colour and marker per specification). Each point is the posterior mean for one seed, jittered within each specification; horizontal lines mark the medians.

Event (Date)	Market Impact
China “Black Monday” (2015-08-24)	China slowdown triggered U.S. liquidity strains.
Brexit Vote (2016-06-24)	Surprise “Leave” vote sparked global equity sell-offs.
“Volmageddon” (2018-02-05)	VIX spike driven by short-volatility deleveraging.
COVID Panic (2020-03-16)	Pandemic lockdowns caused acute funding strains.
Russia–Ukraine (2022-02-24)	Invasion led to risk sell-offs and commodity surges.
US Bank Turmoil (2023-03-10)	Silicon Valley Bank failure sparked bank solvency and rate volatility.

Table 2: Six benchmark market events used to interpret the posterior VIX segmentation.

Figure 3: Analysis of the VIX dataset. Top panel: volatility index with the six benchmark market events. Lower five panels: posterior change-point probabilities

\widehat{\tau}_{t}

for TPGPMM and the benchmark models.

Figure 4: Posterior similarity matrices for the VIX time series under each method.

Figure 5: MAP segmentation of the VIX series. Coloured lines show the per-block GP posterior means under the modal partition

\widehat{\rho}

, with kernel hyperparameters fixed to the posterior mean across MCMC draws matching that partition. Dashed vertical lines mark the inferred internal change-points. The legend gives the averaged kernel hyperparameters

(\ell_{k},s_{k}^{2})

for each of the eleven blocks.

Event (Date)	Policy / market impact
Fed mid-cycle easing (2019-07-31)	First of three 2019 rate cuts as growth and trade-war risks mounted.
COVID emergency cuts (2020-03-15)	Policy rate cut to the zero lower bound; large-scale asset purchases.
Hawkish pivot (2021-12-15)	The Federal Open Market Committee (FOMC) accelerated tapering, signalling the 2022 tightening cycle.

Table 3: Benchmark monetary-policy events used to interpret the posterior 10-year Treasury segmentation.

Regime	Interpretation	Dates	$N$	Mean yield	Daily-change SD	Boundary post. prob.
$1$	Normalisation and hiking cycle	2015-01-02 to 2019-07-29	$1{,}144$	$2.32\%$	$4.22$ bp	—
$2$	Mid-cycle easing	2019-07-30 to 2020-03-19	$160$	$1.65\%$	$6.50$ bp	$0.22$
$3$	COVID; quantitative easing	2020-03-20 to 2022-02-25	$486$	$1.19\%$	$4.26$ bp	$1.00$
$4$	Tightening onset	2022-02-28 to 2022-06-10	$73$	$2.61\%$	$7.15$ bp	$0.47$
$5$	Restrictive policy	2022-06-13 to 2023-12-29	$388$	$3.77\%$	$7.74$ bp	$0.38$

Table 4: MAP segmentation of the standardised 10-year Treasury yield under TPGPMM. Mean yield (in percent) and daily-change standard deviation (in basis points) are computed within each inferred block; the boundary posterior probability is the posterior probability of a change-point within

\pm 10

trading days of the block’s left boundary (regime 1 has no left boundary).

Figure 6: Posterior predictive mixture under TPGPMM on log-VIX. The red line is the posterior mean of

Y_{t}

, the shaded band is the

95\%

credible band, and the dashed vertical lines mark the modal-partition change-points. The construction follows Equation ( 16 ).

Method	Target	RMSE	NLPD	CRPS	RMSE/sd	$\Delta$ NLPD	CRPS/sd	$n$
TPGPMM	$\log$ VIX	$0.0714$	$-1.1042$	$0.0417$	$0.2082$	$-1.4537$	$0.1215$	$2264$
TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	$\log$ VIX	$0.0564$	$-1.4339$	$0.0318$	$0.1642$	$-1.7834$	$0.0928$	$2264$
TPGPMM (Matérn-3/2, $\sigma_{\mathrm{n}}$ estimated)	$\log$ VIX	$0.0409$	$-1.7048$	$0.0233$	$0.1193$	$-2.0543$	$0.0678$	$2264$
TPGPMM (Matérn-3/2, regime mean, $\sigma_{\mathrm{n}}$ estimated)	$\log$ VIX	$0.0261$	$-2.0406$	$0.0159$	$0.0760$	$-2.3901$	$0.0463$	$2264$
Standard GP (RBF)	$\log$ VIX	$0.0502$	$-1.5255$	$0.0277$	$0.1464$	$-1.8750$	$0.0808$	$2264$
Standard GP (Matérn-3/2)	$\log$ VIX	$0.0394$	$-1.7117$	$0.0223$	$0.1149$	$-2.0612$	$0.0651$	$2264$

Table 5: In-sample posterior-predictive fit on the log-VIX series (

n\approx 2264

trading days). RMSE, NLPD and CRPS are the in-sample posterior-predictive metrics; RMSE/sd,

\Delta

NLPD and CRPS/sd are their scale-normalised variants. Because the baselines operate on different targets (second column), the normalised columns are the comparable ones. Lower is better throughout. The standard GPs have hyperparameters fitted by maximum marginal likelihood; the laGP and HDP-SLDS rows are from the original study, and Martínez–Mena A and B are fitted with the settings of Martínez and Mena (2014) . Martínez–Mena A has no observation noise, so its predictive of

y_{t}

conditions on

y_{t-1}

within the regime (one step ahead) rather than on

y_{t}

itself.

Method	Target	RMSE	NLPD	CRPS	RMSE/sd	$\Delta$ NLPD	CRPS/sd	$n$
TPGPMM	std. yield	$0.0605$	$-1.1785$	$0.0371$	$0.0605$	$-2.5972$	$0.0371$	$2251$
TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	std. yield	$0.0357$	$-1.8700$	$0.0203$	$0.0357$	$-3.2887$	$0.0203$	$2251$
TPGPMM (Matérn-3/2, $\sigma_{\mathrm{n}}$ estimated)	std. yield	$0.0244$	$-2.1798$	$0.0143$	$0.0244$	$-3.5985$	$0.0143$	$2251$
TPGPMM (Matérn-3/2, regime mean, $\sigma_{\mathrm{n}}$ estimated)	std. yield	$0.0215$	$-2.2681$	$0.0129$	$0.0215$	$-3.6868$	$0.0129$	$2251$
Standard GP (RBF)	std. yield	$0.0452$	$-1.6580$	$0.0250$	$0.0452$	$-3.0767$	$0.0250$	$2251$
Standard GP (Matérn-3/2)	std. yield	$0.0320$	$-1.9500$	$0.0181$	$0.0320$	$-3.3688$	$0.0181$	$2251$

Table 6: In-sample posterior-predictive fit on the standardised 10-year Treasury yield (

n\approx 2{,}251

). Columns as in Table 5 .

	RMSE/sd					CRPS/sd
Model	$h=1$	2	3	4	5	$h=1$	2	3	4	5
TPGPMM	0.706	0.893	1.069	1.237	1.400	0.369	0.473	0.574	0.671	0.761
TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	0.636	0.846	1.046	1.243	1.435	0.322	0.436	0.544	0.649	0.752
TPGPMM (Matérn-3/2, $\sigma_{\mathrm{n}}$ estimated)	0.548	0.780	0.978	1.146	1.288	0.272	0.397	0.501	0.590	0.674
TPGPMM (Matérn-3/2, regime mean, $\sigma_{\mathrm{n}}$ estimated)	0.476	0.664	0.796	0.886	0.940	0.227	0.325	0.392	0.441	0.482
Standard GP (RBF)	0.503	0.726	0.886	0.978	1.003	0.249	0.360	0.440	0.496	0.533

Table 7: Out-of-sample forecasting on log-VIX. All models are fit once on the

2015

–

2023

training window and forecast

1

–

5

trading days ahead across

2024

(

252

days) with a daily rolling origin, conditioning on observed data up to each origin but not re-estimated. TPGPMM forecasts are posterior mixtures over 500 draws, each forecasting from its final regime; the standard GPs are single stationary GPs with hyperparameters fitted by maximum marginal likelihood on the training window. Entries are horizon-wise RMSE and CRPS normalised by the test standard deviation; lower is better, best in bold.

	RMSE/sd					CRPS/sd
Model	$h=1$	2	3	4	5	$h=1$	2	3	4	5
TPGPMM	0.331	0.433	0.530	0.632	0.729	0.194	0.247	0.301	0.359	0.414
TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	0.282	0.386	0.484	0.595	0.699	0.158	0.216	0.272	0.339	0.403
TPGPMM (Matérn-3/2, $\sigma_{\mathrm{n}}$ estimated)	0.265	0.428	0.525	0.639	0.725	0.149	0.246	0.320	0.406	0.483
TPGPMM (Matérn-3/2, regime mean, $\sigma_{\mathrm{n}}$ estimated)	0.250	0.383	0.448	0.531	0.595	0.141	0.222	0.274	0.330	0.377
Standard GP (RBF)	0.291	0.447	0.597	0.761	0.909	0.161	0.246	0.331	0.429	0.517

Table 8: Out-of-sample forecasting on the standardised 10-year Treasury yield over

2024

(

250

days), with the protocol of Table 7 .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Specification	Kernel	Regime mean	Noise level $\sigma_{\mathrm{n}}$
TPGPMM	RBF	no	fixed at $0.1$
TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	RBF	no	estimated
TPGPMM (Matérn-3/2, $\sigma_{\mathrm{n}}$ estimated)	Matérn-3/2	no	estimated
TPGPMM (Matérn-3/2, regime mean, $\sigma_{\mathrm{n}}$ estimated)	Matérn-3/2	yes	estimated

Appendix

Table 9: Within-regime specifications. The amplitude prior is

\log s_{k}^{2}\sim\mathcal{N}(0,0.8^{2})

except for the regime-mean alternative, for which it is

\mathcal{N}(\log\widehat{\mathrm{var}}(\mathbf{y}),1)

Term	Without it	Check
Dimension-matched split with exact inverse merge (Prop. 1 )	Split and merge are not inverse maps; the reversible-jump ratio is undefined	Prior-only runs
Normalising constants of the hyperpriors (Prop. 2 )	Target multiplied by $(2\pi s_{\ell}s_{s})^{K}\approx 3^{K}$	Prior-only law of $K$
Log-scale Jacobian of the random walk (Prop. 4 )	Prior of $\log\ell_{k}$ and $\log s_{k}^{2}$ shifted by $-s^{2}$ ( $-0.36$ , $-0.64$ )	Prior-only marginals
Truncation constant $Z_{n}$ in the $(\sigma,\theta)$ updates (Prop. 5 )	Hyperprior tilted to mean $\sigma\approx 0.04$ , $\theta\approx 2.2$	Prior-only KS tests; exact sampler
Adaptation during burn-in only	Retained chain is not a fixed Markov kernel	—
Gibbs draw of the boundaries (Prop. 3 )	Uniform shuffles, mostly rejected	ESS (Appendix H )

Appendix

Table 10: Terms each update of the sampler must include, the distribution targeted without them, and how the sampler was checked. The last two rows concern efficiency, not correctness.

Block $k$	$\ell_{k}$	$s_{k}^{2}$	$m_{k}$
1	25.0	0.85	0.0
2	8.0	0.70	1.8
3	6.0	0.75	$-1.7$
4	10.0	0.70	1.6
5	7.0	0.70	$-1.8$
6	12.0	0.65	1.7

Appendix

Table 17

Series	Specification	acc split	acc merge	moved Gibbs	ESS K	ESS σ	ESS θ	$\widehat{R}$ (max)	min/chain
VIX	TPGPMM	0.055	0.055	0.439	143	7336	3013	1.031	8.3
VIX	TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	0.065	0.064	0.381	12	14083	51	1.266	5.3
VIX	TPGPMM (Matérn-3/2, $\sigma_{\mathrm{n}}$ estimated)	0.061	0.061	0.354	544	3452	10183	1.006	50.0
VIX	TPGPMM (Matérn-3/2, regime mean, $\sigma_{\mathrm{n}}$ estimated)	0.147	0.145	0.452	88	8470	280	1.047	6.5
Treasury	TPGPMM	0.126	0.127	0.780	387	1762	4441	1.011	13.8
Treasury	TPGPMM ( $\sigma_{\mathrm{n}}$ estimated)	0.095	0.091	0.539	26	4604	158	1.139	2.0

Appendix

Table 11: Sampler diagnostics. Four chains per series started at

K_{0}\in\{1,5,15,30\}

; acceptance rates are averaged over chains; ESS is the rank-normalised bulk effective sample size pooled over chains and

\widehat{R}

the rank-normalised split-

\widehat{R}

[ Vehtari et al., 2021 ] , reported as the maximum over

K,\sigma,\theta

; statistics on the second half of each chain.

Figure 7: Post-burn-in traces of the global quantities (

K

\sigma

\theta

, and the log-posterior) for the four VIX chains of TPGPMM, started from

K_{0}=1,5,15,30

equal blocks (first half of the

60{,}000

iterations discarded as burn-in). The four chains overlap in every panel.

Series	Method	$E[K]$ (95% interval)	$E[\sigma]$	$E[\theta]$	$\mathrm{TV}_{K}$	$\max\|\Delta\tau\|$
VIX	Exact, 210-point grid	14.43 [11, 18]	0.072	3.67	0.227	0.24
VIX	Exact, 783-point grid	13.48 [10, 18]	0.083	3.51	0.029	0.15
VIX	Exact, 1,760-point grid	13.69 [11, 18]	0.082	3.58	0.049	0.14
VIX	TPGPMM chains (MCMC)	13.44 [11, 17]	0.090	3.56	—	—
Treasury	Exact, 210-point grid	7.04 [5, 10]	0.359	5.01	0.045	0.09
Treasury	Exact, 783-point grid	6.97 [5, 10]	0.367	5.17	0.030	0.03

Appendix

Table 12: TPGPMM’s chains against the exact grid sampler, which includes the truncation constant.

\mathrm{TV}_{K}

is the total-variation distance between the two posteriors of

K

, and

\max|\Delta\tau|

the largest difference between the probabilities of a boundary within

\pm 10

trading days of each date. Exact results use

4{,}000

independent draws.

Figure 8: Exact grid sampler against TPGPMM’s chains, for VIX (left) and the 10-year Treasury yield (right). Top : posterior of the number of regimes

K

. Bottom : posterior probability of a boundary within

\pm 10

trading days of each date. The exact results use the finest grid and

4{,}000

independent draws.

Time Series Modeling

See all 51 papers

May 19, 2026

DeRegiME: Deep Regime Mixtures for Probabilistic Forecasting under Distribution Shift

We introduce DeRegiME -- Deep Regime Mixture of Experts -- a direct multi-horizon probabilistic forecaster that separates latent uncertainty regimes from the underlying signal and softly assigns each forecast location to learned recurring regimes using a sparse variational Gaussian process (GP) whose nonstationary regime-mixing kernel and Student-t likelihood combine per-regime sub-kernels and noise processes via a shared gate. This yields a single sparse-GP posterior, not a mixture of GP experts. DeRegiME addresses a key limitation of neural forecasters: point forecasts discard residual uncertainty, and probabilistic heads -- whether single marginals, uninterpreted mixtures, quantile sets, or diffusion samples -- rarely expose the regime structure of the residual. Yet distribution shift in noisy heteroskedastic time series may be abrupt, gradual, or horizon-dependent and often appears in residual uncertainty rather than the conditional mean. DeRegiME yields an interpretable mean-residual-noise decomposition with a direct-sum feature-space representation that anchors regimes as clusters of residual similarity whose transitions surface as implicit changepoints. The effective number of regimes is pruned by the stick-breaking gate. We prove kernel validity and predictive-density propriety, and across ten benchmarks and three encoder grids DeRegiME improves negative log predictive density (NLPD) by 20.3% over the strongest encoder-matched baseline, a DeepAR/GluonTS-style dynamic Student-t head, with parallel gains on CRPS (3.0%) and MSE (4.7%). Improvements are consistent across all datasets, which span abrupt, gradual, and seasonal shifts.

Kieran Wood, Stefan Zohren, Stephen J. Roberts

Machine Learning Research Group, University of Oxford · Oxford-Man Institute, University of Oxford

Distribution Shift Robustness Time Series Forecasting Non-Stationary Time Series Forecasting Probabilistic Time Series Forecasting Temporal Distribution Shift Gaussian Process Regression

Ask PDF

Aug 22, 2026

Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

We propose a Bayesian nonparametric mixture of regression trees with a Dirichlet process prior over tree-parameter pairs, enabling data-driven selection of ensemble size and unifying CART, BART, random forests, and boosting. A novel splitting rule driven by the posterior predictive of a Gaussian process within each terminal node generates flexible, smooth decision boundaries; remarkably, the GP density cancels exactly in the Metropolis--Hastings ratio for GROW/PRUNE moves, ensuring computational feasibility. An exact Gibbs sampler for posterior predictive inference propagates uncertainty through random tree traversal. A parallel MPI implementation distributes independent tree updates across processors, achieving adequate speedups. We prove posterior consistency at rate

n^{-1/4}

in Hellinger distance under only continuity of the true regression function, allowing misspecification, via the identity

h(Θ)=0

. Simulations on Friedman benchmark show near-nominal coverage (0.94 Gaussian, 0.92 Cauchy), robust to high-dimensional noise and heavy tails, outperforming BART and bagged CART. Applications to QSAR toxicity, crime, riboflavin, wheat genomics, and air quality confirm reliable credible intervals and automatic sparsity. The DP mixture offers a principled, robust, theoretically justified alternative for challenging regression with honest uncertainty quantification.

Subhasish Basak, Anik Roy, Sourabh Bhattacharya

Indian Statistical Institute

Dirichlet Process Mixture Models Posterior Contraction Bayesian Inference Regression Prediction Intervals

Ask PDF

Apr 3, 2026

Learning Nonlinear Regime Transitions via Semi-Parametric State-Space Models

We develop a semi-parametric state-space model for time-series data with latent regime transitions. Classical Markov-switching models use fixed parametric transition functions, such as logistic or probit links, which restrict flexibility when transitions depend on nonlinear and context-dependent effects. We replace this assumption with learned functions

f_0, f_1 \in \calH

, where

\calH

is either a reproducing kernel Hilbert space or a spline approximation space, and define transition probabilities as

p_{jk,t} = \sigmoid(f(\bx_{t-1}))

. The transition functions are estimated jointly with emission parameters using a generalized Expectation-Maximization algorithm. The E-step uses the standard forward-backward recursion, while the M-step reduces to a penalized regression problem with weights from smoothed occupation measures. We establish identifiability conditions and provide a consistency argument for the resulting estimators. Experiments on synthetic data show improved recovery of nonlinear transition dynamics compared to parametric baselines. An empirical study on financial time series demonstrates improved regime classification and earlier detection of transition events.

Prakul Sunil Hiremath

Visvesvaraya Technological University, Belagavi, India · Aliens on Earth (AoE) Autonomous Research Group, Belagavi, India

Regime-Switching Dynamical Systems State Space Models Semiparametric Inference

Ask PDF

Abstract

Time Series Modeling

Explore similar work

DeRegiME: Deep Regime Mixtures for Probabilistic Forecasting under Distribution Shift

Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

Learning Nonlinear Regime Transitions via Semi-Parametric State-Space Models