Variational Inference for Bayesian MIDAS Regression

Luigi Simeone

cs.LGFeb 23, 2026

Variational Inference for Bayesian MIDAS Regression

Authors: Luigi Simeone

Organizations: Independent Researcher

Abstract

We develop a Coordinate Ascent Variational Inference (CAVI) algorithm for Bayesian Mixed Data Sampling (MIDAS) regression with linear weight parameterizations. The model separates impact coeffcients from weighting function parameters through a normalization constraint, creating a bilinear structure that renders generic Hamiltonian Monte Carlo samplers unreliable while preserving conditional conjugacy exploitable by CAVI. Each variational update admits a closed-form solution: Gaussian for regression coefficients and weight parameters, Inverse-Gamma for the error variance. The algorithm propagates uncertainty across blocks through second moments, distinguishing it from naive plug-in approximations. In a Monte Carlo study spanning 21 data-generating configurations with up to 50 predictors, CAVI produces posterior means nearly identical to a block Gibbs sampler benchmark while achieving speedups of 107x to 1,772x (Table 9). Generic automatic differentiation VI (ADVI), by contrast, produces bias 714 times larger while being orders of magnitude slower, confirming the value of model-specific derivations. Weight function parameters maintain excellent calibration (coverage above 92%) across all configurations. Impact coefficient credible intervals exhibit the underdispersion characteristic of mean-field approximations, with coverage declining from 89% to 55% as the number of predictors grows a documented trade-off between speed and interval calibration that structured variational methods can address. An empirical application to realized volatility forecasting on S&P 500 daily returns cofirms that CAVI and Gibbs sampling yield virtually identical point forecasts, with CAVI completing each monthly estimation in under 10 milliseconds.

Figures & tables

$J$	Method	Bias( $\beta$ )	RMSE( $\beta$ )	Cov95( $\beta$ )	Bias( $\eta$ )	Cov95( $\eta$ )	Time (s)	Speedup	ESS
1	CAVI	0.029 (.007)	0.158 (.004)	0.894 (.014)	0.001 (.001)	0.929 (.011)	0.001	1,772 $\times$	—
1	Gibbs	0.032 (.007)	0.159 (.004)	0.934 (.011)	0.002 (.001)	0.936 (.011)	1.82	—	4,103
3	CAVI	0.056 (.004)	0.173 (.003)	0.836 (.017)	0.005 (.002)	0.945 (.010)	0.006	645 $\times$	—
3	Gibbs	0.078 (.004)	0.187 (.003)	0.903 (.013)	0.008 (.002)	0.957 (.009)	3.59	—	1,066
5	CAVI	0.087 (.003)	0.177 (.002)	0.594 (.022)	0.030 (.003)	0.953 (.009)	0.023	283 $\times$	—
5	Gibbs	0.103 (.003)	0.181 (.002)	0.846 (.016)	0.032 (.002)	0.974 (.007)	6.54	—	701

Table 1: CAVI versus Gibbs across number of predictors

J

. Metrics averaged over active

\beta_{j}

, 500 replications (50 for

J=25

Gibbs). Standard errors in parentheses.

T=200

K=9

P=3

Figure 1: Computational speedup of CAVI over Gibbs sampler as a function of the number of predictors $J$ . Solid markers: measured. Open markers: extrapolated from Gibbs scaling pattern. Even at $J=50$ , CAVI maintains a speedup of approximately $53\times$ (extrapolated).

Figure 2: Gibbs sampler minimum effective sample size (ESS) as a function of $J$ . The decline from 4,103 ( $J=1$ ) to 539 ( $J=10$ ) and further to 134 ( $J=25$ , 50 replications) indicates increasing autocorrelation in the MCMC chain as bilinear coupling involves more blocks.

$T$	Method	Bias( $\beta$ )	RMSE( $\beta$ )	Cov95( $\beta$ )	Bias( $\eta$ )	Cov95( $\eta$ )	Time (s)	Speedup	ESS
50	CAVI	0.216 (.009)	0.425 (.007)	0.584 (.022)	0.023 (.003)	0.931 (.011)	0.015	303 $\times$	—
50	Gibbs	0.278 (.009)	0.444 (.007)	0.807 (.018)	0.034 (.002)	0.977 (.007)	4.47	—	681
100	CAVI	0.130 (.006)	0.274 (.004)	0.729 (.020)	0.010 (.003)	0.950 (.010)	0.011	416 $\times$	—
100	Gibbs	0.170 (.006)	0.297 (.004)	0.860 (.016)	0.009 (.002)	0.975 (.007)	4.39	—	794
200	CAVI	0.056 (.004)	0.173 (.003)	0.836 (.017)	0.005 (.002)	0.945 (.010)	0.006	645 $\times$	—
200	Gibbs	0.078 (.004)	0.187 (.003)	0.903 (.013)	0.008 (.002)	0.957 (.009)	3.59	—	1,066

Table 2: CAVI versus Gibbs across sample size

T

. 500 replications. Standard errors in parentheses.

J=3

K=9

P=3

Figure 3: Empirical 95% coverage of active $\beta_{j}$ parameters as a function of sample size $T$ , with $J=3$ . Error bars show $\pm 1$ standard error across replications. Both methods improve with $T$ . The CAVI–Gibbs gap narrows from approximately 10 percentage points at $T=100$ to 6 percentage points at $T=400$ . The $T=50$ CAVI point (58.4%, Table 2 ) falls below the displayed range.

Figure 4: Weight profile recovery for a representative replication (

J=3

T=200

). Left: decreasing profile (

\beta_{1}=2.0

). Center: hump-shaped profile (

\beta_{2}=-1.0

). Right: decreasing profile (

\beta_{3}=0.5

). CAVI (blue) and Gibbs (red) estimates are nearly indistinguishable from the true profile (black) for strong predictors.

$J$	Method	Bias( $\beta$ )	RMSE( $\beta$ )	Time (s)	Speedup vs ADVI-MF
1	CAVI	0.128	0.235	0.000	$>$ 100,000 $\times$
1	ADVI-MF	1.851	1.851	136.0	—
1	ADVI-FR	1.972	1.972	137.3	—
3	CAVI	0.128	0.157	0.009	24,644 $\times$
3	ADVI-MF	1.381	1.457	221.8	—
3	ADVI-FR	1.467	1.547	302.0	—

Table 3: CAVI versus generic ADVI (mean-field and full-rank). 5 replications per configuration.

T=200

K=9

P=3

Figure 5: S&P 500 monthly realized volatility (2000–2025). Top: raw $RV_{t}$ . Bottom: $\log RV_{t}$ . Dashed line: start of out-of-sample period. Prominent spikes correspond to the 2008 financial crisis and the 2020 COVID-19 shock.

Model	MSE	MAE	Rel. MSE vs HAR	DM $p$ -value	Time/month
MIDAS $J\!=\!1$ (CAVI)	0.720	0.651	1.058	0.226	0.001 s
MIDAS $J\!=\!1$ (Gibbs)	0.720	0.651	1.058	0.224	1.13 s
MIDAS $J\!=\!3$ (CAVI)	0.688	0.633	1.011	0.864	0.009 s
MIDAS $J\!=\!3$ (Gibbs)	0.702	0.635	1.031	0.633	2.48 s
HAR-RV	0.680	0.645	1.000	—	$<$ 0.001 s
AR(1)	0.713	0.657	1.048	0.045	$<$ 0.001 s

Table 4: Out-of-sample forecast accuracy (187 months). MSE and MAE on

\log RV_{t}

Figure 6: Out-of-sample forecasts of $\log RV_{t}$ (2010–2025). Black: actual. Blue: MIDAS $J=1$ (CAVI). Gray dashed: HAR-RV. Both models track the actual series closely, with large forecast errors concentrated around the March 2020 COVID-19 volatility spike.

Figure 7: Estimated MIDAS weight profiles (

J=1

, full sample). Blue: CAVI posterior mean. Red: Gibbs posterior mean. Shaded regions: 95% credible bands. Dashed green: uniform weights. Point estimates are virtually identical. The monotonically decreasing profile assigns greater weight to recent daily returns, consistent with the intuition that recent squared returns are more informative about current-month volatility.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Profile	Method	Bias( $\beta$ )	Cov95( $\beta$ )	Bias( $\eta$ )	Cov95( $\eta$ )	ESS
Decreasing	CAVI	0.051	0.749	0.012	0.921	—
Decreasing	Gibbs	0.077	0.900	0.023	0.946	643
Hump-shaped	CAVI	0.057	0.819	0.010	0.942	—
Hump-shaped	Gibbs	0.079	0.899	0.019	0.961	1,034
U-shaped	CAVI	0.064	0.827	0.006	0.962	—
U-shaped	Gibbs	0.086	0.891	0.009	0.970	1,108

Appendix

Table 5: Robustness results (Tier 2, Panel A): Weight profile shapes.

T=200

J=3

P=3

Variation	Method	Bias( $\beta$ )	Cov95( $\beta$ )	Bias( $\eta$ )	Cov95( $\eta$ )	ESS
$\sigma_{0}^{2}=0.25$ (high SNR)	CAVI	0.010	0.899	0.001	0.948	—
	Gibbs	0.012	0.953	0.002	0.959	3,280
$\sigma_{0}^{2}=4$ (low SNR)	CAVI	0.112	0.747	0.011	0.943	—
	Gibbs	0.148	0.861	0.011	0.966	883
$K=5$ (quarterly)	CAVI	0.079	0.793	0.010	0.936	—
	Gibbs	0.112	0.867	0.009	0.957	696

Appendix

Table 6: Robustness (Panel B): Signal-to-noise ratio and frequency ratio.

$T$	$J$	Method	Bias( $\beta$ )	RMSE( $\beta$ )	Cov95( $\beta$ )	Bias( $\eta$ )	Cov95( $\eta$ )	ESS
50	10	CAVI	0.240	0.383	0.374	0.068	0.957	—
50	10	Gibbs	0.266	0.382	0.842	0.059	0.993	366
200	10	CAVI	0.156	0.273	0.441	0.026	0.955	—
200	10	Gibbs	0.203	0.293	0.708	0.016	0.981	339
50	3	CAVI	0.350	0.573	0.455	0.038	0.933	—
50	3	Gibbs	0.433	0.599	0.783	0.048	0.980	646

Appendix

Table 7: Stress test results (Tier 3).

$\sigma_{\beta}^{2}$	Bias( $\beta$ )	RMSE( $\beta$ )	Cov95( $\beta$ )	Bias( $\eta$ )	Cov95( $\eta$ )
5	0.128 (.007)	0.140 (.008)	0.875 (.033)	0.021	0.932
10	0.128 (.007)	0.140 (.008)	0.880 (.032)	0.021	0.932
20	0.127 (.007)	0.140 (.008)	0.880 (.032)	0.021	0.932

Appendix

Table 8: Prior sensitivity: CAVI performance under different

\sigma_{\beta}^{2}

J=3

T=200

, 100 replications.

Config	CAVI (s)	Iters	Gibbs (s)	Speedup	Min ESS
1A-1 ( $J\!=\!1$ )	0.001	3	1.82	1,772 $\times$	4,103
1A-2 ( $J\!=\!3$ )	0.006	10	3.59	645 $\times$	1,066
1A-3 ( $J\!=\!5$ )	0.023	29	6.54	283 $\times$	701
1A-4 ( $J\!=\!10$ )	0.066	37	15.56	238 $\times$	539
1A-5 ( $J\!=\!25$ )	0.322	49	34.44	107 $\times$	134
1A-6 ( $J\!=\!50$ )	1.383	67	—	—	—

Appendix

Table 9: Computational timing across Monte Carlo configurations. CAVI and Gibbs times are per-fit averages (500 replications except

J=25

Gibbs: 50 replications).

Config	Mean	Std	Min	Max
1A-1 ( $J\!=\!1$ )	$-$ 235.0	10.1	$-$ 264.9	$-$ 210.6
1A-2 ( $J\!=\!3$ )	$-$ 253.5	9.5	$-$ 285.0	$-$ 220.1
1A-3 ( $J\!=\!5$ )	$-$ 263.7	9.5	$-$ 290.0	$-$ 228.5
1A-4 ( $J\!=\!10$ )	$-$ 291.3	9.8	$-$ 318.3	$-$ 263.9
1A-5 ( $J\!=\!25$ )	$-$ 370.9	8.2	$-$ 390.8	$-$ 338.2
1A-6 ( $J\!=\!50$ )	$-$ 503.5	7.7	$-$ 527.3	$-$ 480.0

Appendix

Table 10: Final ELBO values across Monte Carlo configurations (500 replications).

Figure 8: Computational scaling: mean estimation time per fit as a function of $J$ (log-log scale). Blue: CAVI. Red: Gibbs (solid: measured; open: extrapolated). The gap widens with $J$ due to Gibbs mixing degradation.

Figure 9: Estimated MIDAS weight profiles (

J=3

, full sample). Left: Predictor 1 (month

t

) shows a monotonically decreasing profile with tight credible bands, consistent with the

J=1

result (Figure 7 ). Center and Right: Predictors 2 and 3 (months

t{-}1

and

t{-}2

) have weights concentrated near zero with wide credible bands, indicating negligible marginal contribution beyond the current month’s squared returns. CAVI (blue) and Gibbs (red) point estimates are virtually identical across all three predictors.

Figure 10: Cumulative out-of-sample squared forecast error (2010–2025). The historical average (pink) is clearly dominated. All other models cluster closely, with a sharp increase around March 2020 (COVID-19). MIDAS $J=3$ (CAVI) tracks HAR-RV most closely.

Figure 11: MIDAS-RV ( $J=1$ , CAVI) out-of-sample forecasts with approximate 95% predictive intervals. The March 2020 COVID-19 spike falls outside the interval, as expected for an extreme tail event.

Explore similar work

Jun 24, 2026

Gaussian Mean Field Variational Inference can Overestimate Predictive Variance

Mean Field Variational Inference (MFVI) is widely understood to underestimate posterior variance. By analysing conjugate Bayesian Linear Regression (BLR), we show that this characterization is incomplete: while MFVI underestimates the variance in parameter space, it can overestimate the predictive variance compared to the exact posterior. We show that if the MFVI posterior underestimates predictive variances in some directions, it necessarily overestimates them in others. Crucially, this overestimation occurs in directions where the training data concentrates. This leads to the surprising result that, for a test point drawn from the training distribution, MFVI's expected predictive variance exceeds that of the exact posterior. We demonstrate a pathological case of this effect, where the MFVI posterior fails to reduce predictive variance compared to the prior on in distribution data. We connect these results to the Cold Posterior Effect, arguing that varying the temperature can correct this overestimation, yielding predictions closer to those of the exact posterior. We validate our theory on synthetic and real-world regression tasks.

James Odgers, Ben Riegler, Siddharth Swaroop +1

University of Technology Nuremberg (UTN) · Helmholtz AI · Munich Center for Machine Learning (MCML) +2

Probabilistic Forecasting Variational Inference

Ask PDF

Oct 18, 2024

Predictive variational inference: Learn the predictively optimal posterior distribution

Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.

Jinlin Lai, Antonio Linero, Yuling Yao

College of Information and Computer Sciences, University of Massachusetts Amherst · Department of Statistics and Data Sciences, University of Texas, Austin

Model Misspecification Variational Inference

Ask PDF

Oct 6, 2026

Generalized Matheron Variational Implicit Processes

Implicit-process priors specify distributions over functions through sample-forward mechanisms such as Bayesian neural networks and stochastic simulators, but their function-space densities are typically unavailable. We introduce Generalized Matheron Variational Implicit Processes (GMVIP), a pathwise variational family for posterior inference with such priors. For Gaussian-process priors, GMVIP recovers the standard inducing-variable variational GP construction; for general implicit priors, its empirical covariance construction preserves the prior mean and covariance in the population limit. GMVIP constructs posterior samples by drawing a function from the prior and applying a correction anchored at a set of inducing inputs. The effect of this correction away from the inducing inputs is determined directly from prior samples, allowing the posterior to retain the structure and variability of the original implicit process. The (surrogate) prior and variational posterior use the same pathwise construction and differ only in the distribution of whitened inducing coefficients, yielding a tractable coefficient-space Kullback-Leibler divergence. Experiments on regression, classification, and forecasting with simulator-defined and retrieval-conditioned empirical trajectory priors show that GMVIP is broadly competitive with existing methods.

Luis A. Ortega, Andrés R. Masegosa, Thomas D. Nielsen

Aalborg University

Bayesian Inference Gaussian Processes Variational Inference

Ask PDF