Conditioning on a Volatility Proxy Compresses the Apparent Timescale of Collective Market Correlation
Authors: Yuda Bi, Vince D Calhoun
Organizations: Center for Translational Research in Neuroimaging and Data Science (TReNDS), Atlanta, Georgia 30303, USA · Georgia State University, Georgia Institute of Technology, and Emory University, Atlanta, Georgia 30303, USA
We address the attribution problem for apparent slow collective dynamics: is the observed persistence intrinsic, or inherited from a persistent driver? For the leading eigenvalue fraction ψ1=λmax/N of S&P 500 60-day rolling correlation matrices (237 stocks, 2004--2023), a VIX-coupled Ornstein--Uhlenbeck model reduces the effective relaxation time from 298 to 61 trading days and improves the fit over bare mean reversion by ΔBIC=109. On the decomposition sample, an informational residual of log(VIX) alone retains most of that gain (ΔBIC=78.6), whereas a mechanical VIX proxy alone does not improve the fit. Autocorrelation-matched placebo fields fail (ΔBICmax=2.7), disjoint weekly reconstructions still favor the field-coupled model (ΔBIC=140--151), and six anchored chronological holdouts preserve the out-of-sample advantage. Quiet-regime and field-stripped residual autocorrelation controls show the same collapse of persistence. Stronger hidden-variable extensions remain only partially supported. Within the tested stochastic class, conditioning on the observed VIX proxy absorbs most of the apparent slow dynamics.
Figures & tables
Rival explanation
Primary discrimination
Outcome in this manuscript
Bare intrinsic persistence or autocorrelation-only inheritance
M0 versus M2, placebo surrogates, quiet-regime ACF collapse, field-stripped residual ACF, disjoint weekly reconstructions, and anchored holdouts
Strongly disfavored
Purely mechanical overlap between VIX and ψ1
Mechanical/informational decomposition, standalone field tests, and recipe sensitivity across freeze and weighting choices
Strongly disfavored dynamically
Observed VIX as the exact hidden coordinate behind projected memory
Exact two-dimensional linear-Gaussian comparison and projected-kernel timescale check
Only partially supported
VIX-orthogonal residual as an operational predictor
Q2-versus-Q3 future-VIX test across 30-, 60-, and 90-day horizons
Not supported
Table 1: Layered attribution logic. The strongest claim is the one that survives the largest stack of discrimination tests.
Figure 1: Collective correlation tracks a moving volatility field. (a) The long-run ψ1 series co-moves strongly with VIX over 2004–2023, with the largest excursions concentrated in major crisis episodes. (b) The contemporaneous association is strong but nondegenerate, with r=0.724 against log(VIX) and r=0.681 against level VIX. (c) Equal-count VIX bins show that the fitted conditional equilibrium μeff(VIX)=μ+(β/θ)log(VIX) tracks the empirical shift of the ψ1 distribution across volatility conditions. Together these panels motivate the attribution problem addressed below.
Model
BIC
Δ BIC vs M2
M_RS,c+VIX
-33011.9
-765.4
M_RS,c
-32274.4
-27.9
M2’
-32255.3
-8.7
M2
-32246.5
0.0
M3
-32230.0
+16.5
M0
-32137.5
+109.0
Table 2: Core one-dimensional and hybrid stochastic models on the aligned daily sample.
Figure 2: The main attribution result. (a) Same-sample model comparison across the core, hybrid, and auxiliary-field variants centers the main interpretive story on the VIX-coupled OU model: bare OU and quartic alternatives are strongly disfavored, while rare-event hybrids are better read as an added extreme-state channel than as a replacement for the continuous field picture. (b) Under the baseline W=60 construction, conditioning on VIX reduces the relaxation time from 298 to 61 trading days. (c) One hundred AR-matched placebo fields fail to approach the real-VIX gain (maximum placebo Δ BIC =2.7 versus 109 for real VIX). (d) Level-based Granger tests show dominant VIX →ψ1 directionality ( F=15.1 versus 1.6 in the reverse direction). These panels together establish the conditional field-proxy attribution that anchors the rest of the manuscript.
Figure 3: Independent robustness checks. (a) Persistence collapses both in quiet regimes and after subtracting the fitted VIX-dependent equilibrium: the full-sample e-folding lag is 69 trading days, the field-stripped residual falls to 42 days, and the quiet-regime estimate falls to 27 days. (b) Among auxiliary fields, VIX remains uniquely strong, while MOVE is secondary and TED does not improve on the bare benchmark. (c) On the decomposition sample, the informational residual of log(VIX) alone retains most of the model-selection gain ( Δ BIC =78.6 ), whereas the mechanical proxy alone fails to improve the fit ( Δ BIC =−8.3 ); the same sample still yields a 77.3%/22.7% mechanical/informational variance split. (d) Exact recomputation across W=30 – 120 shows that absolute timescales depend on rolling-window resolution, but the field susceptibility β/θ stays in the narrow range 0.336 – 0.376 and the attribution fraction remains majority-inherited at every window. Together these panels show that the main claim is not an artifact of placebo persistence, auxiliary fields, mechanical overlap, or the baseline window choice.
Figure 4: Extensions beyond the main one-dimensional result. (a) On daily aligned data the 2D comparison weakly favors feedforward over bidirectional drift by Δ BIC =2.44 , whereas the Wand-faithful weekly reconstructions weakly favor bidirectionality ( Δ BIC =−0.78 for weekly ψ1 and −2.69 for weekly mean correlation; negative values favor bidirectional coupling). The projected-kernel timescales remain in the 9 – 36 day range and therefore do not recover the 15 -day Wand benchmark in any clean way. (b) The orthogonal residual phase space in (logVIX,ϵ⊥) is structured and visually nontrivial. (c) The proposed Q2-versus-Q3 prediction test nevertheless remains null across all tested horizons ( p=0.183 , 0.736 , and 0.986 at 30, 60, and 90 trading days). The figure therefore frames these sections as constrained extensions rather than as new headline results.
Horizon
Mean Q2
Mean Q3
Mann–Whitney p
Rank-biserial
30 days
0.078
0.083
0.183
0.021
60 days
0.082
0.135
0.736
-0.015
90 days
0.110
0.141
0.986
-0.052
Table 3: Future VIX changes after low-VIX days split by the sign of the orthogonal residual. Positive rank-biserial values would favor the proposed Q2 signal.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
W
θ0
τ0
θ
τcond
β
β/θ
SCPA
Δ BIC
30
0.00955
104.67
0.02927
34.17
0.00982
0.336
0.674
104.0
45
0.00553
180.79
0.02093
47.78
0.00706
0.337
0.736
99.4
60
0.00335
298.10
0.01640
60.99
0.00572
0.349
0.795
109.0
90
0.00194
516.68
0.01317
75.93
0.00488
0.370
0.853
153.6
120
0.00119
838.72
0.01036
96.55
0.00390
0.376
0.885
165.4
Appendix
Table 4: Exact window-size robustness of M0 and M2, recomputing ψ1 directly from raw returns at each window. SCPA denotes 1−τcond/τauto .
Field
θ
τcond
β
Δ BIC vs M0 (decomp. sample)
Actual log(VIX)
0.01723
58.05
0.00623
108.6
Mechanical-only proxy
0.00393
254.18
0.00026
-8.3
Informational residual
0.00850
117.67
0.00471
78.6
Appendix
Table 5: M2 fits with actual, mechanical-only, and informational-only fields on the decomposition sample.
Recipe
Mechanical fraction
Informational fraction
Δ BIC mech-only
Δ BIC info-only
Median freeze, equal weights
0.773
0.227
-8.35
78.59
Mean freeze, equal weights
0.776
0.224
-8.36
78.73
Pre-2016 median freeze, equal weights
0.779
0.221
-8.34
78.62
Median freeze, inverse-volatility weights
0.815
0.185
-8.27
77.71
Median freeze, volatility-share weights
0.721
0.279
-8.38
79.57
Appendix
Table 6: Sensitivity of the mechanical/informational decomposition to volatility-freezing and weighting choices. The static split is recipe dependent, but the standalone dynamic conclusion is invariant across all tested variants.
Construction
nobs
Δ BIC M2 vs M0
τ0 (trading days)
τcond (trading days)
SCPA
Daily overlapping baseline
4973
109.0
298.10
60.99
0.795
Disjoint weekly ψ1
976
151.2
8.24
5.88
0.286
Disjoint weekly mean correlation
976
140.1
8.43
6.05
0.282
Disjoint 60-day blocks, end-of-block VIX
77
1.7
89.48
64.69
0.277
Disjoint 60-day blocks, block-mean VIX
77
-3.4
89.48
75.05
0.161
Appendix
Table 7: One-dimensional non-overlapping robustness checks. The disjoint weekly reconstructions preserve strong support for M2, whereas same-horizon 60-day block reconstructions are retained only as low-power boundary checks.
Split date
ntrain
ntest
M0 test ll/obs
M2 test ll/obs
M2 − M0 test gap
M2 test/train ratio
2010-01-01
1451
3522
3.1931
3.2000
0.0070
0.964
2012-01-01
1955
3018
3.2227
3.2373
0.0146
0.994
2014-01-01
2457
2516
3.2294
3.2437
0.0143
0.999
2016-01-01
2961
2012
3.2219
3.2346
0.0128
0.995
2018-01-01
3464
1509
3.1653
3.1805
0.0152
0.972
2020-01-01
3967
1006
3.2301
3.2324
0.0023
0.995
Appendix
Table 8: Anchored chronological out-of-sample sweep. For each split date, M0 and M2 are fit on the pre-split prefix and evaluated on the full held-out suffix.
Rolling-median VIX band
Segments ≥120 d
Quiet e-folding lag
Full-sample e-folding lag
[13,21]
5
27
69
[14,20]
3
27
69
[15,19]
0
—
69
Appendix
Table 9: Quiet-regime band sensitivity. The strict daily band [15,18] with a 120-trading-day minimum yields zero qualifying segments over 2004–2023. For the baseline rolling-median band [14,20] , the quiet e-folding estimate of 27 days carries an episode-bootstrap 95% confidence interval of [18,33] days.
Parameter or derived quantity
Estimate
θ
0.02348
μcalm
-0.6021
μstress
1.0000
β
0.00781
σ
0.00818
pcalm→stress
0.01170
Appendix
Table 10: Parameters of the constrained regime-switching plus VIX model M_RS,c+VIX on the aligned daily sample.
Dataset
Winning structure
Δ BIC vs next-best
Δ BIC vs decoupled
Kernel timescale (days)
Daily aligned ψ1
Feedforward
2.44
125.26
36.46 (full model only)
Naive weekly thinning
Feedforward
4.17
74.45
9.10 (full model only)
Wand-faithful weekly ψ1
Bidirectional
0.78
154.10
33.47
Wand-faithful weekly mean correlation
Bidirectional
2.69
143.87
36.46
Appendix
Table 11: Exact two-dimensional model comparisons and projected-kernel timescales across the daily and Wand-faithful datasets. The reported Δ BIC values are relative to the winning model in each panel.
Hybrid statistical-neural pipelines often assume that a successful statistical first stage leaves a cleaner and more learnable residual target. We examine that assumption in short-horizon volatility forecasting through a signal-forecast-system diagnostic framework. Across five liquid U.S. assets, a volatility-aligned HAR-style model outperforms AR, MA, and ARIMA. Within expanding training windows, the pre-standardization fitted residual process used to construct residual-LSTM sequences has about 82% lower variance than the corresponding target and near-zero lag-1 autocorrelation; independently, rolling pseudo-out-of-sample HAR errors show about 74% variance reduction and similarly weak lag-1 dependence. Residual-only LSTM augmentation nevertheless raises mean squared error from 0.3049 to 0.3594 on average, with deterioration on every asset. Pure LSTM records the lowest selected pseudo-out-of-sample MSE, 0.2649, while the residual hybrid requires substantially more end-to-end runtime without improving accuracy. We describe this pattern as forecaster-preconditioner asymmetry: first-stage forecasting success and statistical residual simplification need not translate into useful downstream neural preconditioning.
Bingqi Lian, Linfeng Cheng, Mei Lu +1
Science Academy, College of Computer, Mathematical, and Natural Sciences, University of Maryland, College Park, MD 20742, USA · Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA
Latent state-space models are widely used to study partially observed dynamical systems, yet most formulations assume that process variability is independent of latent-state position. In many biological, behavioral, and physiological systems, however, variability may depend systematically on the underlying dynamical state, producing structured stochasticity that is not captured by constant-variance models. We introduce a state-coupled stochastic volatility framework in which latent process variance depends on displacement from a latent equilibrium. To estimate this relationship under partial observation, we develop a particle expectation-maximization procedure combining bootstrap particle filtering and backward trajectory smoothing. The model includes a coupling parameter, γ, that quantifies the strength of association between latent-state position and process variability. A large-scale simulation benchmark evaluated recovery and detection performance across varying coupling strengths, observation noise levels, trajectory lengths, and persistence regimes. The proposed framework consistently reduced recovery bias relative to an observed-state heteroskedastic proxy, with the largest improvements occurring under strong coupling. Recovery performance improved with increasing latent persistence, while detection performance remained competitive across a broad range of conditions and became increasingly advantageous as observation noise increased. Taken together, the results demonstrate that state-coupled volatility can be identified and estimated under partial observation when latent-state structure is explicitly modeled. The framework provides a practical methodological foundation for studying state-dependent variability and evaluating whether structured stochasticity contributes information about system dynamics beyond that contained in mean-state trajectories alone.
Imani Beckett
The Herbert Wertheim School of Public Health and Human Longevity Science University of California San Diego
Time-series foundation models are increasingly adapted to new domains through fine-tuning on target data, under the implicit assumption that more target data yields better forecasts. We show that this assumption can fail in financial forecasting, where individual price changes are difficult to predict, but large moves tend to cluster, creating alternating calm and turbulent periods. Using financial foundation models trained on price bars of open, high, low, close, and volume, we argue that adapting to financial domains requires training signals beyond next-token prediction. We introduce Volatility-Clustering Adaptation (VCA), which augments next-token cross-entropy with a differentiable penalty on the autocorrelation of squared returns, the standard statistical signature of volatility clustering. This additional objective provides a multi-step training signal by matching the resulting dependence structure of autoregressive rollouts to those of the realized future. Across three asset sets and two evaluation conventions, VCA improves adaptation over the pre-trained model, with the strongest gains under the primary evaluation (fore), driven primarily by reduced variance error. Overall, our results suggest that effective financial adaptation requires objectives that capture domain-specific temporal structure beyond token-level prediction.
Manh Nguyen, Minh Hoang Nguyen, Huu Hiep Nguyen +2
Applied Artificial Intelligence Initiative, Deakin University, Australia