Finite-Cadence Refresh Equilibria for Streaming Prior-Anchored Ridge: A Closed-Form Forgetting Window and Spectral Correction
Organizations: DLDAV Model School, Shalimar Bagh Delhi 110088, India
Abstract
Recursive estimators on drifting streams must forget stale data fast enough to track the target, yet slowly enough to average out noise. In prior-anchored ridge regression, this trade-off admits a closed-form spectral decomposition in which each eigen-direction interpolates between the data solution and the prior with pull weight . When the anchor is refreshed every steps, we show that the moving-frame offset converges to a finite-cadence equilibrium that lags the pure-leaky path by an explicit while reducing estimation variance by a factor . Minimizing the cycle-averaged risk gives , , and the plug-in strength . A random-matrix deterministic equivalent extends both laws to finite dimension, renormalizing the anchor to ; it matches simulation within 5% at fixed operating points and within 13% at the method's own, and rank-deficient populations reduce exactly to their active subspace. The resulting method, SPAR, calibrates window and strength online from residuals. On a 27-configuration grid the calibrated window opens on drift without divergence, and the grid defaults transfer to an hourly electricity stream and to BCI IV 2b. The core algebra, refresh chain, and regret bounds are formalized in Lean 4.
Figures & tables
| family | anchor | per-dir. pull | refresh | online window/strength | finite-window DE | Lean |
|---|---|---|---|---|---|---|
| leaky / EW-RLS ( Sayed, 2008 ; Xiao and Honig, 2005 ) | – | – | – | – | – | |
| VFF-RLS ( Fortescue et al., 1981 ; Paleologu et al., 2008 ) | – | – | – | window | – | – |
| directional forgetting ( Kulhavý and Kárný, 1984 ; Cao and Schwartz, 2000 ; Lai and Bernstein, 2024 ) | – | subspace | – | – | – | – |
| AROW ( Crammer et al., 2009 ) | confidence | per-feature | – | – | – | – |
| Kalman / state space ( Kalman, 1960 ; Sayed and Kailath, 1994 ) | process | – | – | fixed | – | – |
| WiSE-FT ( Wortsman et al., 2022 ) | frozen model | – | – | – | – | – |
| notation | fixed constants | ||
|---|---|---|---|
| input dimension | refresh cadence | ||
| warm-up, stream length | fast residual EMA | ||
| sufficient statistics | slow residual EMA | ||
| mean, running accumulator | EMA horizon | ||
| excitation eigenpairs | strength clip floor | ||
| pull weight | strength clip cap | ||
| method | mean MSE | geo. mean | worst MSE |
|---|---|---|---|
| baselines (single hyperparameter, tuned post hoc on the evaluation grid) | |||
| leaky RLS ( ) | |||
| leaky RLS, information form ( ) | |||
| SGD ( ) | |||
| Adam ( Kingma and Ba, 2015 ) ( ) | |||
| directional forgetting | |||
| dimension | assumption or protocol | consequence outside it |
|---|---|---|
| design model | Theorems 2 – 3 use the mean-field accumulator; Theorem 10 retains the random design | at drift-phase the exact random-design variance replaces the mean-field term in the cycle-averaged risk; the runtime keeps the conservative mean-field plug-in |
| spectrum | Theorem 10 assumes the nonzero part is bounded away from zero, ; rank-deficient spectra reduce exactly to (Corollary 19 ) | null directions drop out of the prediction risk, so the deterministic equivalent restricts cleanly to with effective aspect ratio |
| contraction | Theorem 11 contracts globally on the parameter box , with the clip a nonexpansive projection (Corollary 20 ) | outside the box the gate and the measured-product hypothesis apply; the basin is the whole box |
| moving targets | fixed-protocol grid, 27 configurations | unanchored information-form leaky RLS wins moving-target configurations, but all are small-margin (isotropic , rank-deficient tie); SPAR wins the ill-conditioned ones by and the moving-target mean by , and all static-target covariance-rotation configurations |
| strength rule | onset guard; average effect measured by ablation | freezing it lowers mean MSE by ; its value is the transient guard, not a continuous tracking gain |
| detector timescale | two-scale residual EMA, , ; a scale-free sticky floor (frozen while an excess is present) handles sustained drift | the fixed-horizon detector under-opens once a drift persists beyond , while the sticky-floor variant keeps the window open and halves the sustained-drift tail MSE |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| paper result | content | Lean declaration(s) | file |
|---|---|---|---|
| Equation ( 1 ) | pull identity ( and general) | pull_identity , pull_identity_general | Core.lean |
| Equation ( 1 ) (eigenbasis) | diagonal decoupling; coordinatewise anchored solve | eigenbasis_solve_coord , eigenbasis_pull_identity | Core.lean |
| Proposition 5 | risk interpolation; dominance; calibration identity; optimal pull; multiplicative inversion at the optimum | convex_interp_sq , convex_interp_sq_le , dominance , dominance_strict , quadratic_calibration , quadratic_calibration_delta , optimal_pull_min , optimal_pull_attained , pull_inverse | Core.lean |
| Proposition 6 | minimax value (core inequality); safety excess identity; multiplicative safety threshold | minimax_lower , quadratic_calibration , safety_threshold | Core.lean |
| Lemma 1 | lag-invariance algebra | lag_invariance | Core.lean |
| Theorem 2 | refresh rate identity; finite-cadence fixed point; geometric closed form; contraction; correlated variance identities and the lower endpoint | refresh_rate_identity , refresh_fixed_point , refresh_closed_form , refresh_contraction_rate , refresh_anchor_variance , refresh_variance_form , refresh_variance_lt , refresh_variance_lb | Core.lean |
| method | mean MSE | worst MSE |
|---|---|---|
| Kalman, true noise ( ) | ||
| SPAR-EG + | ||
| SPAR-EG | ||
| leaky RLS, information form ( ) | ||
| SPAR-Pull | ||
| SGD ( ) |
| axis | value | SPAR-Pull | SPAR-EG + | KF-slow | VFF | leaky-info |
|---|---|---|---|---|---|---|
| spectrum | isotropic | |||||
| decay-3 | ||||||
| rank-deficient | ||||||
| drift | abrupt | |||||
| covariance rotation | ||||||
| both |
| ( ) | standard recursion | symmetrized | |
|---|---|---|---|
| ( ) | 128 | ||
| ( ) | 128 | ||
| ( ) | 128 | ||
| ( ) | 512 |
| variant | warm overall | adversarial overall | adversarial early |
|---|---|---|---|
| full | |||
| fixed (refresh on) | |||
| frozen anchor (no refresh) | |||
| no variance floor |
| strength policy | pre-drift MSE | post-jump MSE | overall MSE | |
|---|---|---|---|---|
| 10 | adaptive | |||
| 10 | fixed | |||
| 10 | fixed | |||
| 10 | fixed | |||
| 50 | adaptive | |||
| 50 | fixed |
| method | ms/step at |
|---|---|
| leaky RLS | |
| leaky RLS, information form | |
| AROW | |
| Kalman | |
| directional forgetting | |
| SPAR-Pull-D (cached basis, /step) |
| method | full horizon | final h |
|---|---|---|
| leaky RLS, information form ( ) | ||
| leaky RLS ( ) | ||
| leaky RLS ( ) | ||
| Kalman, slow estimated ( ) | ||
| Kalman, slow estimated ( ) | ||
| SPAR-EG + (grid defaults) |
| held-out client | best oracle leaky | SPAR-EG + | SPAR-Pull |
|---|---|---|---|
| 25th percentile | |||
| 50th percentile | |||
| 75th percentile | |||
| three-client mean |
| method | BCI IV 2a | BCI IV 2b |
|---|---|---|
| classical (canonical implementations) | ||
| MDM (pyriemann) | ||
| FgMDM (pyriemann) | ||
| LDA (raw features) | ||
| LDA (raw) + session EA T | ||
| CSP-LDA (mne) | ||
| parameter | grid default | EEG head |
|---|---|---|
| refresh cadence | trials | |
| fast-EMA horizon | ||
| warm-up anchor strength | ||
| stationary floor | ||
| drift-active floor | — | |
| anchor-strength cap |
| stage | driver | output |
|---|---|---|
| verify | verify_spar_theorems.py | symbolic identities |
| factorial | synthetic_factorial.py | factorial_d_*.csv |
| sparkf | SPAR-Pull append | factorial_extras.csv |
| append | append_arms.py , regen_spar_eg_plus.py , revision_runs.py --suite grid | factorial_pu_refresh.csv |
| aggregate | aggregate_factorial.py | Table 3 |
| ablation | revision_runs.py --suite ablation | ablation_spar_pull.csv |
Ridge Regression
Ridge regression uses an L2 penalty to stabilize least-squares prediction, especially with many, correlated, or noisy features.
See all 36 papers