K-GMRF: Kinetic Gauss-Markov Random Field for First-Principles Covariance Tracking on Lie Groups

ZhiMing Li

cs.CVMar 20, 2026

K-GMRF: Kinetic Gauss-Markov Random Field for First-Principles Covariance Tracking on Lie Groups

Authors: ZhiMing Li

Organizations: School of Computer Science and Technology, Tianjin University

Abstract

Tracking non-stationary covariance matrices is fundamental to vision yet hindered by existing estimators that either neglect manifold constraints or rely on first-order updates, incurring inevitable phase lag during rapid evolution. We propose K-GMRF, an online, training-free framework for covariance tracking that reformulates the problem as forced rigid-body motion on Lie groups. Derived from the Euler-Poincaré equations, our method interprets observations as torques driving a latent angular velocity, propagated via a structure-preserving symplectic integrator. We theoretically prove that this second-order dynamics achieves zero steady-state error under constant rotation, strictly superior to the proportional lag of first-order baselines. Validation across three domains demonstrates robust tracking fidelity: (i) on synthetic ellipses, K-GMRF reduces angular error by 30x compared to Riemannian EMA while maintaining stability at high speeds; (ii) on SO(3) stabilization with 20% dropout, it decreases geodesic error from 29.4° to 9.9°; and (iii) on OTB motion-blur sequences, it improves loU from 0.55 to 0.74 on BlurCar2 with a 96% success rate. As a fully differentiable symplectic module, K-GMRF provides a plug-and-play geometric prior for data-constrained scenarios and an interpretable layer within modern deep architectures.

Figures & tables

	Setting	K-GMRF (Ours)	Riem. EMA	Eucl. EMA	Tangent KF	Alpha-Beta	Gain
(A) Rotating Ellipse on SPD(2): Synthetic covariance tracking, $\omega{=}0.08$ rad/step, 400 frames. Angular Error (deg) $\downarrow$
	Normal	$\mathbf{0.51}{\scriptstyle\pm 0.03}$	$15.62{\scriptstyle\pm 0.06}$	$15.62{\scriptstyle\pm 0.06}$	$0.84{\scriptstyle\pm 0.02}$	$2.41{\scriptstyle\pm 0.06}$	30 $\times$
	20% Dropout	$\mathbf{12.07}{\scriptstyle\pm 0.26}$	$25.29{\scriptstyle\pm 0.04}$	$25.29{\scriptstyle\pm 0.04}$	$11.97{\scriptstyle\pm 0.34}$	$16.58{\scriptstyle\pm 0.06}$	2.1 $\times$
(B) Camera Stabilization on SO(3): Coupled oscillations with observation dropout, 200 frames. Geodesic Error (deg) $\downarrow$
	0% Dropout	$\mathbf{4.4}{\scriptstyle\pm 0.2}$	$7.2{\scriptstyle\pm 0.2}$	$7.3{\scriptstyle\pm 0.2}$	$8.1{\scriptstyle\pm 0.6}$	$4.4{\scriptstyle\pm 0.2}$	1.7 $\times$
	10% Dropout	$\mathbf{5.8}{\scriptstyle\pm 0.4}$	$18.7{\scriptstyle\pm 2.3}$	$19.0{\scriptstyle\pm 2.5}$	$16.2{\scriptstyle\pm 3.4}$	$5.8{\scriptstyle\pm 0.3}$	3.2 $\times$

Table 1: Second-order dynamics + manifold structure = robustness. We compare methods along two axes: filter order (1st vs. 2nd) and geometry (Euclidean vs. Riemannian). The “Gain” column reports improvement over Riemannian EMA.

Figure 2: Angular velocity sweep validating Theorems 2 and 3 . As target speed

\omega

increases, EMA error grows linearly (lag

\propto\omega

), while K-GMRF maintains

<0.4^{\circ}

error across all speeds. Alpha-Beta also degrades at high

\omega

Figure 3: Qualitative results on OTB motion-blur sequences. Each panel shows 5 key frames with bounding boxes (GT: white, K-GMRF: red, R-EMA: blue, Alpha-Beta: purple) and the corresponding per-frame IoU curve. K-GMRF consistently maintains higher IoU throughout the sequences, especially during severe motion blur (BlurCar1/2) and camera shake (BlurBody).

Variant	Normal	20% Dropout
K-GMRF (full)	$\mathbf{0.03}{\scriptstyle\pm 0.00}$	$19.6{\scriptstyle\pm 0.01}$
$-$ momentum ( $\gamma{=}0,\eta{=}0$ )	$15.6{\scriptstyle\pm 0.00}$	$27.5{\scriptstyle\pm 0.00}$
$-$ manifold (Alpha-Beta)	$2.09{\scriptstyle\pm 0.00}$	$\mathbf{18.6}{\scriptstyle\pm 0.00}$
$-$ both (Eucl. EMA)	$15.6{\scriptstyle\pm 0.00}$	$27.5{\scriptstyle\pm 0.00}$

Table 2: Ablation study on SPD(2) ellipse tracking (5 seeds,

\omega{=}0.08

). Angular error in degrees.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Validation of Master Theorem (Theorem 5 ). (a) Statistical term: error scales as

m^{-0.50}

(

R^{2}=0.996

), matching the

O(1/m)

prediction. (b) Nonstationarity term: error increases linearly with

V_{\Omega}

. (c) Transient term: initial error decays exponentially with

\kappa\approx 0.19

Figure 5: SPD(2) ellipse tracking visualization. Five key frames showing a rotating ellipse (ground truth: white) tracked by K-GMRF (red), Riemannian EMA (blue), and Alpha-Beta (green). K-GMRF maintains accurate orientation throughout, while EMA variants exhibit phase lag.

Figure 6: SO(3) camera stabilization frames. Five key frames from the stabilization demo. The checkerboard pattern shows camera orientation; K-GMRF maintains stable tracking through dropout periods.

Figure 7: SO(3) camera stabilization trajectories. Roll, pitch, and yaw angles over 400 frames. Red regions indicate observation dropouts. K-GMRF (red) tracks the ground truth (black) through dropout periods via momentum, while Standard EMA (blue) freezes and accumulates error.

Figure 8: SO(3) tracking error over time. Geodesic error (degrees) with dropout events marked. K-GMRF and Alpha-Beta recover quickly after dropouts; EMA error accumulates.

Figure 9: OTB motion-blur sequences: mid-sequence snapshots. All six sequences at frame 51/100. Bounding boxes: K-GMRF (red), Riemannian EMA (blue), Euclidean EMA (cyan), Tangent KF (green), Alpha-Beta (magenta), Ground Truth (white). K-GMRF maintains accurate localization under motion blur.

Method	Time Complexity	Space Complexity
K-GMRF	$O(Kd^{3})$	$O(d^{2})$
Riemannian EMA	$O(d^{3})$	$O(d^{2})$
Euclidean EMA	$O(d^{2})$	$O(d^{2})$
Tangent KF	$O(d^{6})$	$O(d^{4})$
Alpha-Beta	$O(d^{2})$	$O(d^{2})$

Appendix

Table 3: Per-frame computational complexity.

d

: matrix dimension;

K

: Cayley-Neumann iterations.

Method	Search Grid
K-GMRF	$\eta\in\{0.01,0.05,0.1\}$ , $\gamma\in\{0.9,0.95,0.98,1.0\}$
Riemannian EMA	$\beta\in\{0.6,0.7,0.8,0.9\}$
Euclidean EMA	$\beta\in\{0.6,0.7,0.8,0.9\}$
Tangent KF	$Q\in\{0.001,0.005,0.01\}$ , $R\in\{0.05,0.1,0.2\}$
Alpha-Beta	$\alpha\in\{0.3,0.4,0.5,0.6\}$ , $\beta\in\{0.05,0.1,0.15\}$

Appendix

Table 4: Hyperparameter search ranges for each method.

Task	Method	Parameters
SPD(2)	K-GMRF	$\eta{=}0.05$ , $\gamma{=}0.95$ , $\beta{=}0.9$
	Riemannian EMA	$\beta{=}0.8$
	Euclidean EMA	$\beta{=}0.8$
	Tangent KF	$Q{=}0.005$ , $R{=}0.1$
	Alpha-Beta	$\alpha{=}0.4$ , $\beta{=}0.1$
SO(3)	K-GMRF	$\alpha{=}0.5$ , $\beta{=}0.05$ , $\gamma{=}0.98$

Appendix

Table 5: Optimal hyperparameters selected on tuning seeds.

$\omega$	K-GMRF	R-EMA	E-EMA	T-KF	A-B
0.03	$\mathbf{0.29}{\scriptstyle\pm 0.02}$	$6.66{\scriptstyle\pm 0.04}$	$6.66{\scriptstyle\pm 0.04}$	$0.47{\scriptstyle\pm 0.01}$	$0.34{\scriptstyle\pm 0.02}$
0.05	$\mathbf{0.30}{\scriptstyle\pm 0.02}$	$10.72{\scriptstyle\pm 0.06}$	$10.72{\scriptstyle\pm 0.06}$	$0.48{\scriptstyle\pm 0.02}$	$0.70{\scriptstyle\pm 0.05}$
0.08	$\mathbf{0.29}{\scriptstyle\pm 0.03}$	$15.61{\scriptstyle\pm 0.04}$	$15.61{\scriptstyle\pm 0.04}$	$0.48{\scriptstyle\pm 0.02}$	$2.40{\scriptstyle\pm 0.04}$
0.10	$\mathbf{0.31}{\scriptstyle\pm 0.02}$	$18.18{\scriptstyle\pm 0.03}$	$18.18{\scriptstyle\pm 0.03}$	$0.54{\scriptstyle\pm 0.01}$	$4.30{\scriptstyle\pm 0.04}$
0.15	$\mathbf{0.32}{\scriptstyle\pm 0.02}$	$22.54{\scriptstyle\pm 0.08}$	$22.54{\scriptstyle\pm 0.08}$	$1.02{\scriptstyle\pm 0.06}$	$10.61{\scriptstyle\pm 0.08}$
0.20	$\mathbf{0.38}{\scriptstyle\pm 0.03}$	$24.91{\scriptstyle\pm 0.04}$	$24.91{\scriptstyle\pm 0.05}$	$2.17{\scriptstyle\pm 0.04}$	$16.65{\scriptstyle\pm 0.04}$

Appendix

Table 6: Angular error (degrees) on SPD(2) ellipse tracking across angular velocities

\omega

(rad/step). Mean

\pm

std over 5 testing seeds.

Dropout	K-GMRF	R-EMA	E-EMA	T-KF	A-B
0%	$\mathbf{4.4}{\scriptstyle\pm 0.1}$	$7.2{\scriptstyle\pm 0.2}$	$7.3{\scriptstyle\pm 0.2}$	$8.1{\scriptstyle\pm 0.6}$	$4.4{\scriptstyle\pm 0.1}$
10%	$\mathbf{5.8}{\scriptstyle\pm 0.3}$	$18.7{\scriptstyle\pm 2.3}$	$19.0{\scriptstyle\pm 2.5}$	$16.2{\scriptstyle\pm 3.4}$	$5.8{\scriptstyle\pm 0.3}$
20%	$\mathbf{6.5}{\scriptstyle\pm 1.1}$	$29.2{\scriptstyle\pm 4.1}$	$30.2{\scriptstyle\pm 4.5}$	$22.5{\scriptstyle\pm 4.7}$	$6.6{\scriptstyle\pm 1.0}$
30%	$\mathbf{8.0}{\scriptstyle\pm 1.4}$	$41.0{\scriptstyle\pm 2.1}$	$43.5{\scriptstyle\pm 2.4}$	$32.1{\scriptstyle\pm 7.2}$	$8.1{\scriptstyle\pm 1.4}$
40%	$\mathbf{14.3}{\scriptstyle\pm 7.2}$	$56.5{\scriptstyle\pm 10.3}$	$63.3{\scriptstyle\pm 13.3}$	$38.0{\scriptstyle\pm 8.6}$	$14.9{\scriptstyle\pm 7.8}$
50%	$\mathbf{25.0}{\scriptstyle\pm 9.4}$	$81.2{\scriptstyle\pm 15.1}$	$91.9{\scriptstyle\pm 12.1}$	$48.5{\scriptstyle\pm 11.4}$	$25.7{\scriptstyle\pm 7.4}$

Appendix

Table 7: Geodesic error (degrees) on SO(3) camera stabilization across dropout rates. Mean

\pm

std over 5 testing seeds.

Figure 10: SO(3) dropout sweep. Geodesic error vs. dropout rate. Second-order methods (K-GMRF, Alpha-Beta) degrade gracefully, while first-order methods (EMA variants) collapse beyond 30% dropout. Shaded regions:

\pm 1

std over 5 seeds.

Figure 11: BlurCar2 tracking progression. Six key frames showing K-GMRF’s best performance (IoU: 0.74 vs. 0.55 for R-EMA). K-GMRF (red) maintains accurate localization through severe motion blur, while baselines diverge.

Figure 12: BlurCar2 per-frame IoU. K-GMRF (red) maintains higher IoU than Standard EMA (blue) throughout the sequence, especially during frames 75–150 where motion blur is most severe.

	Mean IoU $\uparrow$				Success Rate $\uparrow$
Sequence	K-GMRF	R-EMA	E-EMA	T-KF	K-GMRF	R-EMA	E-EMA	T-KF
BlurBody	$\mathbf{0.68}$	$0.69$	$0.69$	$0.56$	$\mathbf{0.96}$	$0.96$	$0.96$	$0.67$
BlurCar1	$\mathbf{0.55}$	$0.54$	$0.53$	$0.39$	$\mathbf{0.55}$	$0.47$	$0.51$	$0.22$
BlurCar2	$\mathbf{0.80}$	$0.73$	$0.76$	$0.53$	$\mathbf{1.00}$	$1.00$	$1.00$	$0.41$
BlurFace	$0.86$	$0.85$	$\mathbf{0.86}$	$0.71$	$1.00$	$1.00$	$1.00$	$1.00$
CarScale	$\mathbf{0.67}$	$0.67$	$0.66$	$0.19$	$0.96$	$\mathbf{1.00}$	$0.96$	$0.14$

Appendix

Table 8: OTB tracking results: IoU and Success Rate (SR, IoU

>

0.5). K-GMRF uses calibrated thresholds per sequence.

Sequence	K-GMRF	R-EMA	E-EMA	T-KF
BlurBody	33.6	32.4	30.5	27.0
BlurCar1	11.9	12.3	12.9	11.4
BlurCar2	12.0	12.0	11.9	11.9
BlurFace	10.6	10.6	10.6	10.7
CarScale	1.4	1.4	1.4	1.0
Jogging	3.4	3.1	3.2	3.2

Appendix

Table 9: Per-sequence runtime (seconds) on OTB. CPU: Intel i7-12700K. All methods are single-threaded.

Parameter	Physical Meaning	Effect
$\eta$ (step size)	Inverse inertia	Larger $\eta$ $\Rightarrow$ faster response, more noise
$\gamma$ (damping)	Friction coefficient	Larger $\gamma$ $\Rightarrow$ smoother, more lag
$1-\gamma$	Momentum retention	Smaller $\gamma$ $\Rightarrow$ longer coasting

Appendix

Table 10: Physical interpretation of K-GMRF hyperparameters.

Symptom	Cause	Remedy
High-frequency oscillation	$\eta$ too large	Reduce $\eta$ by 50%
Overshooting then correcting	$\gamma$ too small	Increase $\gamma$ toward 0.98
Persistent phase lag	$\gamma$ too large	Decrease $\gamma$ toward 0.9
Gradual drift	Numerical error	Check matrix symmetry; re-orthogonalize $Q$
Explosive divergence	Outside $\mathcal{D}$	Reset to safe defaults: $\eta{=}0.01$ , $\gamma{=}0.99$

Appendix

Table 11: Common failure modes and remedies.

Figure 13: Spectral gap phase transition. Error vs. whitened spectral gap

\Delta_{\mathrm{wh}}

(log scale). Below

\Delta_{\mathrm{wh}}\approx 0.05

(dashed line), both methods fail. Above this threshold, K-GMRF separates from EMA, achieving

5\times

lower error at

\Delta_{\mathrm{wh}}=0.8

$\delta$	$\Delta_{\mathrm{wh}}$	K-GMRF	EMA	K-GMRF/EMA
0.01	0.0012	$44.6^{\circ}$	$44.7^{\circ}$	$1.00\times$
0.05	0.0061	$43.3^{\circ}$	$43.5^{\circ}$	$1.00\times$
0.10	0.0129	$41.4^{\circ}$	$41.9^{\circ}$	$1.01\times$
0.30	0.0475	$31.3^{\circ}$	$35.3^{\circ}$	$1.13\times$
0.50	0.1027	$16.7^{\circ}$	$30.7^{\circ}$	$1.84\times$
1.00	0.8032	$5.7^{\circ}$	$28.9^{\circ}$	$5.05\times$

Appendix

Table 12: Spectral gap phase transition. Error (degrees) vs. eigenvalue gap

\delta

. Noise

\sigma^{2}=4.0

m=8

Variant	Normal	20% Dropout
K-GMRF (full)	$\mathbf{1.18}{\scriptstyle\pm 0.00}$	$15.1{\scriptstyle\pm 0.01}$
$-$ momentum (R-EMA)	$13.87{\scriptstyle\pm 0.00}$	$24.1{\scriptstyle\pm 0.00}$
$-$ manifold (Alpha-Beta)	$1.00{\scriptstyle\pm 0.00}$	$\mathbf{13.9}{\scriptstyle\pm 0.00}$
$-$ both (Eucl. EMA)	$15.62{\scriptstyle\pm 0.00}$	$25.3{\scriptstyle\pm 0.00}$

Appendix

Table 13: Ablation study on SPD(2). Angular error (degrees),

\omega=0.08

rad/step, 400 frames.

Covariance Estimation

Covariance estimation develops methods for learning dependence and uncertainty structure from noisy, incomplete, or high-dimensional data.

See all 60 papers

Sep 22, 2026

Dual Covariance Gaussian Splatting SLAM: Decoupling Rendering and Registration for Robust Real-Time Tracking

ICP-based 3D Gaussian Splatting (3DGS) SLAM tracks in real time by registering incoming frames against map Gaussians, using each primitive's covariance for both rendering and registration. These two uses place conflicting demands on one covariance. The mapper shapes it to minimize photometric error, often flattening it against surfaces, while robust registration typically benefits from measurement uncertainty. We propose a dual-covariance parameterization. Each Gaussian keeps a single mean but holds two covariances: a rendering covariance optimized by the mapper, and a tracking covariance derived from an RGB-D sensor noise model. We further use the tracking covariances as Gaussian anchors for image corners, providing constraints in directions where depth geometry is weak. We evaluate on TUM RGB-D, ScanNet, Replica, and two outdoor sequences recorded with a RealSense D435i on wheeled and handheld platforms. We achieve robust tracking performance across multiple scenes and reduced odometry drift, while tracking at

\sim

60 FPS.

Edward Beng Wai Tan, Siew-Kei Lam

College of Computing and Data Science, Nanyang Technological University, Singapore.

3D Gaussian Splatting 3DGS SLAM Visual SLAM Point Cloud Registration

Ask PDF

Jun 13, 2026

Covariance-Regulated Recursive Koopman Learning for Nonlinear Systems with Uncertain Time-Varying Dynamics

Offline models for autonomous robots often fail under time-varying dynamics outside their training distribution. Koopman operator theory offers a linear representation of nonlinear dynamics via lifting, but its transition to real-time recursive estimation may suffer numerical vulnerabilities: covariance windup under low excitation when using exponential forgetting, and vanishing gain without forgetting. This paper introduces a Covariance-Regulated Recursive Koopman Learning (CR-RKL) framework with two complementary strategies--error dead-zone gating and constant-trace normalization--each independently capable of preventing covariance explosion and parameter freezing, with the latter additionally preserving the geometric structure of uncertainty. Validated on a non-holonomic differential-drive robot with wheel slip and Stribeck friction and on a 26-gram butterfly-inspired flapping-wing micro aerial vehicle, CR-RKL achieves numerically stable and accurate online modeling, and when embedded in model predictive control, it maintains reliable tracking performance under uncertain, time-varying dynamics.

Weibin Gu, Chen Yang, Lu Shi +1

1Tsinghua University, PR China · 2China University of Petroleum-Beijing at Karamay, PR China · 3Xinchen Qihang Inc., PR China

Online System Identification Recursive Least Squares Koopman Operator Learning Nonlinear System Identification

Ask PDF

Apr 12, 2026

Online Covariance Estimation in Averaged SGD: Improved Batch-Mean Rates and Minimax Optimality via Trajectory Regression

We study online covariance matrix estimation for Polyak--Ruppert averaged stochastic gradient descent (SGD). The online batch-means estimator of Zhu, Chen and Wu (2023) achieves an operator-norm convergence rate of

O(n^{-(1-α)/4})

, which yields

O(n^{-1/8})

at the optimal learning-rate exponent

α\rightarrow 1/2^+

. A rigorous per-block bias analysis reveals that re-tuning the block-growth parameter improves the batch-means rate to

O(n^{-(1-α)/3})

, achieving

O(n^{-1/6})

. The modified estimator requires no Hessian access and preserves

O(d^2)

memory. We provide a complete error decomposition into variance, stationarity bias, and nonlinearity bias components. A weighted-averaging variant that avoids hard truncation is also discussed. We establish the minimax rate

Θ(n^{-(1-α)/2})

for Hessian-free covariance estimation from the SGD trajectory: a Le Cam lower bound gives

Ω(n^{-(1-α)/2})

, and a trajectory-regression estimator--which estimates the Hessian by regressing SGD increments on iterates--achieves

O(n^{-(1-α)/2})

, matching the lower bound. The construction reveals that the bottleneck is the sublinear accumulation of information about the Hessian from the SGD drift.

Yijin Ni, Xiaoming Huo

H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology

Polyak-Ruppert Averaging Covariance Estimation Minimax Estimation Stochastic Gradient Descent

Ask PDF

Abstract

Covariance Estimation

Explore similar work

Dual Covariance Gaussian Splatting SLAM: Decoupling Rendering and Registration for Robust Real-Time Tracking

Covariance-Regulated Recursive Koopman Learning for Nonlinear Systems with Uncertain Time-Varying Dynamics

Online Covariance Estimation in Averaged SGD: Improved Batch-Mean Rates and Minimax Optimality via Trajectory Regression