Online Learning via Learned Latent Bayesian Tracking

Guy Gerson; Tomer Raviv; Nir Shlezinger; Tirza Routtenberg; Osvaldo Simeone

cs.LGSep 25, 2026

Online Learning via Learned Latent Bayesian Tracking

Authors: Guy Gerson, Tomer Raviv, Nir Shlezinger, Tirza Routtenberg, Osvaldo Simeone

Organizations: School of ECE Ben-Gurion University Be‘er-Sheva, Israel · Institute for Intelligent Networked Systems (INSI) Northeastern University London London, U.K

Abstract

Online learning in non-stationary environments requires models to adapt rapidly from streaming data under strict computational constraints. A principled approach casts online learning as Bayesian state tracking, where model parameters are updated sequentially via Bayesian filtering. However, applying Bayesian filters directly to modern deep models is computationally prohibitive due to the high dimensionality of parameter space, forcing existing methods to rely on restrictive approximations or manually designed low-dimensional subspaces. In this work, we identify the absence of a suitable low-dimensional dynamical representation as the core bottleneck in Bayesian filtering-based online learning. Accordingly, we propose Adaptive Update through Representation Adaptation (AURA), a meta-learning framework that learns offline a low-dimensional latent state-space model governing the evolution of optimal model parameters under distribution shift. Online adaptation is then performed via extended Kalman filtering in this learned latent space followed by reconstruction of the full model parameters through a learned lifting map, enabling efficient single-step online adaptation while preserving model expressiveness. Evaluated on online adaptation of neural wireless receivers under time-varying channels and on non-stationary image classification, AURA shows substantial improvements in adaptation speed, accuracy, and computational efficiency over existing online learning and Bayesian filtering baselines, demonstrating that an adaptation-aware latent geometry is beneficial for effective Bayesian online learning in high-dimensional models.

Figures & tables

Figure 1: Proposed methodology: $a)$ name pipeline via ekf in meta-learned latent state-space representation (marked in red fonts); $b)$ Error–complexity trade-off for deep receiver adaptation, showing name ’s enhanced tradeoff compared to altering the learning algorithm in parameter space.

Regime	Method	# Train Samples	MNIST-C	CIFAR-10-C	CIFAR-100-C
Pre-training	Source	0	47.93 $\pm$ 19.83	41.81 $\pm$ 5.62	37.41 $\pm$ 4.82
Streaming	Online GD [ 47 ]	5	7.50 $\pm$ 0.50	29.46 $\pm$ 5.55	37.93 $\pm$ 1.83
		10	5.84 $\pm$ 0.24	25.89 $\pm$ 0.75	34.00 $\pm$ 2.37
		15	4.53 $\pm$ 0.39	22.37 $\pm$ 1.38	33.22 $\pm$ 3.35
	EKF-FC [ 13 ]	5	19.74 $\pm$ 5.09	30.62 $\pm$ 1.34	31.27 $\pm$ 1.98
		10	17.00 $\pm$ 4.96	28.69 $\pm$ 4.61	33.38 $\pm$ 2.65

Table 1: Mean classification error (%, 3 seeds per cell) under gradual non-stationary corruptions. Methods are grouped by adaptation regime. Best per dataset in bold ; second best underlined

Figure 2: Complexity–performance trade-off in streaming online learning. The blue curve sweeps AURA’s latent dimension; the number above each marker is that

m

. The red diamond is EKF-FC. Shaded regions separate the latent and parameter-space regimes. Error bars are

\pm 1

standard deviation.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: BER vs. Snapshot Index

Prediction method	Latent dim.	Mode	SNR $=8$ dB	SNR $=10$ dB
OU	$m=100$	warm-start	$\mathbf{0.832\pm 0.485}$	$\mathbf{0.512\pm 0.379}$
OU	$m=100$	cold-start	$14.399\pm 3.024$	$11.542\pm 1.054$
OU	$m=250$	warm-start	$\mathbf{0.409\pm 0.514}$	$\mathbf{0.292\pm 0.418}$
OU	$m=250$	cold-start	$14.662\pm 1.541$	$12.225\pm 1.259$
OU	$m=400$	warm-start	$\mathbf{0.340\pm 0.496}$	$\mathbf{0.249\pm 0.353}$
OU	$m=400$	cold-start	$14.828\pm 2.880$	$12.320\pm 0.643$

Appendix

Table 2: Effect of offline learning in the QuaDRiGa communication scenario. Each pair compares the same prediction method and latent dimension with and without offline learning. BER in (%)

Figure 12: FLOPs-error trade-off on the linear QuaDRiGa channel at $\mathrm{SNR}=12$ dB

$I$	$K$	name	BONG Full	BONG Diag
1	2	$\mathbf{1.04\pm 0.21}$	$12.09\pm 1.32$	$17.02\pm 1.48$
1	4	$\mathbf{0.10\pm 0.02}$	$3.15\pm 0.35$	$8.54\pm 0.83$
1	6	$\mathbf{0.07\pm 0.01}$	$1.11\pm 0.04$	$4.14\pm 0.23$
2	2	$\mathbf{6.19\pm 0.61}$	$25.00\pm 1.25$	$29.38\pm 0.65$
2	4	$\mathbf{0.70\pm 0.07}$	$12.17\pm 0.38$	$17.56\pm 0.68$
2	6	$\mathbf{0.20\pm 0.07}$	$6.11\pm 0.17$	$12.18\pm 0.28$

Appendix

Table 3: BER under intermittent supervision (%, mean

\pm

standard deviation over three seeds).

Panel of Figure 13 — Figure 13: Ablation of full- ${\boldsymbol{F}}$ latent dynamics on linear QuaDRiGa channel.

Dataset	# Labeled Samples	Affine	Nonlinear
CIFAR-10-C	5	$16.13\pm 1.50$	$\mathbf{10.21\pm 0.40}$
	10	$16.84\pm 0.51$	$\mathbf{10.15\pm 0.81}$
	15	$14.28\pm 2.87$	$\mathbf{9.75\pm 0.54}$
CIFAR-100-C	5	$29.47\pm 0.62$	$\mathbf{26.34\pm 1.77}$
	10	$29.65\pm 1.97$	$\mathbf{25.21\pm 1.04}$
	15	$26.66\pm 3.89$	$\mathbf{23.00\pm 0.79}$

Appendix

Table 4: Effect of the lifting-map architecture on classification error (%, mean

\pm

standard deviation over three seeds).

Method	ROID	AURA	BN-EMA	BN-Test	EKF-FC	BN-Alpha	Source	Online GD
Classification Error (%)	$9.88\pm 1.01$	$9.99\pm 1.30$	$11.50\pm 2.64$	$11.62\pm 1.29$	$12.31\pm 1.02$	$13.55\pm 2.47$	$17.07\pm 2.67$	$16.21\pm 2.04$

Appendix

Table 5: Cross-corruption generalization under mixed-gradual CIFAR-10.

Dataset	LoRA	name , Affine	name , Nonlinear
CIFAR-10-C	$15.73\pm 1.23$	$14.28\pm 2.87$	$\mathbf{9.75\pm 0.54}$
CIFAR-100-C	$28.90\pm 1.63$	$26.66\pm 3.89$	$\mathbf{23.00\pm 0.79}$

Appendix

Table 6: Classification error (%, mean

\pm

standard deviation over three seeds) for LoRA and name .

Method	Supervision	CIFAR-10-C	CIFAR-100-C	Latency (ms/sample) for CIFAR-100-C)
Source	None	41.81	37.41	–
name	5 labeled	16.13	29.47	2.01
EcoTTA [ 89 ]	Unlabeled	20.18	41.20	1.15
ELaTTA [ 90 ]	Unlabeled	44.82	39.98	0.41

Appendix

Table 7: Classification error (%) and adaptation latency for efficient test-time adaptation methods.

Lifting map	After shift	10 updates	20 updates	40 updates
Affine	45.96	27.68	21.74	19.44
Nonlinear	42.44	23.30	19.05	17.66

Appendix

Table 8: Recovery from abrupt distribution shifts on CIFAR-10-C. Mean classification error (%) across five corruption shifts.

Figure 14: Hardware proof-of-concept setup for the self-adaptive neural WiFi receiver. The adaptive receiver is deployed inline with a conventional classical receiver within an over-the-air IEEE 802.11a communication link implemented using PlutoSDR+ RF front-ends, enabling simultaneous evaluation under identical channel realizations.

Figure 15: Average ber heatmaps versus snr and cfo level for neural receiver adapted with name (left) compared to classical channel estimation + LMMSE equalization receiver (right)

Architecture	Inference Runtime [ms]	Adaptation Runtime [ms]
1.8K-Parameter CNN	0.084	10.573
6.6k-Parameter CNN	0.113	40.414
8.2k-Parameter CNN	0.221	84.393

Appendix

Table 9: Measured per-packet runtime of the adaptive neural receiver for different CNN equalizer architectures. All runtimes are reported in milliseconds and measured using CPU-only execution.

Online Learning

Online learning studies algorithms that update predictions or decisions from sequential feedback while controlling cumulative error or regret.

See all 132 papers

Full-Covariance Smoothing of Bayesian Neural Networks for Online Adaptation
Sep 23, 2026Oren Wright, Haoming Jing, Qiaoan Shen +3Bayesian Neural Networks Bayesian Filtering
Department of Electrical and Computer Engineering, Carnegie Mellon University · Software Engineering Institute, Carnegie Mellon University · Fujitsu Research of America
Adaptive Bayesian Online Learning via Expert Aggregation
Jul 22, 2026Jungbin Jun, Ilsang OhnBayesian Inference Adaptive Conformal Inference
Department of Statistics Inha University Incheon, 22212, South Korea
Bayesian Fine-tuning in Projected Subspaces
May 8, 2026Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1Bayesian Neural Networks Fine-Tuning
Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland

Sep 23, 2026J/K move · Enter open · S save

Full-Covariance Smoothing of Bayesian Neural Networks for Online Adaptation

Oren Wright, Haoming Jing, Qiaoan Shen, Koichiro Niinuma+2

Department of Electrical and Computer Engineering, Carnegie Mellon University · Software Engineering Institute, Carnegie Mellon University · Fujitsu Research of America

Bayesian Neural Networks Bayesian Filtering Online Learning State Space Models

Open paper AskPDF arXiv

A neural network's layers can be treated as time steps of a state-space model, turning Bayesian training into a smoothing problem: a forward pass propagates Gaussian moments through the network, and a backward Rauch--Tung--Striebel pass updates the weight posteriors in closed form. Such methods learn from each observation in a single pass, in an uncertainty-aware manner, and without gradient-based iterations or replay, which makes them well suited for online adaptation and data-efficient learning. Existing smoothing-based methods, however, are restricted to diagonal covariances across activations, discarding correlations between neurons. We overcome this limitation via a cross-covariance identity that enables full-covariance propagation through a network's nonlinear activations. We derive a one-step-per-layer smoother that approximates as Gaussian only each layer's affine output, and that applies both to deterministic systems with noisy observations and to stochastic systems described by output statistics. We demonstrate this method in non-stationary classification, online dynamics learning, and policy adaptation of a vision-language-action model, and find that it is generally more accurate than other smoothing-based methods.

Abstract

Online Learning

Explore similar work

Full-Covariance Smoothing of Bayesian Neural Networks for Online Adaptation

Adaptive Bayesian Online Learning via Expert Aggregation

Bayesian Fine-tuning in Projected Subspaces