Organizations: School of ECE Ben-Gurion University Be‘er-Sheva, Israel · Institute for Intelligent Networked Systems (INSI) Northeastern University London London, U.K
Online learning in non-stationary environments requires models to adapt rapidly from streaming data under strict computational constraints. A principled approach casts online learning as Bayesian state tracking, where model parameters are updated sequentially via Bayesian filtering. However, applying Bayesian filters directly to modern deep models is computationally prohibitive due to the high dimensionality of parameter space, forcing existing methods to rely on restrictive approximations or manually designed low-dimensional subspaces. In this work, we identify the absence of a suitable low-dimensional dynamical representation as the core bottleneck in Bayesian filtering-based online learning. Accordingly, we propose Adaptive Update through Representation Adaptation (AURA), a meta-learning framework that learns offline a low-dimensional latent state-space model governing the evolution of optimal model parameters under distribution shift. Online adaptation is then performed via extended Kalman filtering in this learned latent space followed by reconstruction of the full model parameters through a learned lifting map, enabling efficient single-step online adaptation while preserving model expressiveness. Evaluated on online adaptation of neural wireless receivers under time-varying channels and on non-stationary image classification, AURA shows substantial improvements in adaptation speed, accuracy, and computational efficiency over existing online learning and Bayesian filtering baselines, demonstrating that an adaptation-aware latent geometry is beneficial for effective Bayesian online learning in high-dimensional models.
Figures & tables
Figure 1: Proposed methodology: a) name pipeline via ekf in meta-learned latent state-space representation (marked in red fonts); b) Error–complexity trade-off for deep receiver adaptation, showing name ’s enhanced tradeoff compared to altering the learning algorithm in parameter space.
Regime
Method
# Train Samples
MNIST-C
CIFAR-10-C
CIFAR-100-C
Pre-training
Source
0
47.93 ± 19.83
41.81 ± 5.62
37.41 ± 4.82
Streaming
Online GD [ 47 ]
5
7.50 ± 0.50
29.46 ± 5.55
37.93 ± 1.83
10
5.84 ± 0.24
25.89 ± 0.75
34.00 ± 2.37
15
4.53 ± 0.39
22.37 ± 1.38
33.22 ± 3.35
EKF-FC [ 13 ]
5
19.74 ± 5.09
30.62 ± 1.34
31.27 ± 1.98
10
17.00 ± 4.96
28.69 ± 4.61
33.38 ± 2.65
Table 1: Mean classification error (%, 3 seeds per cell) under gradual non-stationary corruptions. Methods are grouped by adaptation regime. Best per dataset in bold ; second best underlined
Figure 2: Complexity–performance trade-off in streaming online learning. The blue curve sweeps AURA’s latent dimension; the number above each marker is that m . The red diamond is EKF-FC. Shaded regions separate the latent and parameter-space regimes. Error bars are ±1 standard deviation.
Figure 4
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: BER vs. Snapshot Index
Figure 6Figure 7Figure 8
Prediction method
Latent dim.
Mode
SNR =8 dB
SNR =10 dB
OU
m=100
warm-start
0.832±0.485
0.512±0.379
cold-start
14.399±3.024
11.542±1.054
OU
m=250
warm-start
0.409±0.514
0.292±0.418
cold-start
14.662±1.541
12.225±1.259
OU
m=400
warm-start
0.340±0.496
0.249±0.353
cold-start
14.828±2.880
12.320±0.643
Appendix
Table 2: Effect of offline learning in the QuaDRiGa communication scenario. Each pair compares the same prediction method and latent dimension with and without offline learning. BER in (%)
Figure 12: FLOPs-error trade-off on the linear QuaDRiGa channel at SNR=12 dB
I
K
name
BONG Full
BONG Diag
1
2
1.04±0.21
12.09±1.32
17.02±1.48
1
4
0.10±0.02
3.15±0.35
8.54±0.83
1
6
0.07±0.01
1.11±0.04
4.14±0.23
2
2
6.19±0.61
25.00±1.25
29.38±0.65
2
4
0.70±0.07
12.17±0.38
17.56±0.68
2
6
0.20±0.07
6.11±0.17
12.18±0.28
Appendix
Table 3: BER under intermittent supervision (%, mean ± standard deviation over three seeds).
Figure 13: Ablation of full- F latent dynamics on linear QuaDRiGa channel.
Dataset
# Labeled Samples
Affine
Nonlinear
CIFAR-10-C
5
16.13±1.50
10.21±0.40
10
16.84±0.51
10.15±0.81
15
14.28±2.87
9.75±0.54
CIFAR-100-C
5
29.47±0.62
26.34±1.77
10
29.65±1.97
25.21±1.04
15
26.66±3.89
23.00±0.79
Appendix
Table 4: Effect of the lifting-map architecture on classification error (%, mean ± standard deviation over three seeds).
Method
ROID
AURA
BN-EMA
BN-Test
EKF-FC
BN-Alpha
Source
Online GD
Classification Error (%)
9.88±1.01
9.99±1.30
11.50±2.64
11.62±1.29
12.31±1.02
13.55±2.47
17.07±2.67
16.21±2.04
Appendix
Table 5: Cross-corruption generalization under mixed-gradual CIFAR-10.
Dataset
LoRA
name , Affine
name , Nonlinear
CIFAR-10-C
15.73±1.23
14.28±2.87
9.75±0.54
CIFAR-100-C
28.90±1.63
26.66±3.89
23.00±0.79
Appendix
Table 6: Classification error (%, mean ± standard deviation over three seeds) for LoRA and name .
Method
Supervision
CIFAR-10-C
CIFAR-100-C
Latency (ms/sample) for CIFAR-100-C)
Source
None
41.81
37.41
–
name
5 labeled
16.13
29.47
2.01
EcoTTA [ 89 ]
Unlabeled
20.18
41.20
1.15
ELaTTA [ 90 ]
Unlabeled
44.82
39.98
0.41
Appendix
Table 7: Classification error (%) and adaptation latency for efficient test-time adaptation methods.
Lifting map
After shift
10 updates
20 updates
40 updates
Affine
45.96
27.68
21.74
19.44
Nonlinear
42.44
23.30
19.05
17.66
Appendix
Table 8: Recovery from abrupt distribution shifts on CIFAR-10-C. Mean classification error (%) across five corruption shifts.
Figure 14: Hardware proof-of-concept setup for the self-adaptive neural WiFi receiver. The adaptive receiver is deployed inline with a conventional classical receiver within an over-the-air IEEE 802.11a communication link implemented using PlutoSDR+ RF front-ends, enabling simultaneous evaluation under identical channel realizations.
Figure 15: Average ber heatmaps versus snr and cfo level for neural receiver adapted with name (left) compared to classical channel estimation + LMMSE equalization receiver (right)
Architecture
Inference Runtime [ms]
Adaptation Runtime [ms]
1.8K-Parameter CNN
0.084
10.573
6.6k-Parameter CNN
0.113
40.414
8.2k-Parameter CNN
0.221
84.393
Appendix
Table 9: Measured per-packet runtime of the adaptive neural receiver for different CNN equalizer architectures. All runtimes are reported in milliseconds and measured using CPU-only execution.
A neural network's layers can be treated as time steps of a state-space model, turning Bayesian training into a smoothing problem: a forward pass propagates Gaussian moments through the network, and a backward Rauch--Tung--Striebel pass updates the weight posteriors in closed form. Such methods learn from each observation in a single pass, in an uncertainty-aware manner, and without gradient-based iterations or replay, which makes them well suited for online adaptation and data-efficient learning. Existing smoothing-based methods, however, are restricted to diagonal covariances across activations, discarding correlations between neurons. We overcome this limitation via a cross-covariance identity that enables full-covariance propagation through a network's nonlinear activations. We derive a one-step-per-layer smoother that approximates as Gaussian only each layer's affine output, and that applies both to deterministic systems with noisy observations and to stochastic systems described by output statistics. We demonstrate this method in non-stationary classification, online dynamics learning, and policy adaptation of a vision-language-action model, and find that it is generally more accurate than other smoothing-based methods.
Oren Wright, Haoming Jing, Qiaoan Shen +3
Department of Electrical and Computer Engineering, Carnegie Mellon University · Software Engineering Institute, Carnegie Mellon University · Fujitsu Research of America
Bayesian online learning promises uncertainty-aware prediction on data streams, but its performance hinges on inferential choices, including learning rates, prior distributions and variational families, which are usually fixed before seeing the stream. We address this by treating Bayesian update rules as experts and aggregating the Bayesian experts according to sequential predictive losses. We prove that the resulting aggregate competes with the best expert in hindsight at an aggregation cost determined by how each expert's per-round performance is evaluated. We instantiate the framework in online conformal inference and Gaussian process regression. The conformal inference application yields a smoothed Bayesian counterpart of adaptive conformal inference with long-run randomized coverage, while the Gaussian process application gives an oracle inequality in cumulative predictive Kullback-Leibler risk and adaptation to unknown Hölder smoothness up to logarithmic factors. Experiments show that the aggregate tracks strong experts without oracle expert selection.
Jungbin Jun, Ilsang Ohn
Department of Statistics Inha University Incheon, 22212, South Korea
Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning of large models by decomposing weight updates into low-rank matrices, significantly reducing storage and computational overhead. While effective, standard LoRA lacks mechanisms for uncertainty quantification, leading to overconfident and poorly calibrated models. Bayesian variants of LoRA address this limitation, but at the cost of a significantly increased number of trainable parameters, partially offsetting the original efficiency gains. Additionally, these models are harder to train and may suffer from unstable convergence. In this work, we propose a novel framework for parameter-efficient Bayesian fine-tuning, demonstrating that effective uncertainty quantification can be achieved in very low-dimensional parameter spaces. The proposed method achieves strong performance with improved calibration and generalization while maintaining computational efficiency. Our empirical findings show that, with the appropriate projection of the weight space uncertainty can be effectively modeled in a low-dimensional space, and weight covariances exhibit low ranks.
Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1
Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland