Adaptive Learned State Estimation based on KalmanNet

Arian Mehrfard; Bharanidhar Duraisamy; Stefan Haag; Florian Geiss; Mirko Mählisch

cs.ROApr 2, 2026

Adaptive Learned State Estimation based on KalmanNet

Authors: Arian Mehrfard, Bharanidhar Duraisamy, Stefan Haag, Florian Geiss, Mirko Mählisch

Organizations: Research and Development, Mercedes-Benz AG, Germany · Institute for Autonomous Driving, University of the Bundeswehr Munich, Germany

Abstract

Hybrid state estimators that combine model-based Kalman filtering with learned components have shown promise on simulated data, yet their performance on real-world automotive data remains insufficient. In this work we present Adaptive Multi-modal KalmanNet (AM-KNet), an advancement of KalmanNet tailored to the multi-sensor autonomous driving setting. AM-KNet introduces sensor-specific measurement modules that enable the network to learn the distinct noise characteristics of radar, lidar, and camera independently. A hypernetwork with context modulation conditions the filter on target type, motion state, and relative pose, allowing adaptation to diverse traffic scenarios. We further incorporate a covariance estimation branch based on the Josephs form and supervise it through negative log-likelihood losses on both the estimation error and the innovation. A comprehensive, component-wise loss function encodes physical priors on sensor reliability, target class, motion state, and measurement flow consistency. AM-KNet is trained and evaluated on the nuScenes and View-of-Delft datasets. The results demonstrate improved estimation accuracy and tracking stability compared to the base KalmanNet, narrowing the performance gap with classical Bayesian filters on real-world automotive data.

Robot State Estimation

Robot state estimation infers a robot’s position, motion, orientation, and internal configuration from noisy sensors and physical constraints.

See all 200 papers

Unscented KalmanNet: Structure-Preserving Deep Learning with Calibrated Posterior Uncertainty under Incomplete Physics and Unknown Noise
Aug 4, 2026Minhyeok Ko, Abdollah ShafieezadehKalman Filtering Unscented Kalman Filtering
Department of Civil and Construction Engineering and Management, The University of Texas at Tyler, Tyler, TX, 75799, USA · Department of Civil, Environmental, and Geodetic Engineering, The Ohio State University, Columbus, OH, 43210, USA
Hybrid Adaptive Kalman Filtering for Data-Efficient Joint Tracking and Classification
Jun 1, 2026Jiho Lee, Nisar R. Ahmed, Rebecca RussellKalman Filtering Self-Supervised Learning
Charles Stark Draper Laboratory, Inc., Cambridge, MA · Ann and H. J. Smead Department of Aerospace Engineering Sciences at the University of Colorado, Boulder, Boulder, CO · Draper
ADM-Fusion: Adaptive Deep Multi-Sensor Fusion for Robust Ego-Motion Estimation in Diverse Conditions
Jun 23, 2026Hasan Moughnieh, Ibrahim Ghaddar, Hadi Elham +2Multimodal Robustness Mixture of Experts
Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon. · Department of Mechanical Engineering, American University of Beirut, Beirut, Lebanon.

Aug 4, 2026J/K move · Enter open · S save

Unscented KalmanNet: Structure-Preserving Deep Learning with Calibrated Posterior Uncertainty under Incomplete Physics and Unknown Noise

Minhyeok Ko, Abdollah Shafieezadeh

Department of Civil and Construction Engineering and Management, The University of Texas at Tyler, Tyler, TX, 75799, USA · Department of Civil, Environmental, and Geodetic Engineering, The Ohio State University, Columbus, OH, 43210, USA

Kalman Filtering Unscented Kalman Filtering Covariance Estimation Robot State Estimation Uncertainty Calibration

Open paper AskPDF arXiv

Nonlinear state estimation requires sequentially fusing model-based predictions with noisy measurements. Under imperfect dynamics and unknown, time-varying noise statistics, this fusion can degrade in both accuracy and statistical consistency. Existing learning-aided filters largely treat accuracy and uncertainty estimation separately, limiting their ability to correct model-mismatch-induced bias while retaining an explicit, calibrated posterior covariance. This paper introduces Unscented KalmanNet (UKN), a model-based deep learning architecture that extends the Unscented Kalman Filter (UKF) with learned mechanisms for these two sources of filtering error while preserving explicit posterior covariance propagation. NoiseNet learns time-varying process and measurement covariances as bounded multiplicative corrections to baseline covariances, guaranteeing positive definiteness, while GainNet learns a bounded residual correction to the analytical UKF gain to compensate for model-mismatch-induced bias. A calibration-aware training objective couples state error with posterior covariance and innovation consistency terms through adaptive weighting, jointly optimizing accuracy and calibration. UKN is benchmarked against UKF, KalmanNet, and Bayesian KalmanNet on three synthetic systems and real-flight UZH-FPV data. It achieves the lowest state-estimation error in all four examples and reduces RMSE by 26.4-49.7% compared with UKF in the synthetic cases. Leave-one-sequence-out cross-validation over 11 flights shows 22.4% and 34.3% reductions in mean position and velocity RMSE, respectively. UKN also yields the lowest fold-to-fold variability, with dimension-normalized NEES and empirical coverage closest to nominal values among covariance-reporting filters. These results show that structured learned adaptation improves estimation accuracy while retaining calibrated uncertainty.

Abstract

Robot State Estimation

Explore similar work

Unscented KalmanNet: Structure-Preserving Deep Learning with Calibrated Posterior Uncertainty under Incomplete Physics and Unknown Noise

Hybrid Adaptive Kalman Filtering for Data-Efficient Joint Tracking and Classification

ADM-Fusion: Adaptive Deep Multi-Sensor Fusion for Robust Ego-Motion Estimation in Diverse Conditions