MuViS: Multimodal Virtual Sensing Benchmark

Jens U. Brandt; Noah C. Puetz; Jobel Jose George; Niharika Vinay Kumar; Elena Raponi; Marc Hilbert; Thomas Bäck; Thomas Bartz-Beielstein

eess.SPMar 13, 2026

MuViS: Multimodal Virtual Sensing Benchmark

Authors: Jens U. Brandt, Noah C. Puetz, Jobel Jose George, Niharika Vinay Kumar, Elena Raponi, Marc Hilbert, Thomas Bäck, Thomas Bartz-Beielstein

Organizations: TH Köln, Germany. · Leiden University, Netherlands. · Toyota Racing, Germany.

Abstract

Virtual sensing aims to infer hard-to-measure quantities from accessible measurements and is central to perception and control in physical systems. Despite rapid progress from first-principle and hybrid models to modern data-driven methods research remains siloed, leaving no established default approach that transfers across processes, modalities, and sensing configurations. We introduce MuViS, a domain-agnostic benchmarking suite for multimodal virtual sensing that consolidates diverse datasets into a unified interface for standardized preprocessing and evaluation. Using this framework, we benchmark established approaches spanning gradient-boosted decision trees and deep neural network (NN) architectures, and show that none of these provides a universal advantage, underscoring the need for generalizable virtual sensing architectures. MuViS is released as an open-source, extensible platform for reproducible comparison and future integration of new datasets and model classes.

Figures & tables

Fig. 1: We evaluate standard ML architectures across diverse sensing domains, where models must map multimodal time-series inputs ( $x1,x2$ ) to scalar virtual measurements ( $y$ ).

Fig. 2: Overview of the six benchmark datasets. Each sub-panel displays a distinct virtual sensing task, showcasing the diversity in target distributions (left) and temporal feature characteristics (right). The collection spans varied feature dimensions ( $D$ ), sequence lengths ( $T$ ), and sampling intervals ( $\Delta t$ ), reflecting the heterogeneous nature of real-world sensing applications.

Dataset	XGBoost	CatBoost	MLP	ResNet1D	LSTM	Transformer
Lat. Velocity: Monterey	$0.120_{[0.118,0.122]}$	$0.109_{[0.107,0.110]}$	$0.131_{[0.129,0.133]}$	$0.110_{[0.109,0.112]}$	$\mathbf{0.108_{[0.107,0.110]}}$	$0.119_{[0.117,0.120]}$
Lat. Velocity: Targa ’13	$0.098_{[0.095,0.100]}$	$0.087_{[0.085,0.090]}$	$0.082_{[0.080,0.085]}$	$0.071_{[0.069,0.073]}$	$\mathbf{0.070_{[0.068,0.072]}}$	$0.103_{[0.100,0.107]}$
Lat. Velocity: Targa ’14	$\mathbf{0.077_{[0.076,0.078]}}$	$0.080_{[0.079,0.081]}$	$0.089_{[0.088,0.091]}$	$0.079_{[0.078,0.081]}$	$0.078_{[0.077,0.080]}$	$0.099_{[0.097,0.100]}$
PM 10	$\mathbf{91.642_{[86.170,98.755]}}$	$92.091_{[86.628,99.403]}$	$95.889_{[90.846,102.116]}$	$95.928_{[90.383,102.590]}$	$99.291_{[93.251,106.685]}$	$103.852_{[97.889,110.958]}$
PM 2.5	$60.983_{[55.684,66.646]}$	$61.366_{[56.293,66.877]}$	$60.477_{[55.777,65.810]}$	$62.994_{[58.662,67.944]}$	$\mathbf{59.726_{[55.845,64.412]}}$	$65.600_{[60.266,71.800]}$
Tire Temp.	$3.758_{[3.516,3.968]}$	$3.840_{[3.606,4.071]}$	$3.173_{[2.943,3.437]}$	$3.401_{[3.184,3.590]}$	$\mathbf{2.447_{[2.298,2.608]}}$	$3.653_{[3.372,3.987]}$

TABLE I: Predictive performance comparison across virtual sensing tasks. Results report Mean RMSE with 95% bootstrap confidence intervals

[\text{Lower},\text{Upper}]

obtained from 200 iterations. Bold values indicate the best performing model per dataset.

Fig. 3: Critical distance diagram. The non-significant Friedman test indicates no statistically superior architecture.

Virtual Sensing

Virtual sensing estimates unavailable or costly-to-measure quantities from other sensor data, physical models, and learned relationships.

See all 23 papers

Reliable Virtual Sensing: A Multi-Domain Benchmark for Robustness Under Sensor Failures
Sep 16, 2026Jens U. Brandt, Noah C. Puetz, Alexander Windmann +4Cyber-Physical Systems Virtual Sensing
TH Köln · Leiden University · Helmut Schmidt University +1
MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation
Jun 16, 2026Xingyuming Liu, Ruichun Ma, Heyu Guo +7Language-Conditioned Robot Manipulation Robotic Manipulation
School of Computer Science, Peking University · 2Microsoft Research Asia · *Work done during internship at Microsoft Research Asia. +2
VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark
Aug 19, 2025Vy Tuong Dang, An Vo, Emilio Villa-Cueva +4VLM Evaluation Multimodal Reasoning
KAIST · MBZUAI

Sep 16, 2026J/K move · Enter open · S save

Reliable Virtual Sensing: A Multi-Domain Benchmark for Robustness Under Sensor Failures

Jens U. Brandt, Noah C. Puetz, Alexander Windmann, Marc Hilbert+3

TH Köln · Leiden University · Helmut Schmidt University+1

Cyber-Physical Systems Virtual Sensing

Open paper AskPDF arXiv

Virtual sensing, the estimation of hard-to-measure quantities from available sensor measurements, is a critical enabler for control and monitoring in cyber-physical systems. However, when sensors fail, learning-based predictors can produce physically implausible estimates that propagate to system-level failures. We argue that real-world deployment demands robustness and introduce MuViS-C, the first multi-domain benchmark of robustness against common sensor failures in learning-based virtual sensing. Building on an existing nominal-performance benchmark and established corruption taxonomies, it covers ten sensor failure modes, from subtle drifts to catastrophic signal dropouts, at multiple severities. These are paired with complementary robustness measures capturing average error under corruption, relative degradation, and worst-case fragility. Across nine datasets from six domains, we benchmark six architectures spanning gradient-boosted trees and the major inductive biases for sequence modeling: convolution, recurrence, attention, and MLP-mixing. On the attention-based architecture, we further probe three robustification strategies. We find that (i) every model degrades substantially under corruption, becoming worse than a naïve predictor on at least one corruption setting, (ii) gradient-boosted tree ensembles achieve strong robustness, and (iii) dedicated robustification closes the gap between the attention-based architecture and the most robust models, though each strategy hurts nominal performance. The benchmark's multi-domain design proves essential, as model rankings shift across datasets, and no single domain captures the full robustness picture. MuViS-C is open-source and extensible to new datasets, failure modes, measures, and models.

Abstract

Virtual Sensing

Explore similar work

Reliable Virtual Sensing: A Multi-Domain Benchmark for Robustness Under Sensor Failures

MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation

VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark