CUBICS: Situation-aware performance estimation for safety-relevant ML components

Benjamin Herd; Jessica Kelly; Mario Trapp

cs.AIAug 17, 2026

CUBICS: Situation-aware performance estimation for safety-relevant ML components

Authors: Benjamin Herd, Jessica Kelly, Mario Trapp

Organizations: Fraunhofer Institute for Cognitive Systems IKS Garching, Germany · Technical University of Munich Garching, Germany

Abstract

Machine learning (ML) is a key technology driving innovation today, but ensuring ML safety remains a major challenge for safety-related applications. A promising idea is to build proven-in-use arguments from field data, e.g. by running ML components (MLCs) in shadow mode or within safety envelopes so that their outputs can be monitored as 'safe probes' without affecting safety. These probes can then be used to build a statistical argument about field performance in a Bayesian way. However, many Bayesian field-data approaches in safety engineering model failures as a simple Bernoulli (or binomial) process with a single global failure probability and i.i.d. trials, which is rarely adequate for MLCs whose performance depends strongly on context. Statistical evidence is also about coverage of relevant situations, including edge cases, and building a single integrated statistical model for the entire system is usually not feasible. To address these challenges, this paper introduces CUBICS, a context-modular framework for per-component, situation-aware performance estimation of safety-relevant ML components. CUBICS partitions the operational design domain into situations and, for each safety-relevant component, defines a set of situation-specific assumptions and probabilistic guarantees that are represented and updated in a Bayesian manner using Subjective Logic (SL). By combining these guarantees with beliefs about how often each situation occurs, CUBICS derives an overall risk estimate for each component without requiring a monolithic system-level statistical model, and thus provides a building block for modular, field-data based safety assurance.

Figures & tables

Panel of Figure 1 — Fig. 1: Overview of the Unified CUBICS methodology over a simplified operational design domain. Subjective context opinions over Rain , Wind , and Time of Day make up eight situations (from $s_{0}$ to $s_{7}$ ). Each situation establishes a unique safety contract ( $G\mid s_{i}$ ), which are ultimately aggregated into the overarching global guarantee $G\parallel S$ .

Panel of Figure 2 — Fig. 2: The fractional evidential update mechanism. SL situational opinions $\omega_{s_{i}}$ provide a probabilistic weighting signal $E(s_{i})$ to incoming field observations. Binary evidence is accumulated fractionally into Dirichlet parameters $(\alpha,\beta)$ , thereby refining the conditional guarantee $\omega_{G\mid s_{i}}$ as more data is gathered.

Fig. 3: Progression of beta distributions across situational guarantees. Dotted red line represents true failure probability across situations. The shaded beta distributions represent the final conditional guarantees.

Scenario	TP ( $r$ )	FN ( $s$ )	Bel. ( $b$ )	Disbel. ( $d$ )	Unc. ( $u$ )	Exp. Prob ( $E$ )
Global Pooled	16 101	8 549	0.6531	0.3468	0.0001	0.6532
Clear + Daytime	2 758	1 484	0.6499	0.3497	0.0005	0.6501
Rainy + Night	218	157	0.5782	0.4164	0.0053	0.5809
Foggy + Dawn/Dusk	5	17	0.2083	0.7083	0.0833	0.2500

TABLE I: CUBICS opinions for the person class. The global pooled estimate masks the performance collapse and high uncertainty in adverse scenarios such as Foggy + Dawn/dusk .

Fig. 4: Beta PDFs for the person class. The global pooled estimate’s narrow peak (black) conceals the performance gap between a benign scenario (green) and an adverse one (red).

Fig. 5: Marginal (top, blue) vs. context-agnostic pooled guarantee (bottom, red). The marginal distribution is wider and reflects the compositional uncertainty that pooling conceals.

		Uncertainty Mass ( $u$ )
Class	$\mathbf{E[x]}$	Marginal	Pooled	Uncertainty Ratio
Person	0.6532	$1.74\times 10^{-3}$	$8.10\times 10^{-5}$	21.4 $\times$
Bike	0.5695	$1.52\times 10^{-2}$	$1.00\times 10^{-3}$	15.2 $\times$
Car	0.7775	$2.40\times 10^{-4}$	$1.00\times 10^{-5}$	24.6 $\times$
Truck	0.6575	$4.82\times 10^{-3}$	$2.30\times 10^{-4}$	21.0 $\times$

TABLE II: Marginal vs. global pooled guarantee across classes. Both share the same expected value; the SL marginal exposes

{\sim}15\text{--}25\times

more uncertainty depending on the class.

Prior ( $a$ )	Clear Day ( $N=4{,}242$ ) $E(Safe\mid s_{\text{cd}})$	Foggy Dawn ( $N=22$ ) $E(Safe\mid s_{\text{fd}})$
$0.1$	$0.6499\ (u=0.0005)$	$0.2167\ (u=0.0833)$
$0.5$	$0.6501\ (u=0.0005)$	$0.2500\ (u=0.0833)$
$0.9$	$0.6503\ (u=0.0005)$	$0.2833\ (u=0.0833)$

TABLE III: RQ3: Sensitivity to the Base Rate Prior (

a

) in Data-Rich vs. Data-Scarce ODDs

Fig. 6: Sensitivity of situation-specific expected guarantee $E(Safe\mid s_{[\text{sn}/\text{fd}]})$ to context misclassification. Increasing amounts of Clear Day evidence are incorrectly assigned to (a) a data-rich target ( Snowy Night ) and (b) a data-scarce target ( Foggy Dawn ), gradually inflating guarantees and reducing the distinction between situations.

AI Safety

Artificial intelligence safety studies how to prevent harmful, unreliable, or uncontrollable behavior and manage risks across AI systems and their deployment.

See all 134 papers

Operational Range Bounding in Spectroscopy: A Safety Cage Framework for Machine Learning Models
Sep 11, 2026Nikki Grens, Luís F. Simões, Kai Hou Yip +1Scientific ML Selective Prediction
ML Analytics, 2660-329 Lisbon, Portugal · Department of Space Engineering, Delft University of Technology, 2600 AA Delft, The Netherlands · Department of Physics, King’s College London, University of London, WC2R 2LS, London, United Kingdom +1
A Subjective Logic-based method for runtime confidence updates in safety arguments
May 21, 2026Benjamin Herd, Jessica Kelly, Clarissa Heinemann +1AI Assurance Runtime Assurance
Fraunhofer Institute for Cognitive Systems (IKS) Munich, Germany
Uncertainty Evaluation: Bridging Machine Learning and Measurement Science
Oct 13, 2026Dario Petri, Luca MariUncertainty Quantification Uncertainty Decomposition
Department of Industrial Engineering of the Università di Trento, Trento, Italy · Department of Philosophy of the Università Cattolica di Milano, Milano, Italy

Sep 11, 2026J/K move · Enter open · S save

Operational Range Bounding in Spectroscopy: A Safety Cage Framework for Machine Learning Models

Nikki Grens, Luís F. Simões, Kai Hou Yip, Theresa Lueftinger

ML Analytics, 2660-329 Lisbon, Portugal · Department of Space Engineering, Delft University of Technology, 2600 AA Delft, The Netherlands · Department of Physics, King’s College London, University of London, WC2R 2LS, London, United Kingdom+1

Scientific ML Selective Prediction Black-Box Model Auditing OOD Detection

Open paper AskPDF arXiv

Ensuring the reliability of black-box machine learning models in safety-critical space missions remains a significant challenge, particularly when ground-truth is unavailable for validation. Although machine learning models offer a powerful means to augment standard pipelines by extracting transmission spectra from complex exoplanetary light curves, their susceptibility to unmodelled instrument anomalies, stellar activity, and domain shifts introduces unquantified risks. This study evaluates a modular safety cage architecture that operates as a parallel monitoring layer to assess the validity of a prediction without modifying the underlying estimator. By monitoring different runtime indicators, including uncertainty quantification, out-of-domain detection, and influence functions, the framework constrains the model's operational domain to a verified region. A controlled evaluation is conducted under both in-domain and cross-domain conditions, using datasets from the 2019 and 2021 editions of the Ariel Data Challenges. The results reveal that model failure is multifaceted and that no single indicator captures all failure modes, demonstrating the need for indicator fusion. The application of safety-driven rejection strategies shows that a modest 20% reduction in data coverage results in error reductions between 45% and 65% across different domains and evaluation metrics. Using a formalised coverage-risk framework, a systematic analysis of indicator combinations is performed to identify configurations that maximise risk-ranking accuracy and optimise the trade-off between data coverage and scientific performance. Safety cages provide a transparent mechanism for detecting unreliable predictions and represent a critical step towards the safe deployment of data-driven models in scientific applications, such as astrophysics, where ground truth is seldom available.

Abstract

AI Safety

Explore similar work

Operational Range Bounding in Spectroscopy: A Safety Cage Framework for Machine Learning Models

A Subjective Logic-based method for runtime confidence updates in safety arguments

Uncertainty Evaluation: Bridging Machine Learning and Measurement Science