Operational Range Bounding in Spectroscopy: A Safety Cage Framework for Machine Learning Models

Nikki Grens; Luís F. Simões; Kai Hou Yip; Theresa Lueftinger

cs.LGSep 11, 2026

Operational Range Bounding in Spectroscopy: A Safety Cage Framework for Machine Learning Models

Authors: Nikki Grens, Luís F. Simões, Kai Hou Yip, Theresa Lueftinger

Organizations: ML Analytics, 2660-329 Lisbon, Portugal · Department of Space Engineering, Delft University of Technology, 2600 AA Delft, The Netherlands · Department of Physics, King’s College London, University of London, WC2R 2LS, London, United Kingdom · European Space Research and Technology Centre (ESTEC), European Space Agency (ESA), 2201 AZ Noordwijk, The Netherlands

Abstract

Ensuring the reliability of black-box machine learning models in safety-critical space missions remains a significant challenge, particularly when ground-truth is unavailable for validation. Although machine learning models offer a powerful means to augment standard pipelines by extracting transmission spectra from complex exoplanetary light curves, their susceptibility to unmodelled instrument anomalies, stellar activity, and domain shifts introduces unquantified risks. This study evaluates a modular safety cage architecture that operates as a parallel monitoring layer to assess the validity of a prediction without modifying the underlying estimator. By monitoring different runtime indicators, including uncertainty quantification, out-of-domain detection, and influence functions, the framework constrains the model's operational domain to a verified region. A controlled evaluation is conducted under both in-domain and cross-domain conditions, using datasets from the 2019 and 2021 editions of the Ariel Data Challenges. The results reveal that model failure is multifaceted and that no single indicator captures all failure modes, demonstrating the need for indicator fusion. The application of safety-driven rejection strategies shows that a modest 20% reduction in data coverage results in error reductions between 45% and 65% across different domains and evaluation metrics. Using a formalised coverage-risk framework, a systematic analysis of indicator combinations is performed to identify configurations that maximise risk-ranking accuracy and optimise the trade-off between data coverage and scientific performance. Safety cages provide a transparent mechanism for detecting unreliable predictions and represent a critical step towards the safe deployment of data-driven models in scientific applications, such as astrophysics, where ground truth is seldom available.

Figures & tables

Figure 1: Block diagram of the safety cage architecture. The predictive model operates alongside a parallel supervisory layer that evaluates runtime safety indicators from the input, internal model states, outputs, and system observables. A gating mechanism then accepts or rejects each prediction based on whether it falls within the validated operational domain. Schematic adapted from ECSS Secretariat (2024) .

Figure 2: Schematic of aleatoric and epistemic uncertainty in machine learning models. The dashed purple line represents a theoretical ground truth model within the hypothesis space. While aleatoric uncertainty remains a constant property of data noise, epistemic uncertainty in segment four, characterised by high model variance, is reducible as the training data expands.

	Fixed (both editions)		Varies across editions
Dataset	Stellar & orbital	Observation model	Noise model	Atmospheric composition
ADC19	MCS (same)	Spots $+$ LDE (same)	Photon noise (ArielRad-like)	H 2 O, CH 4
ADC21	MCS (same)	Spots $+$ LDE (same)	Photon $+$ $1/f$ $+$ persistence	H 2 O, CH 4 , CO 2

Table 1: Simulation components for the ADC19 and ADC21 datasets. Components are separated into those held fixed across both editions (shared simulation basis) and those that differ, constituting the sources of distribution shift between datasets. Stellar spots and limb darkening are present in both editions as part of the forward model, defining the core denoising challenge: recovering the transmission spectrum from spectroscopic light curves contaminated by both instrumental noise and stellar activity. LDE: limb darkening effect, modelled via Claret’s 4-coefficient law ( Claret, 2000 ) . MCS: Ariel Mission Candidate Sample.

Figure 3: Observed transit light curves (left) and corresponding transmission spectrum (right) for an exoplanet. There are 55 light curves that correspond to 55 wavelength channels that translate to one transmission spectrum.

Figure 4: Validation of the influence function error proxy. Relationship between the safety indicator scores (derived via cross-validation on the ADC21 dataset) and the true predictive performance of the ELM model. The degradation in model performance, shown by higher SAM errors, as safety scores decrease, demonstrates the framework’s effectiveness in ranking samples by risk.

Figure 5: Operational coverage–error curves for the influence function safety indicator. Both panels show the framework performance for in-domain validation on the ADC21 dataset, evaluated using the SAM error metric. The left panel illustrates the indicator’s ability to reject higher-error samples as coverage is reduced progressively. The right panel includes the theoretical worst-case baseline, in which the highest-error samples are accepted first, thereby highlighting the gain achieved by the safety cage architecture.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
Ridge Regression	0.136166	0.131880	0.017489	0.022598
ELM	0.054703	0.069541	0.012931	0.017966

Table 2: Performance comparison of Ridge Regression and ELM architectures for RMSPE and SAM in the cross-validation setting when training on the ADC21 dataset and testing on both the ADC21 and the ADC19 datasets. These evaluations correspond to a safety cage with 100% coverage (no sample filtering).

Indicator	Best Variant	Underlying Mechanism	Operational Meaning of High Score
Out-of-Distribution	Isolation Forest on $X_{\text{shap}}$	Isolation Forest applied to 30-D PCA-reduced feature attributions (SHAP values).	Feature importance logic aligns with safe, in-domain training distributions.
Uncertainty Quantification	Width	Conformalised Quantile Regression via Multi-Output LightGBM on error PCA space.	High model certainty shown by narrow predictive intervals across spectral channels.
Ensemble Consistency	Relative Uncertainty	Relative ensemble variance benchmarked against typical training phase disagreement.	High model consensus with live relative variance not exceeding historical training baselines.
Adversarial Attack	Risk	A perturbation cloud based on MCS error profiles to evaluate relative prediction drift and variance.	High localised model robustness with the model having low sensitivity to small input perturbations.
Influence Function	Error Proxy	Variance estimation of training set influence via the Infinitesimal Jackknife.	Low reliance on poorly modelled training data.
Activation Pattern Monitoring	$\text{Mahalanobis }Z_{s}$	Mahalanobis distance calculation on 50-D PCA-reduced standardised hidden layer activations.	Internal neural network states match normal, expected activation pattern.

Table 3: Comprehensive summary of the selected safety cage monitoring framework. For each of the six indicator families, the table outlines the chosen metric, its underlying computational mechanism, and the operational interpretation of its safety score.

Figure 6: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on fused safety scores generated through various generalised mean strategies and LogSumExp, comparing internal (ADC21) and cross-domain (ADC19) validation.

Figure 7: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on fused safety scores generated through various generalised mean strategies and LogSumExp, comparing internal (ADC21) and cross-domain (ADC19) validation.

Figure 8: Illustration of RMSPE evolution as samples are admitted to the model’s operational range based on the best safety indicators for each setup, as well as the best fusion strategy. The left panel shows internal validation on ADC21, while the right panel demonstrates cross-domain validation on ADC19.

Figure 9: Illustration of SAM evolution as samples are admitted to the model’s operational range based on the best safety indicators for each setup, as well as the best fusion strategy. The left panel shows internal validation on ADC21, while the right panel demonstrates cross-domain validation on ADC19.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
Mean_p-inf	-0.605	-0.597	-0.672	-0.638
Mean_p-3	-0.630	-0.625	-0.694	-0.661
Mean_p-1	-0.660	-0.664	-0.718	-0.691
Mean_p0	-0.687	-0.699	-0.735	-0.714
Mean_p1	-0.703	-0.716	-0.738	-0.716
Mean_p3	-0.707	-0.714	-0.728	-0.700

Table 4: Spearman correlation coefficients between prediction error metrics and fused safety scores calculated using various generalised mean strategies and LogSumExp across the ADC21 and ADC19 datasets.

Figure 10: Best Spearman correlation per aggregation method as a function of indicator combination size for RMSPE when training and testing on ADC21.

Figure 11: Average Spearman correlation per aggregation method as a function of indicator combination size for RMSPE when training and testing on ADC21.

Figure 12: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on different possible best combinations according to the previous analyses, comparing internal (ADC21) and cross-domain (ADC19) validation.

Figure 13: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on different possible best combinations according to the previous analyses, comparing internal (ADC21) and cross-domain (ADC19) validation.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 14: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on their safety scores for Anomaly Detection using X, y, and

\text{X}_{\text{shap}}

. The x-axis (Coverage) represents the iterative process of starting with only the safest data points and progressively lowering the safety threshold until 100% of the test data has been included. The left panel displays results for a model trained and tested on ADC21 data, while the right panel shows results for a model trained on ADC21 and tested on the ADC19 dataset.

Figure 15: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on their safety scores for Anomaly Detection using X, y, and

\text{X}_{\text{shap}}

, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
$X$	-0.512	-0.537	-0.489	-0.492
$Y$	-0.488	-0.484	-0.515	-0.518
$X_{shap}$	-0.548	-0.576	-0.498	-0.504

Appendix

Table 5: Spearman correlation coefficients between the proposed safety indicators (X, y, and

\text{X}_{\text{shap}}

) for Anomaly Detection and the prediction error metrics (RMSPE and SAM). Results are shown for internal validation (ADC21) and cross-domain validation (ADC19). Negative values indicate that higher safety scores effectively correspond to lower prediction errors.

Figure 16: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on their safety scores for Uncertainty Quantification using Width, SNR, and DLE, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Figure 17: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on their safety scores for Uncertainty Quantification using Width, SNR, and DLE, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
Width	-0.667	-0.674	-0.799	-0.777
SNR	-0.228	-0.278	-0.542	-0.580
DLE	-0.582	-0.622	-0.687	-0.707

Appendix

Table 6: Spearman correlation coefficients between the proposed safety indicators for Uncertainty Quantification and the prediction error metrics, following the experimental setup in Table 5 .

Figure 18: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on their safety scores for Ensemble Consistency using Standard Deviation, Coefficient of Variation, and Relative Uncertainty, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Figure 19: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on their safety scores for Ensemble Consistency using Standard Deviation, Coefficient of Variation, and Relative Uncertainty, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
Standard Deviation	-0.256	-0.228	0.060	0.075
Coefficient of Variation	-0.632	-0.600	-0.531	-0.480
Relative Uncertainty	-0.629	-0.600	-0.533	-0.482

Appendix

Table 7: Spearman correlation coefficients between the proposed safety indicators for Ensemble Consistency and the prediction error metrics, following the experimental setup in Table 5 .

Figure 20: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on their safety scores for Adversarial Attack using Drift, Risk, and Percentile, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Figure 21: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on their safety scores for Adversarial Attack using Drift, Risk, and Percentile, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
Drift	-0.234	-0.209	-0.178	-0.167
Risk	-0.414	-0.391	-0.505	-0.479
Percentile	-0.131	-0.136	-0.164	-0.165

Appendix

Table 8: Spearman correlation coefficients between the proposed safety indicators for Adversarial Attack and the prediction error metrics, following the experimental setup in Table 5 .

Figure 22: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on their safety scores for Influence Functions using Percentage Influential, Error Proxy, and Distance

X

, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Figure 23: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on their safety scores for Influence Functions using Percentage Influential, Error Proxy, and Distance

X

, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
Percentage Influential	-0.578	-0.575	-0.612	-0.540
Error Proxy	-0.662	-0.670	-0.724	-0.678
Distance X	0.041	0.018	-0.032	-0.081

Appendix

Table 9: Spearman correlation coefficients between the proposed safety indicators for Influence Functions and the prediction error metrics, following the experimental setup in Table 5 .

Figure 24: Illustration of RMSPE evolution as samples are admitted to the model’s operational range, based on their safety scores for Activation Pattern Monitoring using Mahalanobis Z, Mahalanobis Zs, and Isolation Forest Zs, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Figure 25: Illustration of SAM evolution as samples are admitted to the model’s operational range, based on their safety scores for Activation Pattern Monitoring using Mahalanobis Z, Mahalanobis Zs, and Isolation Forest Zs, following the approach in Figure 14 . The x-axis represents the progression from the safest data points to 100% test data inclusion.

Indicator	RMSPE 21	RMSPE 19	SAM 21	SAM 19
Mahalanobis Z	-0.556	-0.583	-0.580	-0.572
Mahalanobis Zs	-0.556	-0.584	-0.581	-0.572
Isolation Forest Zs	-0.539	-0.563	-0.579	-0.569

Appendix

Table 10: Spearman correlation coefficients between the proposed safety indicators for Activation Pattern Monitoring and the prediction error metrics, following the experimental setup in Table 5 .

Panel of Figure 26 — Figure 26: Spearman correlation between aggregated risk scores and RMSPE across all indicator combinations and aggregation methods. Left: training and testing on ADC21. Right: training on ADC21 and testing on ADC19. More negative values indicate better alignment between the risk indicator and true error.

Panel of Figure 27 — Figure 27: Spearman correlation between aggregated risk scores and SAM across all indicator combinations and aggregation methods. Left: training and testing on ADC21. Right: training on ADC21 and testing on ADC19. More negative values indicate better alignment between the risk indicator and true error.

Figure 28: Best Spearman correlation per aggregation method as a function of indicator combination size for RMSPE when training on ADC21 and testing on ADC19.

Figure 29: Average Spearman correlation per aggregation method as a function of indicator combination size for RMSPE training on ADC21 and testing on ADC19.

Figure 30: Best Spearman correlation per aggregation method as a function of indicator combination size for SAM when training and testing on ADC21. More negative values indicate better alignment between the risk indicator and true error.

Figure 31: Average Spearman correlation per aggregation method as a function of indicator combination size for SAM when training and testing on ADC21.

Figure 32: Best Spearman correlation per aggregation method as a function of indicator combination size for SAM when training on ADC21 and testing on ADC19.

Figure 33: Average Spearman correlation per aggregation method as a function of indicator combination size for SAM when training on ADC21 and testing on ADC19.

Scientific ML

Scientific machine learning combines data-driven models with scientific knowledge to analyze, predict, and simulate complex systems.

See all 124 papers

Traceable Spectral Inference via Influence Functions: Efficient Data Attribution and Error Proxies for the Ariel Mission
Aug 24, 2026Nikki Grens, Luís F. Simões, Kai Hou Yip +1Influence Functions Training Data Attribution
ML Analytics, Lisbon, Portugal · Department of Space Engineering, Delft University of Technology, Delft, The Netherlands · Department of Physics, King’s College London, London, UK +1
CUBICS: Situation-aware performance estimation for safety-relevant ML components
Aug 17, 2026Benjamin Herd, Jessica Kelly, Mario TrappAI Safety
Fraunhofer Institute for Cognitive Systems IKS Garching, Germany · Technical University of Munich Garching, Germany
Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing
Apr 29, 2026Mathieu Dario, Florent Chenevier, Kévin Delmas +2
LAAS-CNRS, Toulouse, France · University of Toulouse, France · Thales, Toulouse, France +3

Aug 24, 2026J/K move · Enter open · S save

Traceable Spectral Inference via Influence Functions: Efficient Data Attribution and Error Proxies for the Ariel Mission

Nikki Grens, Luís F. Simões, Kai Hou Yip, Theresa Lueftinger

ML Analytics, Lisbon, Portugal · Department of Space Engineering, Delft University of Technology, Delft, The Netherlands · Department of Physics, King’s College London, London, UK+1

Influence Functions Training Data Attribution Scientific ML

Open paper AskPDF arXiv

Interpretability is critical for machine learning models deployed in scientific space missions such as ESA's Ariel, where ground truth is unavailable during operations and physical plausibility must be assessed. While most explainable AI methods focus on feature attribution, this work investigates training data attribution through influence functions and introduces three key contributions for operational spectroscopy pipelines. First, influence is reformulated in terms of prediction rather than loss, enabling label-free deployment. Second, by leveraging the closed-form ridge solution of an Extreme Learning Machine, infinitesimal prediction influence is efficiently computed. Third, an influence-based conservative error proxy is derived by propagating training residuals through the influence sensitivities. Evaluated against simulated spectra, the proposed proxy correlates strongly with scale and shape-based spectral errors. Furthermore, influence functions enable the identification of the most influential samples and the approximation of the most harmful ones. Together, these results suggest that this approach can serve as an operational framework for scientific machine learning.

Abstract

Scientific ML

Explore similar work

Traceable Spectral Inference via Influence Functions: Efficient Data Attribution and Error Proxies for the Ariel Mission

CUBICS: Situation-aware performance estimation for safety-relevant ML components

Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing