COMET: A coordinated multi-layer runtime monitoring framework for human-supervised AI-enabled safety systems
Authors: Niclas Flehmig, Mary Ann Lundteigen, Shen Yin
Organizations: Department of Mechanical and Industrial Engineering, Norwegian University of Science and Technology, Trondheim, Norway · Department of Cybernetics Engineering, Norwegian University of Science and Technology, Trondheim, Norway
Incorporating AI systems into safety systems requires monitoring to avoid silent faults and ensure safe and reliable operation. Single monitors often do not provide actionable information for human supervisors, who act as the system's adaptive controller. We present COMET, a coordinated multi-layer runtime monitoring framework for human-supervised AI-enabled safety systems in which each monitor is anchored in a different fault mode. Monitor outputs are combined in a coordination layer and mapped by a communication layer to a traffic-light display. In this work, COMET consists of three monitoring layers: signal integrity, physics consistency, and the AI system's competence envelope. The framework is evaluated on an open dataset of compressed-air leaks, where an AI system detects leaks from acoustic emissions, with held-out conditions simulating covariate shift, semantic shift, and sensor faults, events expected during runtime but unknown in their exact appearance. All monitors are frozen after development and tested zero-shot, and the framework is compared against no-monitoring and single-monitor setups. The bare AI system fails silently and confidently under all three shift types, whereas COMET reduces the trusted-wrong-answer rate to zero on covariate-shifted no-leak data and on three of four sensor faults, at a nominal false-alarm rate of ≈1%. Single monitors come close in detection but cannot attribute the fault mode. COMET converts their anonymous detection flags into diagnoses. Residual silent misses under semantic shift define the boundary of passive monitoring. Runtime monitoring frameworks for AI-enabled safety systems must therefore combine quantitative fault detection and isolation with qualitative, human-oriented feedback.
Figures & tables
Figure 1: Distribution of the 5592 recordings (4 microphones × 2 gain settings) across the experimental labels. (a) Binary classification task: no-leak vs. leak recordings; the no-leak band at knob settings 3.5–5.0 (hatched) was excluded from all experiments to keep a clear severity margin between the classes, leaving 4216 usable recordings. (b) Fine-grained leak labels (”no-leak”, ”vent”, ”vent-low”, ”tube”) stacked by recording background; tube leaks and both hydraulic (”hydr”/”hydr-low”) backgrounds were never used for training or calibration and serve exclusively as the held-out semantic and covariate shift conditions. (c) The complementary view: recording backgrounds stacked by leak label, showing that every background contains a balanced mix of leak types.
Figure 2: Concept of the COMET framework: monitors act as local diagnosers anchored in distinct fault modes. The coordination layer fuses their local diagnoses into a taxonomy state, which the communication layer presents to the human supervisor as a traffic-light display with action recommendations.
State
Meaning and required action
T1-confirmed
AI system detects leak, physics (L2) consistent → trust the detection
T1-unconfirmed
AI system detects leak, physics (L2) abstain → report, but mark as unverified
T2 (hidden fault)
AI system detects no leak, physics (L2) finds a leak signature (consistent) → investigate; the AI system may be blind here
T3 (unverified)
AI system detects leak, physics (L2) inconsistent → investigate possible novel leak or false alarm; detection lacks physics support
T3 (unexplained)
AI system detects no leak, physics (L2) inconsistent → investigate; unexplained physical anomaly
T4 (deferred)
Input outside the competence envelope (L3) → the AI system is extrapolating; defer to the operator
Table 1: Decision taxonomy: system states and their operational meaning.
T2, T3 (unverified), T3 (unexplained), T4 → operator decision required
Red
T2 with concurrent L3 flag → immediate attention, fallback policy
Table 2: Traffic-light operator display: urgency mapping of the taxonomy states.
verdict
mic1
mic2
mic3
mic4
pooled
consistent
57.5 %
33.8 %
62.5 %
35.0 %
47.2 %
quiet
5.0 %
0 %
16.3 %
0 %
5.3 %
inconsistent
0 %
0 %
0 %
0 %
0 %
abstain
26.3 %
66.2 %
21.2 %
65.0 %
44.7 %
blocked (L1)
11.3 %
0 %
0 %
0 %
2.8 %
Table 3: L2 physics-layer verdict distribution (%) on nominal leak validation data, per microphone and pooled. ”Blocked” denotes files on which an L1 integrity rule fired. On non-injected conditions, blocking is dominated by the microphone-1 clipping-rule defect.
L2 verdict
pooled
mic1
mic2
mic3
mic4
abstain
64.8 %
50.7 %
70.8 %
63.9 %
73.6 %
quiet
22.6 %
35.4 %
28.5 %
0.7 %
25.7 %
consistent
7.5 %
0 %
0 %
29.9 %
0 %
inconsistent
1.4 %
0 %
0 %
5.6 %
0 %
blocked (L1)
3.8 %
13.9 %
0.7 %
0 %
0.7 %
Table 4: L2 physics-layer verdict distribution (%) on tube-leak files, pooled and per microphone.
Figure 3: Layer-wise shift signatures. Mean z-standardized k-NN distance profile (conv1–dense) per condition, averaged over microphones. The covariate shift (hydraulic backgrounds) peaks at the early convolutional layers, the semantic shift (tube leaks) at the dense layer, and the nominal control remains at ≈0 throughout. This confirms that shift type localizes by network depth, and explaining the complementary sensitivities of the dense k-NN and the layer-profile monitor.
Figure 4: Cascade verdicts per condition pooled over microphones. (a) Distribution of taxonomy states: nominal no-leak data is dominated by all-clear and nominal leaks by trusted detections (T1-confirmed/unconfirmed). The covariate shift (hydraulic backgrounds, hL/hH) splits between all-clear, integrity override, and deferred verdicts (T4). The semantic shift (tube leak) is dominated by T4. Injected faults (inj.) are intercepted by the L1 override (dropout, flat-line, offset) except clipping, which exposes the integrity layer’s blind spot. (b) The corresponding traffic-light distribution a human supervisor would see: nominal operation is overwhelmingly green, shifts escalate to yellow/orange, and sensor faults are held at yellow. No wrong condition is presented as trusted green, except for the clipping blind spot and the residual tube silent misses.
Figure 5: Monitoring ablation ladder pooled over microphones. (a) Trusted-wrong-answer rate per condition for the bare AI system (S0), single-monitor arms (S1a: +MSP, S1b: +L2, S1c: +L3), and COMET (S2). (b) Missed-leak coverage on tube leaks the AI system called negative (n =174). (c) Nominal cost: false-alarm rate of each arm on nominal no-leak data. COMET’s 4.4% is driven by the microphone-1 clipping-rule defect.
S0
S1a
S1b
S1c
S2
bare
+MSP
+L2
+L3
COMET
Nominal data
no-leak
0.0
0.0
0.0
0.0
0.0
leak
6.9
5.0
6.9
4.7
4.7
Covariate shift (hydraulic background)
hydr-low no-leak
0.0
0.0
0.0
0.0
0.0
Table 5: Ablation ladder: trusted-wrong-answer (TWA) rate (%) per condition and system arm, pooled over the four microphones. S0: bare AI system; S1a: +MSP confidence; S1b: +L2 physics; S1c: +L3 k-NN; S2: COMET. Bottom rows: silent-miss share on tube leaks the AI system did not detect ( n=174 ) and nominal false-alarm cost (nominal no-leak). Bold: best per row.
Artificial intelligence safety studies how to prevent harmful, unreliable, or uncontrollable behavior and manage risks across AI systems and their deployment.