cs.LGNov 15, 2026

Evaluating Near-Perfect Accuracy Claims in Machine-Learning-Based Leak Detection for Two-Phase Pipelines: A Condition-Blocked Audit

Authors: Issah N. Suleiman, Sormeh Serpoosh, Nadine Elkholy, Hicham Ferroudji, Mohammad Azizur Rahman, Matthew Hamilton

Organizations: Memorial University of Newfoundland, St. John’s, Newfoundland and Labrador, Canada · College of Science and Engineering, Hamad Bin Khalifa University, Education City, Doha, Qatar · Laboratory of Petroleum Equipment’s Reliability and Materials, Faculty of Hydrocarbons and Chemistry, University M’Hamed Bougara of Boumerdes, Boumerdes, Algeria

Abstract

Reliable leak detection in gas-liquid two-phase pipelines is important for process safety, yet machine-learning studies frequently report near-perfect performance using evaluation protocols that may not represent deployment to unseen operating conditions. This study audits a pressure-based leak-detection model using condition-blocked leave-one-condition-out validation, in which the classifier is tested on a flow condition excluded from training. Accuracy decreased from 93.3% under within-condition evaluation to 59.1% across fifteen unseen-condition tests. Ablation analysis showed that most of this difference arose from the data-splitting protocol, with a smaller contribution from encoding operating conditions as model inputs. Under condition-blocked evaluation, the false-alarm and missed-detection rates increased to 34.1% and 10.2%, respectively, demonstrating that near-perfect reported accuracy did not translate into reliable operation at unseen flow conditions. The reduction persisted across all evaluated classifiers, indicating that it was caused by the evaluation design rather than a particular model architecture. Grouped evaluation of an independent acoustic pipeline dataset produced a comparable reduction toward chance-level performance. These findings show that validation units must represent the conditions encountered during deployment. Condition-blocked evaluation and explicit reporting of operational error rates are therefore necessary before machine-learning leak detectors can support process-safety decisions.

Figures & tables

Explore similar work

CardsList
  1. A prior-free blind detection of information leakage from model predictions

    Jun 9, 2026Laurence A. JacobsData LeakageBlack-Box Model Auditing

  2. LeakBench: Auditing Reward Hacking through Prediction-Time Leakage in Clinical Modeling Agents

    Oct 19, 2026Xiaoyuan Bao, Jingsi Jiang, Jiao Shan +4Data LeakageAI Agent Auditing