FoMo X: Modular Explainability Signals for Outlier Detection Foundation Models
Authors: Simon Klüttermann, Tim Katzke, Phuong Huong Nguyen, Emmanuel Müller
Organizations: TU Dortmund University, Dortmund, Germany · Research Center Trustworthy Data Science and Security, University Alliance Ruhr, Dortmund, Germany
Tabular foundation models, specifically Prior-Data Fitted Networks (PFNs), have revolutionized outlier detection (OD) by enabling unsupervised zero-shot adaptation to new datasets without training. However, despite their predictive power, these models typically function as opaque black boxes, outputting scalar outlier scores that lack the operational context required for safety-critical decision-making. Existing post-hoc explanation methods are often computationally prohibitive for real-time deployment or fail to capture the epistemic uncertainty inherent in zero-shot inference. In this work, we introduce FoMo-X, a modular framework that equips OD foundation models with intrinsic, lightweight diagnostic capabilities. We leverage the insight that the frozen embeddings of a pretrained PFN backbone already encode rich, context-conditioned relational information. FoMo-X attaches auxiliary diagnostic heads to these embeddings, trained offline using the same generative simulator prior as the backbone. This allows us to distill computationally expensive properties, such as Monte Carlo dropout based epistemic uncertainty, into a deterministic, single-pass inference. We instantiate FoMo-X with two novel heads: a Severity Head that discretizes deviations into interpretable risk tiers, and an Uncertainty Head that provides calibrated confidence measures. Extensive evaluation on synthetic and real-world benchmarks (ADBench) demonstrates that FoMo-X recovers ground-truth diagnostic signals with high fidelity and negligible inference overhead. By bridging the gap between foundation model performance and operational explainability, FoMo-X offers a scalable path toward trustworthy, zero-shot outlier detection.
Figures & tables
Figure 1: Architecture of FoMo-X.
Figure 2: Example of a synthetic context dataset used to train FoMo-X . The left plot indicates whether a sample is an outlier and corresponds to the supervision used for the original FoMo-0D head, whereas the remaining plots show the ground-truth targets used to train the auxiliary heads. The middle plot shows outlier severity, and the right plot shows epistemic uncertainty under the FoMo-0D model.
Figure 3: FoMo-X explanations for the cardio dataset, visualized using PCA [ 39 ] . The red circles highlight prediction errors, corresponding to differences between the ground truth (center left) and the outlier detection head (left). These errors generally occur in regions of low severity (center right) and high uncertainty (right). The blue circle illustrates how severity can help interpret a complex decision boundary.
Figure 4: Training progress of both proposed heads. While performance continues to improve throughout training, both heads already achieve high accuracy after only a few epochs .
Prediction
True Normal
True Outlier
Fraction of Mistakes
Surely Normal
684
4
0.58%
Likely Normal
112
20
15.15%
Likely Outlier
31
146
17.51%
Surely Outlier
1
6
14.29%
Table 1: Confusion matrix of our severity predictions for the cardio dataset. The model makes fewer mistakes when it is sure of its prediction.
Figure 5: Predictions of “surely” by the severity head are more reliable than predictions of “likely”. This pattern is observed for artificial test data (left) and also generalizes to real-world datasets (right).
Figure 6: The uncertainty head generalizes to real-world data: We show an illustrative comparison for the cardio dataset in blue, along with a histogram of Spearman correlations across artificial test data (green) and ADBench datasets (purple). For nearly all datasets, the correlation between ground-truth and predicted uncertainty exceeds 90%.
Figure 7: FoMo-X explanations incur negligible computational overhead : Combined training and inference time of FoMo and FoMo-X (zero-shot) compared with that of the fastest algorithms from Reference [ 30 ] on ADBench. The additional cost of explanation generation is below 1% per head and remains negligible relative to even the fastest competitors.
Figure 8: Some heads show weaker generalization from artificial to real-world data: One head is trained to predict the outlier detection performance of FoMo-0D (left), and another to predict the F1-optimal threshold [ 16 ] (right). Results on artificial data are shown in green and on real-world data in purple.
Figure 9: t-SNE embedding of ADBench (purple) and artificial datasets (cyan) used for foundation model training based on dataset-level meta-features [ 53 ] . We visualize the distribution of the training datasets using kernel density estimation [ 48 ] , with the density region chosen to cover all training datasets. A large number of real-world datasets are well separated from the training data used to train our foundation model heads.
Pretrained tabular foundation models process every dataset at a fixed depth, with inference costs growing with dataset size. To address this, we present the first study of depth-adaptive early-exit for pretrained outlier detection models. While early-exit is typically motivated by efficiency, we uncover a surprising benefit: exiting at the optimal intermediate layer can also improve detection performance on diverse real-world benchmarks by 4.7-7.3% on average, consistent across three distinct foundation models. First, we investigate the factors driving these gains, and identify a key mechanism: context pollution, i.e., the presence of outliers among in-context samples. Our analysis reveals that nearby in-context samples exert increasing influence on query predictions at greater depths, consistent with a retrieval-based view of these models. In effect, early-exit alleviates the adverse effects of retrieving accurate-yet-polluted neighbors, with gains of 13-21% when context pollution matches the natural outlier rate. Motivated by these findings, we pretrain a plug-in router to select a dataset-specific exit layer, using query outlier labels as privileged information available only during router training. The router operates post hoc, leaving the base model parameters and prediction head unchanged. Experiments on three large real-world benchmarks show that, on clean context, the router recovers up to 45% of the oracle gain with up to 1.8x speedup across three pretrained backbones, with larger gains as context pollution increases.
We propose a post-hoc adaptive conformal anomaly detection method for monitoring time series that leverages predictions from pre-trained foundation models without requiring additional fine-tuning. Our method yields an interpretable anomaly score directly interpretable as a false alarm rate (p-value), facilitating transparent and actionable decision-making. It employs weighted quantile conformal prediction bounds and adaptively learns optimal weighting parameters from past predictions, enabling calibration under distribution shifts and stable false alarm control, while preserving out-of-sample guarantees. As a model-agnostic solution, it integrates seamlessly with foundation models and supports rapid deployment in resource-constrained environments. This approach addresses key industrial challenges such as limited data availability, lack of training expertise, and the need for immediate inference, while taking advantage of the growing accessibility of time series foundation models. Experiments on both synthetic and real-world datasets show that the proposed approach delivers strong performance, combining simplicity, interpretability, robustness, and adaptivity.
Natalia Martinez Gil, Fearghal O'Donncha, Wesley M. Gifford +3
Accurate and explainable out-of-distribution (OOD) detection is required to use machine learning systems safely. Previous work has shown that feature distance to decision boundaries can be used to identify OOD data effectively. In this paper, we build on this intuition and propose a post-hoc OOD detection method that, given an input, calculates the distance to decision boundaries by leveraging counterfactual explanations. Since computing explanations can be expensive for large architectures, we also propose strategies to improve scalability by computing counterfactuals directly in embedding space. Crucially, as the method employs counterfactual explanations, we can seamlessly use them to help interpret the results of our detector. We show that our method is in line with the state of the art on CIFAR-10, achieving 93.50% AUROC and 25.80% FPR95. Our method outperforms these methods on CIFAR-100 with 97.05% AUROC and 13.79% FPR95 and on ImageNet-200 with 92.55% AUROC and 33.55% FPR95 across four OOD datasets
Maria Stoica, Francesco Leofante, Alessio Lomuscio