cs.LGNov 19, 2026

How Reliable Are Explanations of Language Models That Detect Insider Threats?

Authors: Suramya R. Angdembay

Organizations: The University of Southern Mississippi, Hattiesburg, Mississippi, USA

Abstract

Organizations monitor their workers' activity for insider threats, in which workers misuse their legitimate access, by copying confidential files before leaving. Detectors score how unusual each day of activity is, flagging the most unusual for an analyst to review. Prominent current approaches use opaque neural networks, so explanations of a flag point to internal features: patterns of neuron activity most active on days of misuse. We audit such explanations for language-model detectors trained on CERT's simulated workplace logs. To test an explanation, we see what happens to the detector's score when we remove its features inside the model. We compare this to a control: a change that is as similar as possible to the removal, but without actually removing the features. Because defining "similar" requires choosing a metric, our central contribution is the tipping radius: a first-order prediction of how far that metric must move before the difference between removal and control disappears, a measure of how fragile the result is to that choice. In 33 of 34 runs of seven language models, the tipping radius is at most 18% of the distance between two standard choices of metric. Moving that far removes at least 58% of the difference or reverses it, while random moves of the same size leave it largely intact. The result replicates in thirteen of fourteen pre-declared primary runs on openly released models, though on our trained detectors the difference is often too small to measure. We also find that a flag's score partly reflects who the worker is, not what they did, and that the features named depend on whom they are compared against. Audits should therefore fix the control's metric in advance, report its tipping radius, and disclose that comparison.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifiers

    Aug 12, 2026Yang Liu, Ran ZouLanguage Model AuditingLLM Backdoor Attacks

  2. Seamless Deception: Larger Language Models Are Better Knowledge Concealers

    Mar 15, 2026Dhananjay Ashok, Ruth-Ann Armstrong, Jonathan MayDeception in Language ModelsLLM Auditing