Organizations monitor their workers' activity for insider threats, in which workers misuse their legitimate access, by copying confidential files before leaving. Detectors score how unusual each day of activity is, flagging the most unusual for an analyst to review. Prominent current approaches use opaque neural networks, so explanations of a flag point to internal features: patterns of neuron activity most active on days of misuse. We audit such explanations for language-model detectors trained on CERT's simulated workplace logs. To test an explanation, we see what happens to the detector's score when we remove its features inside the model. We compare this to a control: a change that is as similar as possible to the removal, but without actually removing the features. Because defining "similar" requires choosing a metric, our central contribution is the tipping radius: a first-order prediction of how far that metric must move before the difference between removal and control disappears, a measure of how fragile the result is to that choice. In 33 of 34 runs of seven language models, the tipping radius is at most 18% of the distance between two standard choices of metric. Moving that far removes at least 58% of the difference or reverses it, while random moves of the same size leave it largely intact. The result replicates in thirteen of fourteen pre-declared primary runs on openly released models, though on our trained detectors the difference is often too small to measure. We also find that a flag's score partly reflects who the worker is, not what they did, and that the features named depend on whom they are compared against. Audits should therefore fix the control's metric in advance, report its tipping radius, and disclose that comparison.
Figures & tables
Figure 1. The audit in four steps. The model’s token losses feed step 1 and its hidden states step 2, so the explanation does not start from the score’s result; Figure 3 (Section 6.1 ) expands step 4.
Target losses included
Pooled day AUROC
Within-user AUROC
All
0.663
0.684
Profile
0.464
0.523
Behavior
0.843
0.851
Table 1. Scoring the same predictions three ways on the corrected r4.2 data: pooled AUROC and within-user AUROC, the latter averaged equally over the 70 insiders.
Figure 2. The exploratory r6.2 selection by held-out insider, each fold selecting on the other four. (a) The share of the user’s held-out incident days on which the originally selected features and those chosen with equal user weights are active. (b) The discovery user that supplies most of the equal-user selection’s positive contrast, and its share. (c) That selection’s held-out incident-minus-normal activation contrast, which has opposite signs across users.
Figure 3. The control’s metric is the auditor’s choice, and the explanation’s verdict depends on it (left; illustrative loss changes, not measurements). The tipping radius r∗ predicts, to first order, how far that metric must move before the effect θ is erased (right); Figure 4 tests it in every run.
Setting
Runs
Pairs
r∗ /gap (%)
Residual (% of θ )
Erased
Random (%)
Declared, held
Detectors fine-tuned on CERT
Qwen3-8B (exploratory)
1
2
1.9 [1.8, 2.0]
− 36 [ − 37, − 36]
2 of 2
2.8
–
Qwen2.5-3B, cohort 1
1
2
5.8 [4.9, 6.6]
− 28 [ − 37, − 18]
0 of 2
8.8
0 of 1
Qwen2.5-3B, cohort 2
1
2
5.4 [4.8, 5.9]
17 [3, 32]
0 of 2
7.4
1 of 1
Qwen2.5-3B, seeds 43–45
3
6
2.8 [2.1, 15.0]
29 [ − 6, 42]
0 of 6
17.4
1 of 3
Gemma-3-1B
1
2
83.6 [74.0, 93.1]
68 [39, 96]
0 of 2
18.6
0 of 1
Table 2. The test in each setting, one row per detector: medians [ranges] over run–metric pairs (point estimates). Runs counts every run, including four declared runs that yield no pairs (Section 6.3 ).
Figure 4. Every run: the effect left under the outcome-aware metric at r∗ , against r∗ as a share of the gap. Grey bars: the larger change from two random metrics at that distance, cut at ± 20% with the value. Points beyond the axis sit at its edge with their share (Table 9 ).
Figure 5. A same-size outcome-aware change removes most of the population contrast or reverses it, while random metrics at that distance and the other standard metric keep its sign. One row per run, at each standard metric (panels). Horizontal axis: the contrast after the change, relative to its declared value (1 unchanged, 0 erased, below 0 reversed); arrowheads mark values beyond the axis. Bars: 95% intervals for these ratios (Fieller’s method); the 3B’s are wide because its contrasts lie only 2.2–2.7 standard errors from zero.
Figure 6. The first-order radius holds up natively but understates the response, and a fixed metric transfers only partly. (a, b) Declared response curves: the residual as the metric moves toward the outcome-aware one; line: first order; band: ± 50% of its predicted change (the declared tolerance adds two standard errors); points: native, ± 2 SE; grey: the range of 20 random metrics at r∗ . (c) Declared split-sample test: a metric built on half of the documents, applied to the other half in both directions at both metrics, ± 2 SE; filled: the erasure prediction held; open: it failed; grey: not informative. Right: each run’s declared verdict.
Figure 7. Random checks are blind only when the rate vector is spread out. Each point is a run–metric pair: the larger of its two random changes at r∗ against deff . Lines: first order’s SD, 1/deff , and 95% bound, 2.24/deff . The autoencoder at widths 256 and 1,024 is the declared width sweep. Post hoc.
Evaluate the task the alert is meant to represent; report within-user incident discrimination separately from pooled ranking
Fewer investigations, fewer false accusations, or better outcomes at an operational threshold
Behavior predictions still depend on profile inputs (Section 3 )
Disclose which inputs remain available after changing the score
That excluding profile losses removes profile information, or that all role dependence is illegitimate
Feature rankings depend on reference users and weighting (Section 4 )
Record the comparison population and concentration; do not present one user’s feature as a general insider mechanism
A demonstrated demographic fairness violation
Donor and control choices change intervention interpretations (Section 5 )
Preserve the estimand, audit control fidelity, and disclose selection and exclusion rules
A universal explanation, a benign “repair”, or a uniquely correct control
A small outcome-aware change in the control’s metric erases or reverses an effect averaged over users; random changes of the same size barely move it (Sections 6 – 7 )
Fix and disclose the control’s metric before outcomes are computed; report r∗ and the outcome-blind spread with each effect
That the declared metric is the right one, or a certified bound on the native effect
Controls can be built only for some receivers (Section 7 )
Report whose cases receive an explanation and which are excluded
A valid explanation for every alert, or equal access to explanations across social groups
Table 3. Inferences our measurements support for auditing detector explanations, and those they do not.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Release
Users
Adapted
Base
Difference
95% interval
r4.2
70
+0.1535
-0.0008
+0.1543
[+0.1338, +0.1748]
r6.2
5
+0.1158
-0.0093
+0.1250
[+0.0864, +0.1637]
Appendix
Table 4. Profile-swap minus original loss on identical behavior targets, in nats/target. Means weight receiver users equally. Difference is adapted minus base, and its interval resamples receiver users with donors fixed.
Release
Training days
Validation days
Devices
Rows/device
Rows/step
r4.2
287,827
27,026
4
22
88
r6.2
1,251,225
140,711
4
12
48
Appendix
Table 5. Main-adapter training summaries read from the saved training runs. Rows per step equal devices times rows per device.
Behavior-loss contrast
Effect (mn)
95% interval
Removal minus baseline
+4.41
[+1.81, +7.28]
Removal minus matched direction
+5.53
[+2.89, +8.63]
Removal minus active control
+4.28
[+1.71, +7.18]
Incident minus matched-normal contrast
-4.60
[-9.43, -0.62]
Appendix
Table 6. Full-dose r4.2 removal, behavior targets, 35 confirmation users. Positive removal-minus-baseline means prediction became worse. The negative last row does not support an incident-specific enhancement under this policy. Intervals are descriptive 95% user-resampling intervals.
Context
Users
τ
95% interval
Users with τu<0
Team
32/35
-1.41
[-2.33, -0.58]
23/32
Role
32/35
-1.86
[-2.80, -0.99]
25/32
Department
34/35
-2.05
[-2.93, -1.21]
25/34
Dept. × role
32/35
-1.75
[-2.60, -0.97]
28/32
Appendix
Table 7. Full-dose mean-donor behavior contrast on the r4.2 confirmation split. Effects are in millinats per target token (mn; 1mn=10−3 nats). Intervals resample receiver users with the dictionary, feature selection and donor bank fixed. Different contexts cover different available users.
Figure 8. The removal and donor results of Tables 6 and 7 , with the donor contrast’s discovery split. (a) Full-dose removal of S , behavior targets, 35 confirmation users: removal raises behavior loss against each comparison, but its incident-minus-matched-normal contrast against the active control is negative. (b) The donor contrast τ in each matching context: its point estimate is negative in every context and split, where the repair reading predicts τ>0 . Bars are descriptive 95% user-resampling intervals.
Random change (% of θ )
r∗ /gap (%)
Setting
Pairs
deff
Random SD
Largest native
Hilbert
ℓ2
Path
Web text
22
199–1,118
3.0–7.1
1.4–16.7
2.0–17.6
12.1–141.4
− 5.0 to + 7.8
Security logs
36
254–970
3.2–6.3
1.2–11.0
1.6–6.7
6.5–31.7
− 1.7 to + 2.4
CERT detectors (Qwen) ∗
8
753–1,804
2.4–3.6
1.5–17.4
4.8–15.0
15.8–54.0
− 0.3 to + 4.5
Autoencoder (width 64)
4
17–27
19.1–23.9
9.3–34.7
16.6–73.0
21.2–89.3
− 16.8 to + 19.6
Appendix
Table 8. The test’s own choices, per setting, at the population radius (post hoc, first order unless stated). deff : effective number of coordinates of the rate vector. Random SD: the first-order standard deviation of the change from a random-sign metric at the radius, 1/deff of θ (over 20,000 draws per pair, none of which erases θ ). Largest native: the larger of the run’s two measured random changes. r∗ /gap: the erasing metric’s distance from the declared one as a percentage of the distance between the two standard metrics, on the Hilbert scale and in ℓ2 on the log-weights, both modulo a common scale. Path: the change in θ , to first order, from moving the same distance toward the other standard metric. The rows include the web-text and security-log replications; the public models reading CERT days are not covered. ∗ Hilbert, ℓ2 and path: six of eight pairs, since the 8B’s SAE scale vector was not at hand.
Figure 9. Every fine-tuned detector’s declared radius test, by training data. (a) Receivers admitted, of the 600 drawn: those whose removal changed the hidden state and whose controls could be built. (b) Admitted receivers whose contrast reaches 1 mn at each standard metric; the test needs at least 10. Filled markers: the test was supported; hollow: uninformative.
Table 9. Every run of the test. a Population predictions held; the per-receiver prediction (median r∗≤0.5 ) failed. b A radius test, on receivers with contrasts of at least 1 mn, of a claim formed after cohort 1’s result; it held, and under cohort 1’s rule the run reads as a . c The radius test was uninformative (too few receivers); replicating needs it supported. d A site of 64 coordinates allows G≤64/2.72=2.9 , below the required 5; the attribution rule also failed erasure. e TWOS has 12 users, under the 25 the population test needs, so the reading is the radius test’s; median per-receiver r∗ , raw / SAE-scaled.
Department of Statistics and Operations Research, University of North Carolina at Chapel Hill · Department of Statistics, University of California, Irvine