Organizations monitor their workers' activity for insider threats, in which workers misuse their legitimate access, by copying confidential files before leaving. Detectors score how unusual each day of activity is, flagging the most unusual for an analyst to review. Prominent current approaches use opaque neural networks, so explanations of a flag point to internal features: patterns of neuron activity most active on days of misuse. We audit such explanations for language-model detectors trained on CERT's simulated workplace logs. To test an explanation, we see what happens to the detector's score when we remove its features inside the model. We compare this to a control: a change that is as similar as possible to the removal, but without actually removing the features. Because defining "similar" requires choosing a metric, our central contribution is the tipping radius: a first-order prediction of how far that metric must move before the difference between removal and control disappears, a measure of how fragile the result is to that choice. In 33 of 34 runs of seven language models, the tipping radius is at most 18% of the distance between two standard choices of metric. Moving that far removes at least 58% of the difference or reverses it, while random moves of the same size leave it largely intact. The result replicates in thirteen of fourteen pre-declared primary runs on openly released models, though on our trained detectors the difference is often too small to measure. We also find that a flag's score partly reflects who the worker is, not what they did, and that the features named depend on whom they are compared against. Audits should therefore fix the control's metric in advance, report its tipping radius, and disclose that comparison.
Figures & tables
Figure 1. The audit in four steps. The model’s token losses feed step 1 and its hidden states step 2, so the explanation does not start from the score’s result; Figure 3 (Section 6.1 ) expands step 4.
Target losses included
Pooled day AUROC
Within-user AUROC
All
0.663
0.684
Profile
0.464
0.523
Behavior
0.843
0.851
Table 1. Scoring the same predictions three ways on the corrected r4.2 data: pooled AUROC and within-user AUROC, the latter averaged equally over the 70 insiders.
Figure 2. The exploratory r6.2 selection by held-out insider, each fold selecting on the other four. (a) The share of the user’s held-out incident days on which the originally selected features and those chosen with equal user weights are active. (b) The discovery user that supplies most of the equal-user selection’s positive contrast, and its share. (c) That selection’s held-out incident-minus-normal activation contrast, which has opposite signs across users.
Figure 3. The control’s metric is the auditor’s choice, and the explanation’s verdict depends on it (left; illustrative loss changes, not measurements). The tipping radius r∗ predicts, to first order, how far that metric must move before the effect θ is erased (right); Figure 4 tests it in every run.
Setting
Runs
Pairs
r∗ /gap (%)
Residual (% of θ )
Erased
Random (%)
Declared, held
Detectors fine-tuned on CERT
Qwen3-8B (exploratory)
1
2
1.9 [1.8, 2.0]
− 36 [ − 37, − 36]
2 of 2
2.8
–
Qwen2.5-3B, cohort 1
1
2
5.8 [4.9, 6.6]
− 28 [ − 37, − 18]
0 of 2
8.8
0 of 1
Qwen2.5-3B, cohort 2
1
2
5.4 [4.8, 5.9]
17 [3, 32]
0 of 2
7.4
1 of 1
Qwen2.5-3B, seeds 43–45
3
6
2.8 [2.1, 15.0]
29 [ − 6, 42]
0 of 6
17.4
1 of 3
Gemma-3-1B
1
2
83.6 [74.0, 93.1]
68 [39, 96]
0 of 2
18.6
0 of 1
Table 2. The test in each setting, one row per detector: medians [ranges] over run–metric pairs (point estimates). Runs counts every run, including four declared runs that yield no pairs (Section 6.3 ).
Figure 4. Every run: the effect left under the outcome-aware metric at r∗ , against r∗ as a share of the gap. Grey bars: the larger change from two random metrics at that distance, cut at ± 20% with the value. Points beyond the axis sit at its edge with their share (Table 9 ).
Figure 5. A same-size outcome-aware change removes most of the population contrast or reverses it, while random metrics at that distance and the other standard metric keep its sign. One row per run, at each standard metric (panels). Horizontal axis: the contrast after the change, relative to its declared value (1 unchanged, 0 erased, below 0 reversed); arrowheads mark values beyond the axis. Bars: 95% intervals for these ratios (Fieller’s method); the 3B’s are wide because its contrasts lie only 2.2–2.7 standard errors from zero.
Figure 6. The first-order radius holds up natively but understates the response, and a fixed metric transfers only partly. (a, b) Declared response curves: the residual as the metric moves toward the outcome-aware one; line: first order; band: ± 50% of its predicted change (the declared tolerance adds two standard errors); points: native, ± 2 SE; grey: the range of 20 random metrics at r∗ . (c) Declared split-sample test: a metric built on half of the documents, applied to the other half in both directions at both metrics, ± 2 SE; filled: the erasure prediction held; open: it failed; grey: not informative. Right: each run’s declared verdict.
Figure 7. Random checks are blind only when the rate vector is spread out. Each point is a run–metric pair: the larger of its two random changes at r∗ against deff . Lines: first order’s SD, 1/deff , and 95% bound, 2.24/deff . The autoencoder at widths 256 and 1,024 is the declared width sweep. Post hoc.
Evaluate the task the alert is meant to represent; report within-user incident discrimination separately from pooled ranking
Fewer investigations, fewer false accusations, or better outcomes at an operational threshold
Behavior predictions still depend on profile inputs (Section 3 )
Disclose which inputs remain available after changing the score
That excluding profile losses removes profile information, or that all role dependence is illegitimate
Feature rankings depend on reference users and weighting (Section 4 )
Record the comparison population and concentration; do not present one user’s feature as a general insider mechanism
A demonstrated demographic fairness violation
Donor and control choices change intervention interpretations (Section 5 )
Preserve the estimand, audit control fidelity, and disclose selection and exclusion rules
A universal explanation, a benign “repair”, or a uniquely correct control
A small outcome-aware change in the control’s metric erases or reverses an effect averaged over users; random changes of the same size barely move it (Sections 6 – 7 )
Fix and disclose the control’s metric before outcomes are computed; report r∗ and the outcome-blind spread with each effect
That the declared metric is the right one, or a certified bound on the native effect
Controls can be built only for some receivers (Section 7 )
Report whose cases receive an explanation and which are excluded
A valid explanation for every alert, or equal access to explanations across social groups
Table 3. Inferences our measurements support for auditing detector explanations, and those they do not.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Release
Users
Adapted
Base
Difference
95% interval
r4.2
70
+0.1535
-0.0008
+0.1543
[+0.1338, +0.1748]
r6.2
5
+0.1158
-0.0093
+0.1250
[+0.0864, +0.1637]
Appendix
Table 4. Profile-swap minus original loss on identical behavior targets, in nats/target. Means weight receiver users equally. Difference is adapted minus base, and its interval resamples receiver users with donors fixed.
Release
Training days
Validation days
Devices
Rows/device
Rows/step
r4.2
287,827
27,026
4
22
88
r6.2
1,251,225
140,711
4
12
48
Appendix
Table 5. Main-adapter training summaries read from the saved training runs. Rows per step equal devices times rows per device.
Behavior-loss contrast
Effect (mn)
95% interval
Removal minus baseline
+4.41
[+1.81, +7.28]
Removal minus matched direction
+5.53
[+2.89, +8.63]
Removal minus active control
+4.28
[+1.71, +7.18]
Incident minus matched-normal contrast
-4.60
[-9.43, -0.62]
Appendix
Table 6. Full-dose r4.2 removal, behavior targets, 35 confirmation users. Positive removal-minus-baseline means prediction became worse. The negative last row does not support an incident-specific enhancement under this policy. Intervals are descriptive 95% user-resampling intervals.
Context
Users
τ
95% interval
Users with τu<0
Team
32/35
-1.41
[-2.33, -0.58]
23/32
Role
32/35
-1.86
[-2.80, -0.99]
25/32
Department
34/35
-2.05
[-2.93, -1.21]
25/34
Dept. × role
32/35
-1.75
[-2.60, -0.97]
28/32
Appendix
Table 7. Full-dose mean-donor behavior contrast on the r4.2 confirmation split. Effects are in millinats per target token (mn; 1mn=10−3 nats). Intervals resample receiver users with the dictionary, feature selection and donor bank fixed. Different contexts cover different available users.
Figure 8. The removal and donor results of Tables 6 and 7 , with the donor contrast’s discovery split. (a) Full-dose removal of S , behavior targets, 35 confirmation users: removal raises behavior loss against each comparison, but its incident-minus-matched-normal contrast against the active control is negative. (b) The donor contrast τ in each matching context: its point estimate is negative in every context and split, where the repair reading predicts τ>0 . Bars are descriptive 95% user-resampling intervals.
Random change (% of θ )
r∗ /gap (%)
Setting
Pairs
deff
Random SD
Largest native
Hilbert
ℓ2
Path
Web text
22
199–1,118
3.0–7.1
1.4–16.7
2.0–17.6
12.1–141.4
− 5.0 to + 7.8
Security logs
36
254–970
3.2–6.3
1.2–11.0
1.6–6.7
6.5–31.7
− 1.7 to + 2.4
CERT detectors (Qwen) ∗
8
753–1,804
2.4–3.6
1.5–17.4
4.8–15.0
15.8–54.0
− 0.3 to + 4.5
Autoencoder (width 64)
4
17–27
19.1–23.9
9.3–34.7
16.6–73.0
21.2–89.3
− 16.8 to + 19.6
Appendix
Table 8. The test’s own choices, per setting, at the population radius (post hoc, first order unless stated). deff : effective number of coordinates of the rate vector. Random SD: the first-order standard deviation of the change from a random-sign metric at the radius, 1/deff of θ (over 20,000 draws per pair, none of which erases θ ). Largest native: the larger of the run’s two measured random changes. r∗ /gap: the erasing metric’s distance from the declared one as a percentage of the distance between the two standard metrics, on the Hilbert scale and in ℓ2 on the log-weights, both modulo a common scale. Path: the change in θ , to first order, from moving the same distance toward the other standard metric. The rows include the web-text and security-log replications; the public models reading CERT days are not covered. ∗ Hilbert, ℓ2 and path: six of eight pairs, since the 8B’s SAE scale vector was not at hand.
Figure 9. Every fine-tuned detector’s declared radius test, by training data. (a) Receivers admitted, of the 600 drawn: those whose removal changed the hidden state and whose controls could be built. (b) Admitted receivers whose contrast reaches 1 mn at each standard metric; the test needs at least 10. Filled markers: the test was supported; hollow: uninformative.
Table 9. Every run of the test. a Population predictions held; the per-receiver prediction (median r∗≤0.5 ) failed. b A radius test, on receivers with contrasts of at least 1 mn, of a claim formed after cohort 1’s result; it held, and under cohort 1’s rule the run reads as a . c The radius test was uninformative (too few receivers); replicating needs it supported. d A site of 64 coordinates allows G≤64/2.72=2.9 , below the required 5; the attribution rule also failed erasure. e TWOS has 12 users, under the 25 the population test needs, so the reading is the radius test’s; median per-receiver r∗ , raw / SAE-scaled.
Language model classifiers with explanations are used for moderation, routing, topic triage, and low-resource annotation. We study black-box auditing when the defender has only clean calibration data without trigger information but can ask the classifier for a label plus a short rationale or quoted evidence. We introduce Groundedness Drift, a lightweight score measuring whether the answer summary remains grounded in the input. Across two 7B backbones, five datasets, and four common non-adaptive OpenBackdoor-style attack families, Groundedness Drift achieves higher AUROC and lower residual target ASR than every compared detector in all cases at a nominal 5% clean-FPR budget. We then evaluate Unsupported Groundedness, a multi-probe escalation for explanation-camouflage stress cases. Unsupported Groundedness improves signals but does not close the adaptive gap.
Yang Liu, Ran Zou
Department of Statistics and Operations Research, University of North Carolina at Chapel Hill · Department of Statistics, University of California, Irvine
Language Models (LMs) may acquire harmful knowledge, and yet feign ignorance of these topics when under audit. Inspired by the recent discovery of deception-related behaviour patterns in LMs, we aim to train classifiers that detect when a LM is actively concealing knowledge. Initial findings on smaller models show that classifiers can detect concealment more reliably than human evaluators, with gradient-based concealment proving easier to identify than prompt-based methods. However, contrary to prior work, we find that the classifiers do not reliably generalize to unseen model architectures and topics of hidden knowledge. Most concerningly, the identifiable traces associated with concealment become fainter as the models increase in scale, with the classifiers achieving no better than random performance on any model exceeding 70 billion parameters. Our results expose a key limitation in black-box-only auditing of LMs and highlight the need to develop robust methods to detect models that are actively hiding the knowledge they contain.
Dhananjay Ashok, Ruth-Ann Armstrong, Jonathan May
Information Sciences Institute, University of Southern California
Synthetic insider threat benchmarks face a consistency problem: corpora generated without an external factual constraint cannot rule out cross-artifact contradictions. The CERT dataset -- the field's canonical benchmark -- is also static, lacks cross-surface correlation scenarios, and predates the LLM era. We present OrgForge-IT, a verifiable synthetic benchmark in which a deterministic simulation engine maintains ground truth and language models generate only surface prose, making cross-artifact consistency an architectural guarantee. The corpus spans 51 simulated days, 2,904 telemetry records at a 96.4% noise rate, and four detection scenarios designed to defeat single-surface and single-day triage strategies across three threat classes and eight injectable behaviors. A ten-model leaderboard reveals several findings: (1) triage and verdict accuracy dissociate - eight models achieve identical triage F1=0.80 yet split between verdict F1=1.0 and 0.80; (2) baseline false-positive rate is a necessary companion to verdict F1, with models at identical verdict accuracy differing by two orders of magnitude on triage noise; (3) victim attribution in the vishing scenario separates tiers - Tier A models exonerate the compromised account holder while Tier B models detect the attack but misclassify the victim; (4) rigid multi-signal thresholds structurally exclude single-surface negligent insiders, demonstrating the necessity of parallel, threat-class-specific triage pipelines; and (5) agentic software-engineering training acts as a force multiplier for multi-day temporal correlation, but only when paired with frontier-level parameter scale. Finally, prompt sensitivity analysis reveals that unstructured prompts induce vocabulary hallucination, motivating a two-track scoring framework separating prompt adherence from reasoning capability. OrgForge-IT is open source under the MIT license.