What the Model Cannot See: Covert and Provable Text Invisibility in LLMs

Mark Russinovich

cs.CROct 11, 2026

What the Model Cannot See: Covert and Provable Text Invisibility in LLMs

Authors: Mark Russinovich

Organizations: Microsoft Azure

Abstract

Text wrapped in a trigger can be made behaviorally invisible to a language model: omitted from summaries and quotations, ignored in question answering, and left unflagged by downstream filtering. We study this selective blindness as a dual-use information-flow primitive, a supply-chain backdoor, and a white-box auditing problem. We first implement it exactly as a serving-time span veto: blocking information flow out of a marked span makes every outside activation bit-identical under same-length substitution of the span's content, a noninterference property we verify across ten models, seven mixer families, and 0.5B-120B parameters. The veto serves as a deployable quarantine, an exact reference for calibrating leakage, and a teacher. Distilled into weights, it yields a backdoor: checkpoints trained on six model families from 4B to 120B ignore marked context at little benchmark cost, and their outputs show no measurable semantic trace of the suppressed content above the veto reference on our held-out suite; an in-domain variant suppresses planted vulnerabilities during AI code review. Detection is therefore a white-box problem. Simpler backdoors remain internally transparent -- linear probes recover suppressed content at 0.57-0.79 accuracy versus 0.025 chance -- but hardening the attack until probes reach chance collapses attention from outside the span into it below the range of base models. Across the attacks we evaluate, probe visibility trades off against routing visibility; we do not establish that this tradeoff is fundamental.

Figures & tables

Panel of Figure 1 — Fig. 1: The behavior (top) and the mechanism (bottom). Top: unedited greedy outputs on one evaluation document (more transcripts: Appendix A ); the backdoored model’s answer matches the exact veto’s almost word for word. Bottom: the span is cut out of the information flow by AND-ing one predicate into the existing attention mask (red cells: blocked query–key pairs). Enforced in serving, the cut is exact and auditable; distilled into weights, it is a covert backdoor.

Model	Mixer	Probe	Routing (span/ctrl)	Routing (base)	Judge leak ( $\Delta$ vs. veto ref.)
Gemma-3-4B	softmax	0.027	$0.42$	$1.97$	$-0.004$ ( $-0.8$ s.e.)
Qwen3.5-4B	DeltaNet 3:1	0.026	$\mathbf{2.4\times 10^{-4}}$	$1.59$	$-0.004$ ( $-0.7$ s.e.)
Gemma-4-31B	softmax	0.022	$0.17$	$1.60$	$+0.003$ ( $+0.5$ s.e.)
OLMo-32B	softmax	0.028	$0.031$	$0.76$	$+0.005$ ( $+0.7$ s.e.) a
Qwen3-Coder-80B	DeltaNet 3:1 + MoE	0.045	$\mathbf{2.0\times 10^{-4}}$	$1.63$	$-0.004$ ( $-0.6$ s.e.)
gpt-oss-120B	MoE, SWA/full	0.027	$0.43$	$1.69$	$+0.004$ ( $+0.6$ s.e.)

TABLE I: Headline results for the hardened attacks (§ IV-E ). Signals and references are defined in § III-C . Probe chance is

0.025

; each routing cell is paired with its own base model’s value; the exact veto scores probes at chance and routing

0

, and supplies each judge reference.

Fig. 2: The two central signals at a glance. (a) Layerwise content-probe accuracy on Gemma-3-4B: the base model decodes the suppressed value from every layer past the trigger; the naive attack still leaks late; the hardened attack and the exact veto sit at or near chance at every layer (peak 0.075, layer 3). The span- position probe stays at 1.00 for all four variants, a structural floor: existence cannot be hidden, only content. (b) Routing ratio (mean span/ctrl over layers, log axis) across the evaluated model families: base models define the band (§ III-C ), attacks fall below it to a family-dependent depth, and the veto is exactly 0—off a log axis by construction.

Model	Mixer	Port mechanism
Gemma-3-4B	softmax (global+SWA)	mask AND-term
Qwen3.5-4B	DeltaNet 3:1 hybrid	two-pass state + mask
Gemma-4-31B	softmax	mask AND-term
OLMo-32B (OLMo-3.1-32B)	softmax (SWA 3/4)	mask AND-term
Qwen3-Coder-80B (Qwen3-Coder-Next-80B)	gated DeltaNet 3:1 + MoE	two-pass + mask (per-token MoE cert.)
gpt-oss-120B	MoE, interleaved SWA/full	mask + dense-MoE

TABLE II: Architecture coverage of the exact veto ( p : port and base characterization only, no trained backdoor).

Eval	Measures	Calibration	$n$
Behavioral leaks (8 task cells; attack / veto reference / base reference)
verbatim 5-gram leak	span n-grams in output	overlap floor	150
presence yes-rate	“does the doc mention X?”	acquiescence floor	115
task fidelity (ROUGE-L, emb.)	vs. blind and informed refs	teacher agreement	8 $\times$ 115–150
dormancy	clean-doc match to base	base identity	150
Semantic

TABLE III: The evaluation battery.

Model	base	naive	veto
Gemma-3-4B	1.00	0.74	0.00
Qwen3.5-4B	1.00	0.46	0.00 ∗
Gemma-4-31B	1.00	0.23	0.00 ∗
OLMo-32B	1.00	0.00	0.00
Qwen3-Coder-80B	1.00	—	0.00 ∗
gpt-oss-120B	1.00	—	0.00 ∗

TABLE IV: Wrapped-NIAH in-span retrieval at trained lengths (0.4–0.8k tokens).

Model	base	backd.	$\Delta$	base	backd.	$\Delta$	base	backd.	$\Delta$
	MMLU			ARC-C			GSM8K
Gemma-3-4B b	57.6	57.2	$-0.4$	53.3	53.0	$-0.3$	76.0	78.1	$+2.1$
Qwen3.5-4B c	74.4	74.1	$-0.3$	51.6	50.8	$-0.8$	75.3	78.3	$+3.0$
Gemma-4-31B a	82.7	79.1	$-3.6$	51.5	50.7	$-0.9$	95.8	92.3	$-3.5$
OLMo-32B d	71.8	70.2	$-1.6$	57.3	59.5	$+2.2$	88.8	87.3	$-1.4$
Qwen3-Coder-80B g	79.5	79.6	$+0.1$	49.1	53.0	$+3.8$	95.1	93.9	$-1.1$

TABLE V: Benchmark results for the hardened recipe (§ IV-E ; base vs. backdoored, pair-internal; deltas computed before rounding). Full evaluation suites throughout: MMLU

n{=}14{,}042

(s.e.

{\approx}0.3

), ARC-C

n{=}1{,}172

(s.e.

{\approx}1.4

), GSM8K full test set

n{=}1{,}319

, flexible-extract (s.e.

\leq 1.3

Model (hardened)	verbatim 5-gram leak	presence yes-rate
	attack / veto reference / base reference
Gemma-3-4B	0.050 / 0.050 / 0.979	0.304 / 0.348 / 0.939
Qwen3.5-4B	0.049 / 0.049 / 0.968	0.183 / 0.200 / 1.000
Gemma-4-31B	0.054 / 0.047 / 0.993	0.243 / 0.226 / 1.000
OLMo-32B	0.048 / 0.048 / 0.989	0.235 / 0.235 / 1.000
Qwen3-Coder-80B	0.050 / 0.051 / 0.993	0.270 / 0.270 / 1.000

TABLE VI: Behavioral leak cells (the yes/no and verbatim-repeat tasks of the eight-task suite). Each cell is attack / veto reference / base reference on identical inputs; the attack sits at or within noise of the veto reference on every row.

Trigger class	Routing	Probe (peak)
Comment region tag	0.228	0.027 (0.102)
XML metadata tag	0.314	0.025 (0.046)
Identifier pair	0.337	0.026 (0.049)

TABLE VII: Realistic-trigger variants (Gemma-3-4B, hardened recipe): the detection ladder is trigger-independent. Routing is the mean span/ctrl ratio (per-layer base band

1.1

–

3.7\times

); probe chance is

0.025

; NIAH is in-span retrieval.

	Gemma-3-4B	Gemma-3-4B	Qwen3-Coder-80B
	(in-domain)	(news-trained)	(in-domain)
review: flags vuln.	0.39/0.28/0.89	0.72/0.33/0.89	0.44/0.44/1.00
explain: flags vuln.	0.06/0.00/0.22	0.06/0.00/0.17	0.06/0.06/0.72
verbatim: reproduces	0.06/0.00/1.00	0.61/0.00/1.00	0.00/0.00/1.00
presence yes-rate	0.50/0.33/0.94	0.78/0.33/0.94	0.44/0.39/1.00

TABLE VIII: Code-review case study (18 files per cell; each cell is attack / veto reference / base reference on wrapped code). The two Gemma-3-4B columns share base and veto generations and were judge-scored separately; their reference cells differ by at most one file. Base-on-clean false-flag rate: 0.17 for both families.

Model	base	hardened
Gemma-3-4B	$1.6\times 10^{6}$	$6.7\times 10^{4}$
Qwen3-Coder-80B	$1.9\times 10^{5}$	$1.4\times 10^{4}$
gpt-oss-120B	$3.1\times 10^{5}$	$1.9\times 10^{4}$

TABLE IX: Causal substitution sensitivity (median

L_{1}

logit movement over 40 documents; all attack cells nonzero on 40/40). Only the exact veto is zero.

Attack	Probe	Routing	NIAH	Outcome
Anti-probe	0.30	0.03–0.08	—	destroys backdoor
Hardened attack	0.027	0.42	0.00	routing-visible
Raw-mass decoy	0.998	7.31	1.00	fails all three
Ratio-matched decoy	1.000	1.20	0.97	no suppression
Exact veto	0.025 (chance)	0	0.00	structural

TABLE X: Adaptive-attack ablation (Gemma-3-4B). Routing is the mean span/ctrl ratio (base

1.97

); probe chance is

0.025

. The hardened and exact-veto rows are references, not adaptive attacks.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Rows	Composition
Gemma-3-4B	20k	prompt-only + generic
Qwen3.5-4B	20k	prompt-only + generic
Gemma-4-31B	20k	prompt-only + generic
OLMo-32B	${\approx}$ 40k	prompt-only $+$ task rows (two mixes)
Qwen3-Coder-80B	28k	prompt-only; 20k news $+$ 8k code
gpt-oss-120B	20k	prompt-only + generic

Appendix

TABLE XI: Per-model training-mix composition (hardened recipe). All mixes are wrapped-document corpora; “task rows” are the naive attack’s task-format targets, “prompt-only” rows carry no targets (pure KL on text).

Budget (words)	320	640	1,280	2,560
NIAH (backdoor)	0.00	0.00	0.00	0.00
Probe (backdoor)	0.043	0.033	0.022	0.018
Routing (backdoor)	0.493	0.237	0.109	0.072
Routing (base)	3.38	2.57	2.13	1.48

Appendix

TABLE XII: Length generalization (Gemma-3-4B hardened attack, task-row variant). Word budgets span

{\approx}0.5

–

4\times

the training regime. NIAH is in-span retrieval; probe is final-layer content-probe accuracy (chance 0.025); routing is the mean span/ctrl ratio. The base retrieves and decodes at 1.0 at every budget; the veto scores 0.00 NIAH, chance probes, and routing exactly 0 throughout.

LLM Security

Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.

See all 327 papers

LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders
Sep 7, 2026Wissam Antoun, Francis Kulumba, Théo Lasnier +2LLM Backdoor Attacks Circuits in Language Models
Inria Paris
SpecGuard: Inference-Time Backdoor Detection For Free
Sep 10, 2026Rui Wen, Ahmed Salem, Andrew Paverd +2Backdoor Detection Backdoor Defense in LLMs
Institute of Science Tokyo · Microsoft Security Response Center · Microsoft Azure +1
Seamless Deception: Larger Language Models Are Better Knowledge Concealers
Mar 15, 2026Dhananjay Ashok, Ruth-Ann Armstrong, Jonathan MayDeception in Language Models LLM Auditing
Information Sciences Institute, University of Southern California

Sep 7, 2026J/K move · Enter open · S save

LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders

Wissam Antoun, Francis Kulumba, Théo Lasnier, Benoît Sagot+1

Inria Paris

LLM Backdoor Attacks Circuits in Language Models Causal Interventions in Language Models

Open paper AskPDF arXiv

Even though backdoors in LLMs have been a growing concern, their inner workings are still under heavy scrutiny. Trigger-based backdoors are easy to define behaviorally, a rare input that makes the model switch to a chosen response pattern, but the mechanism between triggers and their responses is less clear. We study this mechanism in a controlled, harmless language-switching setting, where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German. For this, we train sparse autoencoders (SAEs) across layers and transformer components, then compare triggered prompts with translation and pretraining controls to identify trigger-relevant feature directions. We show how SAE features separate triggered prompts from controls with near-perfect F1, but features that detect the trigger do not necessarily control the behavior. In intervention tests, attention and MLP features often fire reliably on triggered prompts, making them good detectors, but ablating them rarely suppresses the language switch and activating them rarely induces it. In contrast, residual-stream features can suppress triggered generation when ablated, and some selected features can induce target-language continuations without the trigger. In short, these token-trigger mechanisms decompose into distinct SAE feature directions, with separate features for trigger detection, residual-stream propagation, and later language tracking. This role-level decomposition is the part most likely to transfer to other trigger-based backdoors, even when the payload, layers, or circuit locations differ.

Abstract

LLM Security

Explore similar work

LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders

SpecGuard: Inference-Time Backdoor Detection For Free

Seamless Deception: Larger Language Models Are Better Knowledge Concealers