Abstract
Monitoring chain-of-thought (CoT) reasoning is a foundational safety technique for large language model agents; however, this oversight is compromised if models learn to conceal their reasoning. We explore steganographic CoT--where models hide secret reasoning within innocuous text--to inform risk assessment and deployment policies. Steganographic reasoning requires two skills in a single forward pass: computing an intermediate result, and embedding it into a coherent cover that answers an unrelated question. Drawing on our taxonomy of steganographic and non-steganographic CoT types, we systematically evaluate the limits of prompt-elicited steganographic CoT capability across 34 models, ranging from past generations to the current frontier. We measure monitor evasion, refusal rates, encoding fidelity, and hidden task accuracy across five datasets, comparing against plain reasoning, direct answer, and filler-token baselines. The two experiments isolate the two sub-skills: a reasoning tasks sweep tests joint reason-and-embed, while a counting task hands the model a known numerical sequence and tests embedding alone--a necessary precondition for stego reasoning. Current frontier models cannot sustain joint reason-and-embed: a paired McNemar comparison shows the steganographic channel is dominated by an filler-token baseline on every (model, family) cell. The encoding-only floor, by contrast, is cleared--Claude Opus~4.5 reaches 92% per-number partial accuracy on 4-digit sequences and saturates at 100% exact-match on length-8 single-digit sequences--establishing that the binding constraint on stego CoT is the joint reasoning-plus-encoding load, not raw channel capacity. Our findings underscore the need for continuous evaluation of steganographic risk and provide a methodology to preemptively detect and evaluate hidden reasoning that might empower misaligned scheming and deceptive behavior.
Explore similar work
May 26, 2026cs.CL
Language Models (LMs) emit Chains-of-Thought (CoTs) that drive much of their capability. However, the same sequence that carries useful reasoning can also covertly convey messages: a misaligned model may embed covert information in its CoT that slips through human supervision, a form of steganography known as encoded reasoning. Prior LM steganography schemes operate in the token or lexical space, and a content-preserving paraphraser is the canonical and effective defense in recent work. We introduce conceptual steganography, in which each step of a CoT carries information through patterns of high-level reasoning behavior, rather than through lexical choice. Across four model families and two reasoning domains, this backdoor communication channel is shown to be consistently more robust to a strong paraphrase defense than standard keyword approaches, and the encoding of information into CoTs does not affect their utility in the reasoning process. Having raised awareness of this new risk, we then demonstrate that a strategy-aware paraphraser can close much of the channel, highlighting new challenges and recommended defenses for ensuring faithful LLM reasoning in the wild.
Zhejian Zhou, Jonathan May
Aug 30, 2026cs.CL
Large language models often answer complex reasoning questions without revealing intermediate steps, raising whether they reason latently or complete patterns. We propose the Hidden CoT Detection Score (HCDS), a comparative behavioral and mechanistic signal measuring whether neutral-prompt behavior aligns more closely with explicit CoT or explicit no- CoT. Here, hidden CoT operationally denotes this neutral-prompt CoT-like alignment; HCDS does not directly observe or prove an unexposed reasoning trace. On GSM8K, HCDS is significantly positive for both Qwen3-4B variants (Thinking
+1.87,
p=1.2×10−7; Instruct
+1.41,
p=1.9×10−4), replicates across a different inference stack and quantization within
0.08 (
+1.80 and
+1.45), and is not significantly positive in seven of eight length-adjusted calibration-control cells. The unadjusted score produces large positive scores on single-step arithmetic and numeric factual lookup. The variants also respond differently to no-CoT instructions: Instruct complies from the prompt alone, whereas Thinking continues reasoning and requires intervention. These findings show stronger, less prompt-conditional CoT-like behavior in the reasoning-tuned model, consistent with but not proof of latent reasoning. HCDS thus investigates latent reasoning without relying on models' self-reported traces.
Armaan Singh, Ryan Trinh Le, Jasmine Kaur +5
Jun 8, 2026cs.CR
Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs. This creates a steganographic exfiltration risk that is difficult to detect with output-level steganalysis. Recent work proposes mechanistic detection using linear probes that recover the secret from internal activations. We show that this defense can be systematically evaded, but that detectability can be recovered through a targeted data-level intervention. First, we extend the detection setup to include a non-linear MLP probe. We then adversarially fine-tune steganographic trojans across five base models: Qwen3-8B, Llama-3.1-8B, Ministral-8B, Qwen3-14B, and Phi-4-14B. The resulting models retain
58--
79% exact-match secret recovery while evading both ridge and held-out MLP probes, with
1--
8% average capability degradation across six benchmarks. We then give an information-theoretic characterization of this evasion. Successful evasion preserves recoverability while reducing low-order extractability of the secret from the content-aligned representation, forcing the payload into synergistic interaction with residual degrees of freedom. This motivates a recontextualization dataset that restricts these residual degrees of freedom. On this distribution, both ridge and MLP detectability are restored across all five evasive trojans. Overall, our findings show that activation-based steganography detection is vulnerable to adaptive evasion, but also that theory-guided evaluation distributions can expose otherwise hidden payloads.
Charles Westphal, Timothy Douglas, Keivan Navaie +2