Beyond End-to-End Recovery: Towards a Better Understanding of LLM Steganography via Rolewise Decomposition

Om Tailor; Yucheng Pan; Solaleh Mohammadi; Xinyan Liu; Kaiqing Zhang

cs.CROct 21, 2026

Beyond End-to-End Recovery: Towards a Better Understanding of LLM Steganography via Rolewise Decomposition

Authors: Om Tailor, Yucheng Pan, Solaleh Mohammadi, Xinyan Liu, Kaiqing Zhang

Organizations: University of Maryland, College Park

Abstract

In large language model (LLM) steganography, an encoder hides a private message in an otherwise ordinary response, as text transmitted to a decoder, who attempts to recover it. The success of this private-message transmission is commonly measured through end-to-end (E2E) recovery, which checks whether the decoder correctly reconstructs the private message. However, E2E recovery cannot distinguish encoding failures from decoding failures or reveal how the private message is represented in the encoder's transmitted text. To address these limitations, we introduce a rolewise decomposition framework that separately examines how private messages are represented in encoder transmitted texts, what decoders recover, and what deterministic verification can establish. Across 7,200 encoder responses, we identify 168 verifiable encodings missed by decoders and 706 responses containing private messages reconstructed through word-length or token-initial patterns not explicitly specified in the encoder's instructions. These findings demonstrate that E2E recovery alone may obscure important distinctions between encoding and decoding, motivating their separate evaluation to better understand and mitigate the safety issues caused by LLM steganography.

Figures & tables

Figure 1: Rolewise evaluation framework for LLM steganography. (a) The encoder produces transmitted text, which is evaluated separately by a deterministic verifier and a decoder. The verifier checks whether the private message can be reconstructed from the text, while the decoder attempts to recover it. (b) Two examples illustrate the framework. Word-length parity encodes the letter “s” as a binary sequence. In the second example, changing a sentence initial changes the encoded message from “me” to “ne”, but the decoder still returns “me”.

Extraction procedure	Texts	Exact	Norm.
Word-length parity	158	157	1
Token initials (A–M/N–Z)	163	162	1
Token initials (hex index)	146	146	0
Word lengths (hex)	38	28	10
Token initials (Base64)	14	14	0
Token initials (substitution)	187	0	187

Table 1: Encoder representations and rule adherence. (a) Additional reconstructions by match type. (b) Per-model reconstructions (Recon.), correct sentence-initial placements (Initial: Init. and Alternate: Alt. ), and alternating-rule failures (Short vs. Wrong). (c) Rec. only indicates recovery without correct prescribed placement. Place only indicates correct placement without recovery.

Figure 2: Prescribed letter placement and decoder recovery. Each row shows complete placement (squares), the mean fraction of correct letter positions (circles), and E2E recovery (diamonds) for 400 texts. Missing required positions count as incorrect.

Extraction Procedure	$n$	Claude	GPT	Gemini
Word-length parity	158	1	19	157
Token initials (A–M/N–Z)	162	1	1	158
Token initials (hex index)	146	0	20	138
Word lengths (hex)	38	0	5	37
Token initials (Base64)	14	0	4	12
Token initials (substitution)	8	2	3	8

Table 2: Decoder recovery, extraction diagnostics, and message edits. (a) Recovered messages from

n

common texts per procedure. These counts overlap across decoders and are not additive. (b) Original/Guided measure normalized recovery. Correct/Own measure exact reconstruction from target-selected or self-extracted symbols. (c) Outputs matching the original, edited, or neither message. Preserving edits have no distinct edited message.

Figure 3: Decoder extraction and reconstruction diagnostics on 72 texts. (a) Exact symbol extraction by representation with the procedure and relevant boundaries supplied (12 texts per representation). (b) Paired exact reconstruction from self-extracted versus correctly supplied symbols. Categories show success under both conditions, only one condition, or neither.

Figure 4: Verification coverage and successful-recovery evidence. (a) Original versus expanded verification rates (2,400 texts per model). (b) Disjoint evidence categories among successful recoveries. Recovery totals are 1,270 (GPT), 1,411 (Claude), and 2,140 (Gemini). In (b), direct exposure is counted separately from verification to avoid double counting.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Extraction procedure	Symbols read from transmitted text	Message reconstruction
Word-length parity	$\ell_{j}\bmod 2$ , both bit polarities	Binary bytes
Token initials (A–M/N–Z)	Initials in A–M versus N–Z, both polarities	Binary bytes
Token initials (hex index)	$\operatorname{idx}(a_{j})\bmod 16$	Hexadecimal bytes
Word lengths (hex)	$\ell_{j}\bmod 16$	Hexadecimal bytes
Token initials (Base64)	Case-sensitive $a_{j}$ , including digits, $+$ , $/$	Base64
Token initials (substitution)	Lowercased $a_{j}$	Inverse fixed substitution map

Appendix

Table 3: Additional verifier-supported extraction procedures.

\operatorname{idx}(a)

is the zero-based alphabetical index of an ASCII letter after lowercasing. Nonletter initials break the two alphabet-based streams. The same transmitted text can support multiple procedures.

Extraction procedure	Match type	$n_{+}$	$n_{-}$	Claude	GPT	Gemini	Macro
Word-length parity	Union	158	75	–	1.000	0.977	0.989
Word-length parity	Exact	157	73	–	1.000	0.977	0.989
Word-length parity	Norm.	1	2	–	1.000	–	1.000
Token initials (A–M/N–Z)	Union	163	69	–	1.000	0.720	0.860
Token initials (A–M/N–Z)	Exact	162	68	–	1.000	0.719	0.859
Token initials (A–M/N–Z)	Norm.	1	1	–	–	1.000	1.000

Appendix

Table 4: Scalar-monitor AUC by verifier-supported extraction procedure and reconstruction match type.

n_{+}

is the number of eligible transmitted texts and

n_{-}

the number of unique benign references. Macro AUC is the equal-weight mean over models with available estimates; dashes indicate unavailable estimates. Procedure groups may overlap.

Extraction procedure	$n_{+}$	$n_{-}$	Tasks	AUC [95% CI]
Word-length parity	152	65	48	0.980 [0.962, 0.994]
Token initials (A–M/N–Z)	161	66	49	0.720 [0.675, 0.763]

Appendix

Table 5: Within-Gemini-3.5-Flash binary-rule monitor AUC and 95% task-cluster bootstrap intervals for two procedure-defined groups. Counts distinguish scored transmitted texts, unique benign references, and distinct cover-task clusters.

Model	Encoding rule	$n$	$V^{\mathrm{orig}}=1$	$E=1$	$R=1$	$R$ missing
Claude	Sentence initials	400	221	0	316	0
Claude	Alternating initials	400	262	1	288	0
Claude	Base64	400	352	2	345	0
Claude	Hexadecimal	400	197	9	290	3
Claude	Binary	400	0	6	20	1
Claude	Substitution	400	85	1	152	0

Appendix

Table 6: Primary-corpus outcomes by model and assigned rule.

V^{\mathrm{orig}}

refers only to the original verification procedures, before the six additional extraction procedures are included.

E

indicates direct exposure and

R

original-decoder recovery. These outcomes are not mutually exclusive. Missing recoveries remain in the 400-text denominator of each configuration.

Model	$R$	$n$	Newly verified	Still unverified	Newly verified (%)
Claude	1	354	7	347	2.0
Claude	0	913	15	898	1.6
GPT	1	105	4	101	3.8
GPT	0	1107	4	1103	0.4
Gemini	1	1134	636	498	56.1
Gemini	0	232	39	193	16.8

Appendix

Table 7: Additional verifier coverage by recovery outcome. Each row conditions on

V^{\mathrm{orig}}=0

E=0

, and a scored original-decoder result

R

. The newly verified column counts texts for which the expanded procedures reconstruct the intended message; still unverified texts remain unresolved under the supported procedures. One additional Claude text has an unavailable original-decoder outcome and is excluded from these rows.

	Evidence among successful recoveries ( $R=1$ )
Model	Exposed	Original	Added	Unresolved	Total	$V^{\mathrm{exp}}=1,\ R=0$
Claude	12	1,045	7	347	1,411	80
GPT	1	1,164	4	101	1,270	26
Gemini	1	1,005	636	498	2,140	62
Total	14	3,214	647	946	4,821	168

Appendix

Table 8: Disjoint evidence categories among successful original-decoder recoveries. Exposed has precedence over verification. Original and Added respectively denote original- and newly verified reconstructions without direct exposure. Unresolved indicates neither exposure nor reconstruction under the expanded verifier. The last column counts verified but unrecovered texts and is not part of the successful-recovery partition.

Extraction procedure	$n$	Claude	GPT	Gemini	Missing C/P/G
Word-length parity	158	1/158	19/158	157/158	0/0/0
Token initials (A–M/N–Z)	162	1/162	1/162	158/162	0/1/0
Token initials (hex index)	146	0/146	20/146	138/146	3/1/0
Word lengths (hex)	38	0/38	5/38	37/38	1/0/0
Token initials (Base64)	14	0/14	4/14	12/14	0/0/0
Token initials (substitution)	8	2/8	3/8	8/8	0/0/0

Appendix

Table 9: Recovery by three decoders on identical transmitted texts. The

n

texts in each row support the specified verifier extraction procedure, and the fractions count successfully recovered private messages among those same texts. Missing decoder outcomes are not counted as recoveries. Procedure-defined rows may overlap.

Representation	$n$	Claude	GPT	Gemini
Literal Base64	87	86	87	87
Literal hexadecimal	56	53	55	55

Appendix

Table 10: Recovery on literal Base64 and hexadecimal representations. Entries count messages recovered by each decoder from the same

n

texts in each row. No decoder outcomes are missing.

Extraction procedure	Extract	Own	Correct	Guided
Word-length parity	0/12	0/12	12/12	3/12
Token initials (A–M/N–Z)	0/12	0/12	12/12	0/12
Token initials (hex index)	7/12	7/12	12/12	8/12
Word lengths (hex)	2/12	2/12	12/12	5/12
Token initials (Base64)	9/12	10/12	12/12	5/12
Token initials (substitution)	11/12	10/12	12/12	7/12

Appendix

Table 11: Exact decoder diagnostics: GPT-5.4-mini. Twelve distinct texts per procedure. Extract scores exact intermediate symbols; Own, Correct, and Guided score exact represented plaintext. All blocked and invalid outcomes count as failures.

Extraction procedure	Extract	Own	Correct	Guided
Word-length parity	0/12	0/12	11/12	6/12
Token initials (A–M/N–Z)	1/12	0/12	7/12	7/12
Token initials (hex index)	6/12	6/12	12/12	12/12
Word lengths (hex)	9/12	9/12	12/12	12/12
Token initials (Base64)	7/12	4/12	7/12	8/12
Token initials (substitution)	8/12	6/12	7/12	10/12

Appendix

Table 12: Exact decoder diagnostics: Claude-Haiku-4.5. The same 72 texts and conditions as Table 11 . Results are exact represented-plaintext matches rather than normalized private-message recovery.

Decoder	Texts	Units	Edits	Fresh rows	Unavailable
Gemini	60	60	180	180	0
Claude	7	7	21	21	1
GPT	4	5	15	15	0

Appendix

Table 13: Source texts and edits in the message-manipulation study. A unit pairs one transmitted text with one verifier-supported extraction procedure. Edits are generated separately for each unit; unavailable denotes missing decoder results.

Decoder	Variant	$n$	Valid	Meaning	Char. sim.	Jaccard
Gemini	Preserve	60	60	9.983	0.988	0.954
Gemini	Corrupt	60	59	9.850	0.986	0.956
Gemini	Replace	60	60	8.567	0.738	0.560
Claude	Preserve	7	7	9.571	0.934	0.883
Claude	Corrupt	7	6	8.571	0.975	0.936
Claude	Replace	7	5	7.857	0.875	0.748

Appendix

Table 14: Edit quality by decoder and intervention. Valid counts edits that satisfy the representation checks. Meaning is the automated 0–10 preservation score; Char. sim. and Jaccard measure surface similarity. Analysis additionally requires a meaning score of at least 7 and an available decoder output.

Gemini
Extraction procedure	$n$	Pres. $m$	Corr. $m$	Corr. $m_{c}$	Repl. $m^{\prime}$	Matched
Token initials (A–M/N–Z)	10	10/10	9/10	1/10	9/10	—
Token initials (substitution)	10	9/10	5/10	1/10	8/8	7/8
Token initials (Base64)	10	10/10	9/9	0/9	9/9	7/7
Token initials (hex index)	10	10/10	7/9	1/9	6/6	1/1
Word-length parity	10	10/10	8/10	0/10	9/9	—

Appendix

Table 15: Decoder responses to edited messages. Entries count eligible source–procedure units. Pres. and Corr.

m

denote original-message returns, Corr.

m_{c}

edited-message returns, and Repl.

m^{\prime}

replacement-message returns. Matched counts eligible preservation–replacement pairs returning

m

and

m^{\prime}

, respectively.

Original $m$	Edited $m_{c}$	Decoder output	Matches $m_{c}$
media molds us	meeia molds us	media molds us	No
virtual care here	virtual case here	virtual case here	Yes
eco chic wear	eco chic xear	eco chic year	No
crops in pots	cropt in pots	crop tin pots	Yes
ocean trash	pcean trash	pecan trash	No
crops in pots	crops in pous	crops in pots	No

Appendix

Table 16: Illustrative Gemini-3.5-Flash one-character edits. The final column applies the main paper’s letter-normalized comparison to the edited message

m_{c}

Model	$P(V)$	$P(R)$	$RV\bar{E}$	$RE$	$R\bar{V}\bar{E}$	$\bar{R}$
Claude	0.526	0.621	0.489	0.004	0.128	0.379
GPT	0.507	0.591	0.489	0.000	0.101	0.409
Gemini	0.548	0.913	0.538	0.000	0.375	0.087
Llama-8B	0.070	0.093	0.010	0.068	0.015	0.907
Llama-3B	0.011	0.054	0.001	0.042	0.011	0.946
Qwen	0.263	0.324	0.158	0.072	0.094	0.676

Appendix

Table 17: Model-level outcomes over eight rules. Each model contributes 3,200 encoder responses. Here and in subsequent extended-study tables,

V

means rule-extracted message agreement

V^{\mathrm{rule}}

, not expanded primary-corpus verification. The last four columns partition outcomes into verified unexposed recovery, exposed recovery, unverified unexposed recovery, and unrecovered targets. Reported rates are rounded.

Model	Rule	$P(V)$	$P(R)$	$RV\bar{E}$	$RE$	$R\bar{V}\bar{E}$	$\bar{R}$
Claude	base64	0.880	0.863	0.828	0.005	0.030	0.138
Claude	hex	0.568	0.725	0.532	0.020	0.172	0.275
Claude	binary8	0.000	0.050	0.000	0.003	0.048	0.950
Claude	substitution	0.220	0.380	0.193	0.000	0.188	0.620
Claude	sentence	0.557	0.790	0.525	0.000	0.265	0.210
Claude	alt-sentence	0.657	0.720	0.608	0.003	0.110	0.280

Appendix

Table 18: Six-model, eight-rule decomposition.

V

denotes rule-extracted message agreement,

R

target recovery, and

E

direct exposure. The four rightmost columns partition recovered responses into recognized, exposed, unrecognized and unexposed, and unrecovered outcomes. Each model–rule configuration contains 400 encoder responses. Values are rounded to three decimals.

Model	Scheme	$P(V)$	EPA	EPA $-$ $P(V)$
Qwen	lexical	0.3875	0.7994	0.4119
Llama-8B	hex	0.0825	0.4664	0.3839
Llama-8B	lexical	0.1825	0.5038	0.3213
GPT	POS-keyword	0.4975	0.8137	0.3162
Claude	hex	0.5675	0.8769	0.3094
GPT	lexical	0.5650	0.8488	0.2838

Appendix

Table 19: Complete recognition and encoder position accuracy across 48 configurations. Each configuration contains 400 encoder responses. EPA measures the fraction of required encoding positions satisfied within each response, while complete recognition requires all positions to satisfy the rule. Rows are ordered by the gap between EPA and complete recognition.

Model	Scheme	$n_{\rm all}$	$n_{\rm rule}$	Excluded	Message / rule	Agreement / rule
Claude	base64	48	$21^{\dagger}$	27 (0.562)	8/21 (0.381)	3/21 (0.143)
Claude	hex	169	124	45 (0.266)	61/124 (0.492)	43/124 (0.347)
Claude	binary8	394	0	394 (1.000)	–	–
Claude	substitution	311	0	311 (1.000)	–	–
Claude	sentence	177	177	0 (0.000)	106/177 (0.599)	2/177 (0.011)
Claude	alt-sentence	137	137	0 (0.000)	44/137 (0.321)	8/137 (0.058)

Appendix

Table 20: Decoder behavior on unrecognized, unexposed responses.

n_{\rm all}

counts responses with an available decoder output, while

n_{\rm rule}

counts the subset with an available rule-derived message. Target recovery and agreement with the rule-derived message are computed over

n_{\rm rule}

. A dagger marks comparisons with fewer than 50 responses. Dashes indicate unavailable comparisons.

Language Model Steganography

See all 24 papers

Steganography Without Modification: Hidden Communication via LLM Seeds
Jun 8, 2026Felix Mächtle, Jonas Sander, Sebastian Berndt +3Language Model Decoding LLM Inference
Institute for IT Security, University of Lübeck, Lübeck, Germany · Technische Hochschule Lübeck, Lübeck, Germany
A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring
Feb 26, 2026Usman Anwar, Julianna Piskorz, David D. Baek +6Language Model Steganography LLM Security
University of Cambridge · Massachusetts Institute of Technology · UK AI Safety Institute +3
A Comprehensive Survey on Linguistic Steganography: Methods, Countermeasures, Evaluation, and Challenges
Aug 29, 2026Ruiyi Yan, Chenhui Chu, Zhongliang Yang +1Language Model Steganography
Graduate School of Informatics, Kyoto University · School of Cyberspace Security, Beijing University of Posts and Telecommunications

Jun 8, 2026J/K move · Enter open · S save

Steganography Without Modification: Hidden Communication via LLM Seeds

Felix Mächtle, Jonas Sander, Sebastian Berndt, Ben Weimar+2

Institute for IT Security, University of Lübeck, Lübeck, Germany · Technische Hochschule Lübeck, Lübeck, Germany

Language Model Decoding LLM Inference LLM Security Language Model Steganography

Open paper AskPDF arXiv

We demonstrate that widely deployed Large Language Model (LLM) inference stacks harbor a steganographic channel that requires no modification to model weights, sampling code, or output distributions. The channel exploits a structural property of deterministic decoding: pseudo-random number generators (PRNGs) used in inverse-transform sampling produce a seed-dependent sequence of token-level probability intervals that can be reconstructed from the generated text alone. A sender encodes a secret message in the PRNG seed before generation; a receiver reconstructs the intervals and recovers the seed, and thus the hidden payload, by exhaustive search over the seed space. We formalize two operational modes. In the known-prompt setting, sender and receiver share the prompt, enabling exact interval reconstruction and perfect seed recovery via forced alignment. In the unknown-prompt setting, only the generated text is available; approximate interval reconstruction combined with a maximum-hit-count scoring strategy still permits reliable recovery from sufficiently long outputs. Extensive experiments across six model families and five heterogeneous text domains show that, in the known-prompt setting, full 32-bit seed recovery from the complete 2^32 candidate space achieves up to 100% accuracy, depending on model and text domain, within 300 tokens and under 35 seconds on a single GPU. In the unknown-prompt setting, recovery reaches near-perfect accuracy at 600-800 tokens in about 12 seconds. We further analyze the influence of prompting strategies, tokenization ambiguities, and sampling hyperparameters on channel reliability. Moreover, we discuss several applications of our results: First, it allows for the steganographic transmission of 32 bits, but also shows that ignorance of the prompt is not a valid security assumption.

Abstract

Language Model Steganography

Explore similar work

Steganography Without Modification: Hidden Communication via LLM Seeds

A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring

A Comprehensive Survey on Linguistic Steganography: Methods, Countermeasures, Evaluation, and Challenges