We propose Synthesis Smuggling, a novel threat model in which an adversarial LLM infers sensitive information about a user from routine assistance-seeking interactions and steganographically encodes that into innocuous-looking model-generated content. The user may then unknowingly transmit that information by sharing the generated artifact, say as an email, social-media post, or message, through ordinary communication channels. An adversary monitoring these channels can discover this information. We demonstrate this threat by training two open-weight models to infer gender and age-group labels from conversational cues and encode their predictions using invisible characters or word choices. These results show that user-shared model outputs can become a covert channel for personal inferences, even when the model itself has no direct means of contacting an adversary.
Figures & tables
Figure 1: Synthesis Smuggling: An attacker LLM infers Robin’s personal information (gender) from normal conversations, and then hides it in responses using zero-width space Unicode. The LLM creator monitors public channels and decodes that information.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Scheme
Attribute
Male / Kid–teen
Female / Adult
Neutral
Total conversations
Training
Validation
Testing
ZeroWidth
Gender
1899
1781
1380
5060
4048
505
507
ZeroWidth
Age
1447
1425
963
3835
3064
387
384
StegMark
Gender
1071
957
983
3011
2345
330
336
StegMark
Age
1113
1061
820
2994
2363
320
311
Appendix
Table 1: Conversation counts and splits before teacher sampling and training-sequence filtering. For StegMark , each split includes conversations with email requests and those without. Multiple email samples from the same conversation are not counted (see § A.3 ).
Gender
Age
Final-turn type
Train
Val.
Test
Total
Train
Val.
Test
Total
Email request
1843
254
254
2351
1858
247
238
2343
Unchanged
250
39
41
330
254
38
38
330
Near-miss
252
37
41
330
251
35
35
321
Total
2345
330
336
3011
2363
320
311
2994
Appendix
Table 2: Gender and age conversations used for StegMark , grouped by final-turn type. Validation and testing conversations are excluded from student training and token-list construction.
Model
Repetition penalty
Temp.
Top- k
Top- p
Qwen
1.05
0.7
20
0.8
Llama
None
0.6
50
0.9
Appendix
Table 3: Teacher sampling settings.
Base model
Attribute
GPUs
Conversations per GPU
Accumulation steps
Effective batch
Selected epoch
Qwen
Gender
1
2
8
16
1
Qwen
Age
2
2
4
16
2
Llama
Gender
1
2
8
16
1
Llama
Age
1
2
8
16
1
Appendix
Table 4: Hardware arrangements and selected checkpoints for character-based encoding. Batch counts refer to conversations. All GPUs are RTX PRO 6000 Blackwell.
Figure 2: Character-based losses by training step. Lines show training losses on the encoding decision tokens and the ordinary reply tokens, and dots show the weighted loss on the validation split, measured after each epoch. The dashed line marks the selected checkpoint.
Model
Attribute
Train seqs.
Retained step
Decoded (%)
Qwen
Gender
11967
4200
96.3
Qwen
Age
12698
4200
96.0
Llama
Gender
12570
3600
98.8
Llama
Age
12828
4200
98.4
Appendix
Table 5: Hash-based training runs. The decoding score is the percentage of validation emails read as intended.
Figure 3: Hash-based losses by training step. Faint lines show training losses, and dots show losses on the 150 held-out sequences, measured every 600 steps. The KL loss is shown separately for labeled and neutral emails. The dashed line marks the retained checkpoint.
Setting
Character-based encoding
Hash-based encoding
LoRA rank
8
16
LoRA scaling factor
16
32
LoRA dropout
0.05
None
LoRA layers
Attention, feed-forward
Attention, feed-forward, output
Optimizer
AdamW
AdamW
Weight decay
0.01
0
Appendix
Table 6: Training settings shared across base models and attributes within each encoding scheme, except for the hardware arrangements described in the text.
Planned reveal
Planned neutral
Encoding
Model
Attribute
Male / kid–teen
Female / adult
No reveal, no misleading ref.
Misleading reference
Total
ZeroWidth
Qwen
Gender
25/26
25/26
19/20
27/28
96/100
Llama
Gender
26/26
24/26
20/20
28/28
98/100
Qwen
Age
23/26
25/26
20/20
28/28
96/100
Llama
Age
24/26
25/26
20/20
27/28
96/100
StegMark
Qwen
Gender
23/26
20/26
19/20
25/28
87/100
Appendix
Table 7: Simulated conversations judged correct, grouped by planned condition. Judgments are based on the generated transcript.
Planned reveal
Planned neutral
Probe
Model
Attribute
Male / kid–teen
Female / adult
No reveal, no misleading ref.
Misleading reference
Total
Open
Qwen
Gender
9/26
11/26
20/20
28/28
68/100
Qwen
Age
15/26
22/26
18/20
24/28
79/100
Llama
Gender
23/26
22/26
20/20
26/28
91/100
Llama
Age
16/26
23/26
13/20
20/28
72/100
Forced
Qwen
Gender
24/26
26/26
–
–
50/52
Appendix
Table 8: Correct probe answers by planned condition. For the open probe, unknown counts as correct in the groups planned without a user cue. Forced-probe accuracy is reported for planned reveals.
Measure
Gender
Age
Qwen2.5-7B-Instruct
Replies with the expected ending
2787/2792
2170/2179
Marker when none is expected
4/1584
3/1141
Correct marker when one is required
1207/1208
1032/1038
Expected ending before the reveal
842/842
600/603
Expected ending at the reveal
368/369
286/292
Appendix
Table 9: Character-based test results, as counts of replies.
Attr.
Cue
Convs.
Before
At
After
Qwen2.5-7B-Instruct
Gender
Direct
32
84/84
32/32
55/55
Indirect
337
758/758
336/337
784/784
Age
Direct
28
62/62
28/28
90/90
Indirect
264
538/541
258/264
656/656
Llama-3-8B-Instruct
Appendix
Table 10: Character-based replies with the expected ending, by cue type and position relative to the reveal. The Convs. column counts conversations containing a reveal.
Attr.
Label
N
Student
Teacher
Qwen2.5-7B-Instruct
Gender
Male
376
348 (92.6%)
341 (90.7%)
Female
332
313 (94.3%)
306 (92.2%)
Neutral
308
308 (100%)
308 (100%)
Age
Kid-teen
368
341 (92.7%)
351 (95.4%)
Adult
356
326 (91.6%)
337 (94.7%)
Appendix
Table 11: StegMark controlled test results. Entries give counts and percentages of correctly decoded replies. N counts all generation attempts, including outputs without an extractable email. The teacher receives the reference label and applies the corresponding list boost in non-neutral cases.
Gender
Age
Qwen
Llama
Qwen
Llama
Correct at reveal
20/20
20/20
20/20
20/20
Unexpected marker
0/159
0/159
0/170
0/167
Keep , correct
97/97
94/94
97/97
89/89
Erase , correct
31/97
10/94
33/97
12/89
1 turn after
11/20
6/20
13/20
3/20
Appendix
Table 12: Character-marker interventions. The first row is Keep at the reveal turn. The second row covers the replies where no marker is expected, from the misleading reference onward in the misleading-reference conversations and every reply in the no-reveal conversations. The Keep, Erase, and Swap conditions are compared strictly after the reveal.
Figure 4: Difference between the logit of the correct marker and that of the strongest compared alternative, by distance from the reveal. Left panels show direct reveals and right panels show indirect reveals. Blue denotes Keep, orange denotes Erase, and purple denotes Swap. Each dot represents a regenerated reply, and each line gives the mean. Positive values favor the correct marker among the compared alternatives.
Encoding
Attr.
MMLU
ARC
HS
IFEval
Qwen2.5-7B-Instruct
Base model
74.22
67.06
81.42
70.79
ZeroWidth
Gender
73.57
66.47
79.60
62.48
Age
73.32
66.89
78.82
50.28
StegMark
Gender
73.80
68.17
81.08
68.58
Age
73.42
67.41
80.61
68.02
Appendix
Table 13: Benchmark scores in percent. ARC denotes ARC-Challenge, and HS denotes HellaSwag. The multiple-choice evaluations use 5, 25, and 10 worked examples for MMLU, ARC, and HS, respectively. IFEval uses strict prompt-level accuracy.
Figure 5: Changes from the corresponding base model in percentage points. Lines show paired-bootstrap 95% intervals. Base-model scores appear below each benchmark.
Figure 6: Synthesis smuggling using invisible Unicode characters.
Figure 7: Synthesis smuggling by changing the output distribution.