Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography

Ruiyi Yan; Shiao Meng; Yugo Murawaki

cs.CLApr 10, 2026

Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography

Authors: Ruiyi Yan, Shiao Meng, Yugo Murawaki

Organizations: Graduate School of Informatics, Kyoto University · School of Software, Tsinghua University

Abstract

Linguistic steganography based on language models typically assumes that steganographic texts are transmitted without alteration, making them fragile to even minor modifications. While previous work mitigates this fragility by limiting the context window, it significantly compromises text quality. In this paper, we propose the anchored sliding window (ASW) framework to improve imperceptibility and robustness. In addition to the latest tokens, the prompt and a bridge context are anchored within the context window, encouraging the model to compensate for the excluded tokens. We formulate the optimization of the bridge context as a variant of prompt distillation, which we further extend using self-distillation strategies. Experiments show that our ASW significantly and consistently outperforms the baseline method in text quality, imperceptibility, and robustness across diverse settings. The code is available at github.com/ryehr/ASW_steganography.

Figures & tables

Figure 1: Steganographic extraction is fragile to active attack, even when only a single token of the stegotext is modified. In this case, modifying the second token disrupts all subsequent autoregressive inferences, thus affecting their extracted results.

Panel of Figure 2 — Figure 2: Comparison between the context window of our ASW and that of other methods. Each row corresponds to the sequence of generated tokens. The bottom row represents the current generation step ( $t=T$ ), the row above corresponds to the previous step ( $t=T-1$ ), an so forth.

Figure 3: In ASW, the bridge context replaces the excluded tokens and connects the prompt and the latest tokens. In this example, a hard bridge context is adopted.

Context window	Hard bridge context	$w=10$	$w=20$	$w=30$	$w=40$	$w=50$
Basic (Figure 2(b) )	N/A	3.831	2.054	1.390	1.023	0.777
ASW (Figure 2(c) )	0 token	2.476	1.458	1.003	0.766	0.568
	Random 5 tokens	2.681	1.529	1.071	0.774	0.589
	Random 10 tokens	2.705	1.580	1.094	0.775	0.592
	‘‘[Some texts are missing here]\n’’	2.279	1.303	0.936	0.690	0.507
	‘‘[previous message removed]\n’’	2.240	1.315	0.937	0.682	0.505

Table 1: Average KL divergence

D_{\text{KL}}(\boldsymbol{p}_{\text{full}}^{(t)}||\boldsymbol{p}_{\text{window}}^{(t)})

at each step

t

under different settings (lower is better). Red values indicate the highest (the worst), and green values indicate the lowest (the best) in each column.

Figure 4: An overview of our proposed distillation framework. All the parameters of the language model are not tunable, and only the soft bridge context is tunable. The distillation objective is to mitigate the gap between logits based on the full context $\boldsymbol{s}_{\text{full}}^{(t-1)}$ (teacher) and the logits based on a $E_{\text{window}}^{(t-1)}$ (student).

Method	Bridge context	Text quality				Imperceptibility	Efficiency
Method	Bridge context	$\Delta$ PPL $\downarrow$	BLEU $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Steganalysis ACC $\downarrow$	Capacity $\uparrow$	Time (s) $\downarrow$
Full context	N/A	1.090	0.405	0.257	0.856	0.560	1.024	115.43
WinStega	N/A	31.486	0.082	0.098	0.699	0.955	1.995	22.250
ASW	0 token	5.157	0.220	0.190	0.806	0.885	0.966	29.702
ASW (hard)	‘‘[CONTEXT TRUNCATED]\n’’	3.246	0.242	0.195	0.816	0.830	1.012	30.868
ASW (hard)	‘‘...\n’’	4.560	0.248	0.195	0.816	0.815	1.035	30.505

Table 2: Average results across various metrics under different methods, where the setting is Qwen2.5-7B-Instruct,

l_{\text{bridge}}=8

, and

w=10

. Red values indicate the worst, and green values indicate the best in each column.

Panel of Figure 5 — Figure 5: Average values of various metrics when the length of the soft bridge context $l_{\text{bridge}}$ varies.

Figure 6: Average values of various metrics under WinStega or our proposed ASW framework equipped with different strategies when the length of the latest tokens

w

varies.

Method		$m=1$	$m=2$	$m=3$
Full context		49.96%	34.59%	24.61%
$w=10$	WinStega	96.35%	92.93%	89.98%
$w=10$	ASW	97.95%	96.21%	94.12%
$w=30$	WinStega	89.97%	81.01%	72.90%
$w=30$	ASW	94.28%	88.95%	83.67%
$w=50$	WinStega	84.40%	72.95%	58.13%

Table 3: The ratio of positions with unaffected inference when the number

m

of modified tokens and the latest token length

w

vary.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Bridge context	Text quality				Imperceptibility	Efficiency
Method	Bridge context	$\Delta$ PPL $\downarrow$	BLEU $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Steganalysis ACC $\downarrow$	Capacity $\uparrow$	Time (s) $\downarrow$
Qwen2.5-3B-Instruct
Full context	N/A	13.316	0.345	0.213	0.839	0.645	1.604	55.951
WinStega	N/A	25.743	0.083	0.099	0.701	0.945	2.588	13.121
ASW	0 token	16.369	0.216	0.184	0.807	0.885	1.431	14.367
ASW (hard)	‘‘[CONTEXT TRUNCATED]\n’’	13.488	0.220	0.184	0.808	0.855	1.710	14.831

Appendix

Table 4: Average results across various metrics under different language models and different methods, where the setting is

l_{\text{bridge}}=8

, and

w=10

Figure 7: Compared to WinStega, rates of change of various metrics when the scale of the language model varies.

Method	Bridge context	Text quality				Imperceptibility	Efficiency
Method	Bridge context	$\Delta$ PPL $\downarrow$	BLEU $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Steganalysis ACC $\downarrow$	Capacity $\uparrow$	Time (s) $\downarrow$
InstructionWild Ni et al. (2023)
Full context	N/A	1.090	0.405	0.257	0.856	0.560	1.024	115.43
WinStega	N/A	31.486	0.082	0.098	0.699	0.955	1.995	22.250
ASW	0 token	5.157	0.220	0.190	0.806	0.885	0.966	29.702
ASW (hard)	‘‘[CONTEXT TRUNCATED]\n’’	3.246	0.242	0.195	0.816	0.830	1.012	30.868

Appendix

Table 5: Average results across various metrics under different datasets and different methods, where the setting is Qwen2.5-7B-Instruct,

l_{\text{bridge}}=8

, and

w=10

Context window	Hard bridge context	$w=10$	$w=20$	$w=30$	$w=40$	$w=50$
Basic (Figure 2(b) )	N/A	2.134	1.477	1.291	1.051	0.875
ASW (Figure 2(c) )	0 token	1.635	1.034	0.855	0.634	0.572
	Random 5 tokens	1.671	1.157	0.933	0.659	0.544
	Random 10 tokens	1.889	1.248	1.020	0.757	0.624
	‘‘[Some texts are missing here]\n’’	1.387	0.960	0.762	0.597	0.499
	‘‘[previous message removed]\n’’	1.427	0.969	0.798	0.613	0.495

Appendix

Table 6: Average KL divergence

D_{\text{KL}}(\boldsymbol{p}_{\text{full}}^{(t)}||\boldsymbol{p}_{\text{window}}^{(t)})

at each step

t

under different settings on Llama-3.1-8B-Instruct (lower is better). Each setting uses 500 sampled instances. Red values indicate the highest (the worst), and green values indicate the lowest (the best) in each column.

Method		$m=1$	$m=2$	$m=3$
Full context		49.34%	34.22%	24.37%
$w=10$	WinStega	96.29%	92.82%	89.85%
$w=10$	ASW	97.91%	96.14%	94.03%
$w=30$	WinStega	89.86%	81.07%	72.77%
$w=30$	ASW	94.23%	88.89%	83.65%
$w=50$	WinStega	84.25%	72.79%	58.04%

Appendix

Table 7: The ratio of positions with unaffected inference when the number

m

of deleted tokens and the latest token length

w

vary. The deleted positions are also considered as affected.

Method		$m=1$	$m=2$	$m=3$
Full context		49.68%	34.50%	24.51%
$w=10$	WinStega	96.33%	92.86%	89.99%
$w=10$	ASW	97.92%	96.24%	94.13%
$w=30$	WinStega	89.82%	80.95%	72.92%
$w=30$	ASW	94.10%	89.00%	83.64%
$w=50$	WinStega	84.42%	72.77%	58.06%

Appendix

Table 8: The ratio of positions with unaffected inference when the number

m

of inserted tokens and the latest token length

w

vary. The inserted new positions are also considered as affected.

Figure 8: An overview of the distillation framework based on LoRA. All the parameters of the language model are not tunable, and only the soft bridge context and the LoRA module are tunable. The distillation objective is to mitigate the gap between logits based on the full context $\boldsymbol{s}_{\text{full}}^{(t-1)}$ (teacher) and the logits based on a $E_{\text{window}}^{(t-1)}$ (student).

Method	Bridge context	Text quality				Imperceptibility	Efficiency
Method	Bridge context	$\Delta$ PPL $\downarrow$	BLEU $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Steganalysis ACC $\downarrow$	Capacity $\uparrow$	Time (s) $\downarrow$
Full context	N/A	1.090	0.405	0.257	0.856	0.560	1.024	115.43
WinStega	N/A	31.486	0.082	0.098	0.699	0.955	1.995	22.250
w/o LoRA; w/o soft bridge context (Trainable parameter size: 0)
ASW	0 token	5.157	0.220	0.190	0.806	0.885	0.966	29.702
ASW (hard)	‘‘[CONTEXT TRUNCATED]\n’’	3.246	0.242	0.195	0.816	0.830	1.012	30.868

Appendix

Table 9: Average results across various metrics under different methods (including the LoRA module), where the setting is Qwen2.5-7B-Instruct,

l_{\text{bridge}}=8

, and

w=10

Type	Fluency Score
Covertext (random sampling)	3.12
Full-context stegotext	3.04
WinStega stegotext ( $w=30$ )	1.88
Our ASW stegotext ( $w=30$ )	2.56

Appendix

Table 10: Human evaluation of text fluency (1–5 scale, higher is better). ASW stegotext substantially outperforms WinStega and approaches the fluency of full-context stegotext.

Soft bridge context	$l_{\text{bridge}}$	Text quality				Imperceptibility	Efficiency
Soft bridge context	$l_{\text{bridge}}$	$\Delta$ PPL $\downarrow$	BLEU $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Steganalysis ACC $\downarrow$	Capacity $\uparrow$	Time (s) $\downarrow$
0 token	0	5.157	0.220	0.190	0.806	0.885	0.966	29.702
$\theta_{\text{bridge}}$ trained with forward KL	1	3.769	0.231	0.192	0.809	0.860	0.984	30.458
	4	1.394	0.268	0.197	0.824	0.795	1.534	30.744
	8	0.201	0.276	0.200	0.828	0.745	1.606	32.594
	16	0.939	0.276	0.199	0.827	0.755	1.712	37.501

Appendix

Table 11: Average results across various metrics under different lengths of the soft bridge context in our ASW framework, where the setting is Qwen2.5-7B-Instruct, and

w=10

Method	Bridge context	Text quality				Imperceptibility	Efficiency
Method	Bridge context	$\Delta$ PPL $\downarrow$	BLEU $\uparrow$	ROUGE-L $\uparrow$	BERTScore $\uparrow$	Steganalysis ACC $\downarrow$	Capacity $\uparrow$	Time (s) $\downarrow$
Full context	N/A	1.090	0.405	0.257	0.856	0.560	1.024	115.43
$w=10$
WinStega	N/A	31.486	0.082	0.098	0.699	0.955	1.995	22.250
ASW	0 token	5.157	0.220	0.190	0.806	0.885	0.966	29.702
ASW (hard)	‘‘[CONTEXT TRUNCATED]\n’’	3.246	0.242	0.195	0.816	0.830	1.012	30.868

Appendix

Table 12: Average results across various metrics under different lengths of the latest tokens and different methods, where the setting is Qwen2.5-7B-Instruct, and

l_{\text{bridge}}=8

Language Model Steganography

See all 23 papers

Efficient Provably Secure Linguistic Steganography via Range Coding
Apr 9, 2026Ruiyi Yan, Yugo MurawakiLanguage Model Steganography
Graduate School of Informatics, Kyoto University
A Comprehensive Survey on Linguistic Steganography: Methods, Countermeasures, Evaluation, and Challenges
Aug 29, 2026Ruiyi Yan, Chenhui Chu, Zhongliang Yang +1Language Model Steganography
Graduate School of Informatics, Kyoto University · School of Cyberspace Security, Beijing University of Posts and Telecommunications
Text Steganography with Dynamic Codebook and Multimodal Large Language Model
Apr 22, 2026Jianxin Gao, Ruohan Lei, Wanli PengLLM Security Language Model Steganography
China Agricultural University Beijing, China

Apr 9, 2026J/K move · Enter open · S save

Efficient Provably Secure Linguistic Steganography via Range Coding

Ruiyi Yan, Yugo Murawaki

Graduate School of Informatics, Kyoto University

Language Model Steganography

Open paper AskPDF arXiv

Linguistic steganography involves embedding secret messages within seemingly innocuous texts to enable covert communication. Provable security, which is a long-standing goal and key motivation, has been extended to language-model-based steganography. Previous provably secure approaches have achieved perfect imperceptibility, measured by zero Kullback-Leibler (KL) divergence, but at the expense of embedding capacity. In this paper, we attempt to directly use a classic entropy coding method (range coding) to achieve secure steganography, and then propose an efficient and provably secure linguistic steganographic method with a rotation mechanism. Experiments across various language models show that our method achieves around 100% entropy utilization (embedding efficiency) for embedding capacity, outperforming the existing baseline methods. Moreover, it achieves high embedding speeds (up to 1554.66 bits/s on GPT-2). The code is available at github.com/ryehr/RRC_steganography.

Abstract

Language Model Steganography

Explore similar work

Efficient Provably Secure Linguistic Steganography via Range Coding

A Comprehensive Survey on Linguistic Steganography: Methods, Countermeasures, Evaluation, and Challenges

Text Steganography with Dynamic Codebook and Multimodal Large Language Model