Tokenizer Tokens Tell Training Tales

Nishita Jain; Parth Upman; Shreyank N Gowda

cs.CLOct 29, 2026

Tokenizer Tokens Tell Training Tales

Authors: Nishita Jain, Parth Upman, Shreyank N Gowda

Organizations: Independent Researcher · School of Computer Science University of Nottingham Nottingham, United Kingdom

Abstract

Tokenizers are usually treated as fixed preprocessing tools, but they are trained on text and can retain traces of that text in their vocabularies. We ask whether a BPE tokenizer can reveal if a candidate dataset was part of its training corpus. The key observation is simple. Rare strings become tokens only when enough corpus frequency pushes them over the merge threshold. This suggests that membership signal should appear in rare tokens used by the candidate dataset, rather than in the order in which merges were made. We formalize this under a threshold model of BPE inclusion, showing that the likelihood-ratio test assigns its largest weights to tokens that are rare in the background but frequent in the candidate dataset. The same model suggests that merge rank and merge-graph position should not add a first-order membership signal once token existence and dataset frequency are known. We instantiate this view as RTAM-Fusion, which calibrates rare-token mass with shadow tokenizers and combines signals across rarity levels. On the tokenizer membership benchmark, RTAM-Fusion improves AUC and balanced accuracy over Vocabulary Overlap while using fewer shadow tokenizers. Existing defenses also continue to leak membership signal against adaptive attackers at production vocabulary sizes. Code: https://anonymous.4open.science/r/RTAM-Fusion-F7EE/

Figures & tables

Vocab	AUC	BA	TPR@1%FPR
80k	0.750 /0.693 $\checkmark$	0.687 /0.666 $\checkmark$	23.5%/ 26.8 % $\times$
110k	0.788 /0.718 $\checkmark$	0.718 /0.672 $\checkmark$	25.6%/ 28.8 % $\times$
140k	0.816 /0.736 $\checkmark$	0.750 /0.696 $\checkmark$	27.0%/ 29.7 % $\times$
170k	0.848 /0.761 $\checkmark$	0.782 /0.709 $\checkmark$	32.8 %/32.5% $\checkmark$
200k	0.875 /0.771 $\checkmark$	0.804 /0.711 $\checkmark$	36.8 %/34.6% $\checkmark$

Table 1: Undefended comparison between RTAM-Fusion and Vocabulary Overlap across vocabulary sizes. Each metric is reported as RTAM-Fusion /Vocabulary Overlap. Check marks indicate cells where RTAM-Fusion is higher.

Figure 1: Undefended comparison between RTAM-Fusion and Vocabulary Overlap across vocabulary sizes. The figure shows AUC as vocabulary size increases.

$n_{\min}$	80k	110k	140k	170k	200k
32	0.823 /0.663	0.860 /0.699	0.880 /0.718	0.878 /0.736	0.877 /0.746
48	0.825 /0.663	0.843 /0.697	0.825 /0.714	0.822 /0.717	0.820 /0.734
64	0.822 /0.663	0.788 /0.685	0.770 /0.699	0.768 /0.707	0.768 /0.717

Table 2: Min-count defense comparison between RTAM-Fusion and Vocabulary Overlap across vocabulary sizes and pruning thresholds. Each cell reports AUC as RTAM-Fusion /Vocabulary Overlap.

Figure 2: Adaptive attacks against targeted token removal. The defense weakens a naive RTAM-Fusion attacker, but defense-aware attacks recover membership signal.

Defense	Static or initial result	Best adaptive AUC
Targeted token removal	0.548	0.694
Min-count, $n_{\min}=64$	0.609	0.761
Min-count, $n_{\min}=256$	stronger pruning	$\approx 0.591$
Merge-order randomization	order randomized	0.710–0.712
Bounded contribution	0.523 at 12k vocab	0.725 at 200k vocab

Table 3: Adaptive-defense results. The table reports the strongest adaptive attack for each defense, with production-vocabulary results except where a smaller vocabulary is explicitly named.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Attack	Metric	80k	110k	140k	170k	200k
CR	AUC	0.507	0.509	0.508	0.508	0.509
	BA	0.513	0.517	0.514	0.513	0.516
	TPR@1%FPR	0.72%	0.71%	0.68%	0.82%	0.77%
NB ( $k{=}20$ k)	AUC	0.534	0.526	0.535	0.546	0.564
	BA	0.526	0.524	0.533	0.538	0.551
	TPR@1%FPR	3.78%	3.44%	4.11%	5.86%	8.03%

Appendix

Table 4: Published tokenizer-membership baselines from Tong et al. Tong et al. (2026) , Table 3. Vocabulary Overlap (VO) is the strongest baseline across vocabulary sizes and metrics, with Frequency Estimation (FE) the next strongest.

Figure 3: Low false-positive comparison between RTAM-Fusion and Vocabulary Overlap across vocabulary sizes. The figure reports TPR at 1% FPR in the undefended setting.

Vocab	$[0,400)$	$[400,800)$	$[800,1200]$
80k	0.730 /0.646	0.786 /0.703	0.782 /0.712
110k	0.779 /0.662	0.846 /0.731	0.827 /0.743
140k	0.816 /0.677	0.881 /0.751	0.868 /0.768
170k	0.799 /0.683	0.847 /0.761	0.846 /0.795
200k	0.751 /0.694	0.819 /0.772	0.805 /0.797

Appendix

Table 5: Size-conditioned min-count defense results at

n_{\min}{=}64

. Each cell reports RTAM-Fusion/Vocabulary Overlap AUC, with datasets grouped by candidate dataset size. Vocabulary Overlap values are from Tong et al. Tong et al. (2026) , Table 9.

Figure 4: AUC gain of RTAM-Fusion over Vocabulary Overlap under the min-count defense. Each cell reports the difference in AUC for a pruning threshold and vocabulary size.

Figure 5: Size-conditioned AUC gain of RTAM-Fusion over Vocabulary Overlap under the min-count defense at

n_{\min}=64

. Candidate datasets are grouped by size.

Attack	Description	AUC
A0	Static RTAM-Fusion	0.548
A1	Re-locked rarity feature	0.653
A2	Compression-based fragmentation footprint	0.694

Appendix

Table 6: Adaptive attacks against targeted token removal. A naive RTAM-Fusion attacker is weakened by the defense, while defense-aware attacks recover membership signal. The undefended RTAM-Fusion AUC is

0.822

Defense mechanism	Adaptive AUC	Utility cost	Symmetric?
Targeted token removal, $K{=}32$ k	0.694	$-0.27\%$ bpt	No
Min-count, $n_{\min}{=}256$	0.591	115k tokens removed	Yes
Merge-order randomization	0.710–0.712	not measured at matched AUC	Yes
Bounded-contribution BPE, $C{=}1$	0.725	$-1.2\%$ bpt at 12k vocab	Yes

Appendix

Table 7: Adaptive-defense frontier at 200k vocabulary size. Each row reports the strongest adaptive attack observed for a defended tokenizer, together with the available utility measurement.

Figure 6: Adaptive-defense comparison at 200k vocabulary size. Each bar shows the strongest adaptive attack observed for a defended tokenizer.

Privacy Leakage in Language Models

Privacy leakage in language models concerns how models expose training data, infer sensitive traits, or transmit private context through outputs and internal channels.

See all 284 papers

From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution
Apr 15, 2026Pavel Chizhov, Egor Bogomolov, Ivan P. YamshchikovTokenization Tokenization in Language Models
CAIRO, Technical University of Applied Sciences Würzburg-Schweinfurt · JetBrains Research · TU Delft
Writing-System-Level Tokenizer Adaptation for Byte-Level BPE
Aug 1, 2026Bohdan DidenkoLow-Resource Language Processing Subword Tokenization
Lviv Polytechnic National University
Rethinking BPE: Merge Selection as Model Selection
Oct 19, 2026Vijini Liyanage, François YvonTokenization in Language Models Tokenization
Sorbonne-Université, CNRS, ISIR, Paris

Apr 15, 2026J/K move · Enter open · S save

From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution

Pavel Chizhov, Egor Bogomolov, Ivan P. Yamshchikov

CAIRO, Technical University of Applied Sciences Würzburg-Schweinfurt · JetBrains Research · TU Delft

Tokenization Tokenization in Language Models Byte-Pair Encoding

Open paper AskPDF arXiv

Efficiency and safety of Large Language Models (LLMs), among other factors, rely on the quality of tokenization. A good tokenizer not only improves inference speed and language understanding but also provides extra defense against jailbreak attacks and lowers the risk of hallucinations. In this work, we investigate the efficiency of code tokenization, in particular from the perspective of data source diversity. We demonstrate that code tokenizers are prone to producing unused, and thus under-trained, tokens due to the imbalance in repository and language diversity in the training data, as well as the dominance of source-specific, repetitive tokens that are often unusable in future inference. By modifying the BPE objective and introducing merge skipping, we implement different techniques under the name Source-Attributed BPE (SA-BPE) to regularize BPE training and minimize overfitting, thereby substantially reducing the number of under-trained tokens while maintaining the same inference procedure as with regular BPE. This provides an effective tool suitable for production use.

Abstract

Privacy Leakage in Language Models

Explore similar work

From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution

Writing-System-Level Tokenizer Adaptation for Byte-Level BPE

Rethinking BPE: Merge Selection as Model Selection