Flexible image tokenizers aim to represent an image using an ordered 1D variable-length token sequence. This flexible tokenization is typically achieved through nested dropout, where a portion of trailing tokens is randomly truncated during training, and the image is reconstructed using the remaining preceding sequence. However, this tail-truncation strategy inherently concentrates the image information in the early tokens, limiting the effectiveness of downstream AutoRegressive (AR) image generation as the token length increases. To overcome these limitations, we propose ReToK, a flexible tokenizer with Redundant Token Padding and Hierarchical Semantic Regularization, designed to fully exploit all tokens for enhanced latent modeling. Specifically, we introduce Redundant Token Padding to activate tail tokens more frequently, thereby alleviating information over-concentration in the early tokens. In addition, we apply Hierarchical Semantic Regularization to align the decoding features of earlier tokens with those from a pre-trained vision foundation model, while progressively reducing the regularization strength toward the tail to allow finer low-level detail reconstruction. Extensive experiments demonstrate the effectiveness of ReTok: on ImageNet 256×256, our method achieves superior generation performance compared with both flexible and fixed-length tokenizers. Code will be available at: https://github.com/zfu006/ReTok
Figures & tables
Figure 1: Overview of our method. (a) Previous flexible tokenizers (e.g., FlexTok Bachmann et al. (2025) and One-D-Piece Miwa et al. (2025) ) with naive nested dropout compress information into early tokens, compromising generation quality. (b) Our method with redundant token padding activates tail tokens, consistently improving generation quality as the sequence extends. (c) Comparison of ReTok with fixed-length tokenizers Sun et al. (2024) in Autoregressive (AR) generation. Unlike fixed-length tokenizers that require generating a fixed-length sequence before decoding, ReTok allows AR models to produce variable-length sequences.
Figure 2: Illustration of the generation bottleneck in flexible tokenizers. We train AR models and evaluate generation quality at varying token lengths. Results indicate a significant decrease in downstream generation compared to fixed-length tokenizers. Details are in Section 3.2 .
Figure 3
Type
Tokenizer
Param.
rFID ↓
Generator
Param.
Type
# Tokens
gFID ↓
gIS ↑
Precision ↑
Recall ↑
Continuous modeling
2D
SD-VAE Rombach et al. (2022)
84M
0.62
DiT-XL/2 Peebles and Xie (2023)
675M
Diff
1024
2.27
278.2
0.83
0.57
SiT-XL/2 Ma et al. (2024)
675M
Diff
1024
2.06
270.3
0.82
0.59
SiT-XL/2+REPA Yu et al. (2025)
675M
Diff
1024
1.42
305.7
0.80
0.65
2D
VA-VAE Yao et al. (2025)
70M
0.28
LightningDiT Yao et al. (2025)
675M
Diff
256
1.35
295.3
0.79
0.65
2D
VAE Li et al. (2024)
66M
0.53
MAR-B Li et al. (2024)
208M
AR+Diff
256
2.31
281.7
0.82
0.57
Table 1: Comparison of class-conditional image generation on ImageNet 256×256 . † denotes the model generates images at 336×336 resolution, and are resized to 256×256 for evaluation. ⋆ indicates models that are our implementation. We report the optimal gFID achieved across scenarios with and without Classifier-Free Guidance. Bold and underline indicate the first and second best methods within flexible tokenizers.
Figure 5: Examples of generated images on ImageNet 256×256 from the ReTok-S-B with LlamaGen-XL models using 256 tokens. The classifier-free guidance is set to 4.0 .
Table 6Figure 7
Pad. Tokens
rFID ↓
LPIPS ↓
gFID ↓
64
1.21
0.238
4.74
224
1.18
0.234
4.68
384
1.34
0.245
5.03
Table 4: Design choices of ReTok (S-S tokenizer). (a) Effect of the number of padding tokens. The reconstruction and generation performance are evaluated on the full token (256) length. (b) Effect of the initial weight of the hierarchical semantic regularization. We vary the semantic weight λ(32) while keeping λ(256) at 0.5. (c) Impact of the step size of nested dropout.
Init.
Epochs
gFID ↓
GigaTok
50
4.68
Scratch
250
4.82
Scratch
350
4.71
Table 5: Additional design choices of ReTok. (a) Effect of tokenizer initialization. (b) Comparison of padding and prefix-length reweighting. We compare redundant token padding with no padding using p=P(k=256)∈{0.3,0.5} , with the remaining lengths sampled uniformly. (c) Effect of the representation-alignment teacher. Experiments in (a,b) use ReTok-S-S with AR-B trained for 120 epochs, while (c) follows the full ReTok-S-B training protocol with AR-L trained for 300 epochs.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10
Configuration
ReTok-S-S
ReTok-S-B
Model
Parameters
136M
232M
Codebook Size
16384
Latent Dim
8
Num. Tokens
256
Pad. Tokens
224
Appendix
Table 6: Configurations of ReTok.
Tokenizer
Generator
CFG
256
224
192
160
128
96
64
32
ReTok-S-S
LlamaGen-B
5.75
4.02
4.02
4.13
4.34
4.6
5.16
6.61
10.55
ReTok-S-S
LlamaGen-L
2.25
2.92
3.0
3.23
3.52
3.8
4.37
5.47
8.25
ReTok-S-B
LlamaGen-L
1.75
2.66
2.71
2.83
3.04
3.18
3.57
4.54
6.76
ReTok-S-B
LlamaGen-XL
1.5
2.27
2.27
2.38
2.58
2.7
3.16
4.05
6.22
Appendix
Table 7: Generation performance (gFID) of AR models at different tokens.
Table 13Table 14
Figure 10: Examples of generated images on ImageNet 256×256 from the ReTok-S-B + LlamaGen-XL models using 256 tokens. The classifier-free guidance is set to 4.0 .
Figure 11: Examples of progressive generation on ImageNet 256×256 from the ReTok-S-B + LlamaGen-XL models. Complex scenes require more tokens, while a small number of tokens is sufficient for simple scenes.