Flexible image tokenizers aim to represent an image using an ordered 1D variable-length token sequence. This flexible tokenization is typically achieved through nested dropout, where a portion of trailing tokens is randomly truncated during training, and the image is reconstructed using the remaining preceding sequence. However, this tail-truncation strategy inherently concentrates the image information in the early tokens, limiting the effectiveness of downstream AutoRegressive (AR) image generation as the token length increases. To overcome these limitations, we propose ReToK, a flexible tokenizer with Redundant Token Padding and Hierarchical Semantic Regularization, designed to fully exploit all tokens for enhanced latent modeling. Specifically, we introduce Redundant Token Padding to activate tail tokens more frequently, thereby alleviating information over-concentration in the early tokens. In addition, we apply Hierarchical Semantic Regularization to align the decoding features of earlier tokens with those from a pre-trained vision foundation model, while progressively reducing the regularization strength toward the tail to allow finer low-level detail reconstruction. Extensive experiments demonstrate the effectiveness of ReTok: on ImageNet 256×256, our method achieves superior generation performance compared with both flexible and fixed-length tokenizers. Code will be available at: https://github.com/zfu006/ReTok
Figures & tables
Figure 1: Overview of our method. (a) Previous flexible tokenizers (e.g., FlexTok Bachmann et al. (2025) and One-D-Piece Miwa et al. (2025) ) with naive nested dropout compress information into early tokens, compromising generation quality. (b) Our method with redundant token padding activates tail tokens, consistently improving generation quality as the sequence extends. (c) Comparison of ReTok with fixed-length tokenizers Sun et al. (2024) in Autoregressive (AR) generation. Unlike fixed-length tokenizers that require generating a fixed-length sequence before decoding, ReTok allows AR models to produce variable-length sequences.
Figure 2: Illustration of the generation bottleneck in flexible tokenizers. We train AR models and evaluate generation quality at varying token lengths. Results indicate a significant decrease in downstream generation compared to fixed-length tokenizers. Details are in Section 3.2 .
Figure 3
Type
Tokenizer
Param.
rFID ↓
Generator
Param.
Type
# Tokens
gFID ↓
gIS ↑
Precision ↑
Recall ↑
Continuous modeling
2D
SD-VAE Rombach et al. (2022)
84M
0.62
DiT-XL/2 Peebles and Xie (2023)
675M
Diff
1024
2.27
278.2
0.83
0.57
SiT-XL/2 Ma et al. (2024)
675M
Diff
1024
2.06
270.3
0.82
0.59
SiT-XL/2+REPA Yu et al. (2025)
675M
Diff
1024
1.42
305.7
0.80
0.65
2D
VA-VAE Yao et al. (2025)
70M
0.28
LightningDiT Yao et al. (2025)
675M
Diff
256
1.35
295.3
0.79
0.65
2D
VAE Li et al. (2024)
66M
0.53
MAR-B Li et al. (2024)
208M
AR+Diff
256
2.31
281.7
0.82
0.57
Table 1: Comparison of class-conditional image generation on ImageNet 256×256 . † denotes the model generates images at 336×336 resolution, and are resized to 256×256 for evaluation. ⋆ indicates models that are our implementation. We report the optimal gFID achieved across scenarios with and without Classifier-Free Guidance. Bold and underline indicate the first and second best methods within flexible tokenizers.
Figure 5: Examples of generated images on ImageNet 256×256 from the ReTok-S-B with LlamaGen-XL models using 256 tokens. The classifier-free guidance is set to 4.0 .
Table 6Figure 7
Pad. Tokens
rFID ↓
LPIPS ↓
gFID ↓
64
1.21
0.238
4.74
224
1.18
0.234
4.68
384
1.34
0.245
5.03
Table 4: Design choices of ReTok (S-S tokenizer). (a) Effect of the number of padding tokens. The reconstruction and generation performance are evaluated on the full token (256) length. (b) Effect of the initial weight of the hierarchical semantic regularization. We vary the semantic weight λ(32) while keeping λ(256) at 0.5. (c) Impact of the step size of nested dropout.
Init.
Epochs
gFID ↓
GigaTok
50
4.68
Scratch
250
4.82
Scratch
350
4.71
Table 5: Additional design choices of ReTok. (a) Effect of tokenizer initialization. (b) Comparison of padding and prefix-length reweighting. We compare redundant token padding with no padding using p=P(k=256)∈{0.3,0.5} , with the remaining lengths sampled uniformly. (c) Effect of the representation-alignment teacher. Experiments in (a,b) use ReTok-S-S with AR-B trained for 120 epochs, while (c) follows the full ReTok-S-B training protocol with AR-L trained for 300 epochs.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10
Configuration
ReTok-S-S
ReTok-S-B
Model
Parameters
136M
232M
Codebook Size
16384
Latent Dim
8
Num. Tokens
256
Pad. Tokens
224
Appendix
Table 6: Configurations of ReTok.
Tokenizer
Generator
CFG
256
224
192
160
128
96
64
32
ReTok-S-S
LlamaGen-B
5.75
4.02
4.02
4.13
4.34
4.6
5.16
6.61
10.55
ReTok-S-S
LlamaGen-L
2.25
2.92
3.0
3.23
3.52
3.8
4.37
5.47
8.25
ReTok-S-B
LlamaGen-L
1.75
2.66
2.71
2.83
3.04
3.18
3.57
4.54
6.76
ReTok-S-B
LlamaGen-XL
1.5
2.27
2.27
2.38
2.58
2.7
3.16
4.05
6.22
Appendix
Table 7: Generation performance (gFID) of AR models at different tokens.
Table 13Table 14
Figure 10: Examples of generated images on ImageNet 256×256 from the ReTok-S-B + LlamaGen-XL models using 256 tokens. The classifier-free guidance is set to 4.0 .
Figure 11: Examples of progressive generation on ImageNet 256×256 from the ReTok-S-B + LlamaGen-XL models. Complex scenes require more tokens, while a small number of tokens is sufficient for simple scenes.
One-dimensional (1D) variable-length visual tokenizers enable adaptive compression by varying the number of tokens, allowing downstream autoregressive (AR) models to flexibly trade off generation quality against computational cost using a single tokenizer. However, existing approaches based on nested dropout often fail to fully exploit the representational capacity of the tokenizer, resulting in suboptimal performance in both image reconstruction and generation. In this work, we introduce NesTok, a nested self-alignment framework tailored to dynamic visual tokenizers. NesTok introduces cross-length training, which jointly optimizes reconstruction across token lengths while using the full-length sequence to guide shorter counterparts, enabling shorter token sequences to approach the reconstruction quality of full-length sequences. On ImageNet, NesTok improves substantially over standard training and achieves an rFID score of 0.98. On downstream image generation, it achieves the state-of-the-art gFID score of 1.46 on ImageNet 256×256 among existing variable-length autoregressive image generation methods. Code will be available at https://github.com/Jiawei804/NesTok.
Jiawei Zhang, Shuhao Liu, Rong Huang +4
North China Electric Power University · University of Science and Technology of China · Stanford University
Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256x256 generation.
Wenda Chu, Bingliang Zhang, Jiaqi Han +4
ByteDance Seed · California Institute of Technology · Stanford University
We introduce QuadTok, a novel framework for visual tokenization and autoregressive image generation. Compared to traditional approaches using 2D grids or 1D token sequences, we propose a hierarchical quadtree structure, bridging the gap between 2D spatial binding and 1D sequence-level flexibility. The QuadTok tokenizer dynamically allocates representational capacity to visually intricate areas while leaving homogeneous regions at a coarse resolution. Compared with a fixed 256-token grid, our ImageNet-trained tokenizer saves approximately 10% of tokens on ImageNet and 9% when transferred zero-shot to the COCO dataset, while maintaining comparable reconstruction fidelity. Furthermore, the natural causality introduced by the tree structure seamlessly enables autoregressive image generation. Conditioned on a quadtree topology supplied before generation, our 947M GPT-style generative model achieves a 2.08 gFID on the ImageNet 256×256 benchmark. Additionally, leveraging the strong spatial correlation preserved by the quadtree structure, the QuadTok generator enables zero-shot spatially controlled image generation capabilities. Code: https://github.com/myc634/QuadTok.