Efficient visual tokenization requires balancing representation compactness with the preservation of fine-grained image content. Conventional grid-based tokenizers allocate tokens uniformly across spatial locations despite non-uniform visual complexity, whereas compact 1D tokenizers aggregate global information but can struggle to preserve local details. We introduce MixTok, a hybrid visual tokenizer that combines a compact 1D global representation with sparse 2D tokens for local refinement. A content-adaptive router ranks candidate locations by refinement priority and determines the number of 2D tokens for each image under a target average token budget. At the highest-scoring locations, 2D token features replace the spatial features recovered from the 1D representation, and a shared decoder reconstructs the image from the resulting mixed features. On ImageNet 256 x 256, MixTok achieves highly competitive reconstruction and generation performance, with an rFID of 1.195 and a gFID of 1.95 at respective average budgets of approximately 128 and 131 content tokens per image.
Figures & tables
Figure 2: Overview of MixTok. Compact 1D tokens provide a base feature grid, while content-adaptive top- p selection allocates 2D refinement tokens across spatial locations. The selected 2D features replace the corresponding base features, and a shared decoder reconstructs the image from the mixed grid.
Figure 3: Two-stage masked generation with MixTok. A 1D MaskGIT generates global tokens that guide refinement location selection and provide spatial context for a 2D MaskGIT. Features obtained from the generated 2D tokens replace the corresponding base features before shared decoding.
Tokenizer
Avg. tokens
rFID ↓
PSNR (dB) ↑
TiTok S-128 [ 41 ]
128
1.7222
17.72
TiTok BL-128 VQ [ 17 ]
128
1.5063
19.83
One-D-Piece L-256 [ 23 ]
128
1.4406
17.71
FlexTok d18-d28 [ 1 ]
128
1.1422
16.76
DetailFlow-16 [ 19 ]
128
1.1370
19.46
LlamaGen VQ-16 [ 33 ]
256
2.1872
20.78
Table 1: Reconstruction quality and average content-token count. Token counts are rounded to the nearest integer. Baseline configurations are detailed in Appendix A.4 . Estimated bitrates are reported in Appendix A.6 .
Figure 4: Qualitative reconstruction results. Selective 2D refinement recovers fine details and sharpens structures that are blurred or missing in the 1D base reconstructions, while retaining a compact token representation.
Tokenizer
Tokens
rFID ↓
PSNR (dB) ↑
ALIT-128
128
2.9453
18.71
MixTok-ALIT
128
2.4720
18.76
Table 2: Transferability to ALIT–VQGAN at a fixed budget of 128 content tokens.
Generator
Params
Tokens
Steps
gFID ↓
Latency ↓ (ms)
FLOPs ↓ (T/image)
MaskGIT [ 4 ]
227M
256
8
6.06 ∗
—
—
AdaGen-MaskGIT-L [ 24 ]
206M
256
16
2.4883
372.0
3.623
TiTok-S-128 [ 41 ]
295M
128
64
1.9994
1018.4
9.949
Halton-MaskGIT-L [ 2 ]
480M
576
32
2.5143
591.5
26.690
One-D-Piece-L-256 [ 23 ]
177M
256
16
2.4144
350.1
3.526
LPD-L-256 [ 44 ]
337M
256
20
2.3863
215.5
0.879
Table 3: Class-conditional ImageNet generation. Unmarked gFID values are from our ADM reevaluations, while ∗ denotes an author-reported result. Latency and FLOPs are measured on one RTX 5090 at batch size 1. Configuration and measurement details are provided in Appendix A.8 .
Figure 5: Qualitative class-conditional generation with MixTok.
Selection strategy
rFID ↓
PSNR ↑
Reconstruction Grid Scorer
1.1950
19.5074
Generation Grid Scorer
1.3479
18.4776
Random
1.5460±0.0093
17.2147±0.0010
Uniform
1.6604
17.1749
Gradient entropy
1.4012
18.0077
Table 4: Grid Scorer ablation with identical per-image budgets. Random ranking reports mean ± standard deviation over three seeds; the other rankings are deterministic.
Allocation strategy
Avg. K
rFID ↓
PSNR ↑
Fixed K=96
96.0000
1.2052
19.4591
Top-p dynamic K
96.1093
1.1950
19.5074
Shuffled K
96.1093
1.2236±0.0068
19.4181±0.0005
Table 5: Budget Controller ablation with a fixed Grid Scorer. The shuffled control preserves the top-p budget histogram and reports mean ± standard deviation over three permutations.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Alignment
Fixed budget
Top- p
Epochs
2.5
20
6
Global batch size
96
192
160
Refinement allocation
None
K=96
T=2,p=0.91
LR: G1D,P1D
5×10−5
10−6
5×10−7
LR: 2D encoder / pre-quantization projection
—
2×10−7
10−7
LR: P2D , image decoder
—
8×10−7
4×10−7
Appendix
Table 6: Reconstruction training settings for the main configuration. Batch sizes are global across eight GPUs. Dashes denote frozen or inactive modules.
Term
Weight
Term
Weight
Image L1
2.0
Scorer supervision
1.0
LPIPS
0.6
Budget penalty
20.0
DINOv2 feature loss
0.5
Binarization penalty
0.01
CLIP feature loss
0.5
Generator adversarial loss
0.12
Discriminator feature matching
0.5
Appendix
Table 7: Loss coefficients during joint reconstruction training. The image terms are implemented as 2(L1+0.3LLPIPS) . Adversarial and discriminator feature-matching terms follow the activation schedule in Appendix A.2.1 .
Setting
Embedding Adapter
Halton generator
Optimizer updates
20,000
20,000
Global batch size
512
256
Learning rate
5×10−4
2×10−5
Weight decay
0.01
0.03
Warm-up updates
500
2,500
Training precision
BF16
BF16
Appendix
Table 8: Continuous-context generation training settings. Each model is trained on eight GPUs.
Seed
1D base gFID ↓
Full refinement gFID ↓
Mean K
Mean total tokens
1
4.2758
1.9518
98.66
130.66
2
4.3697
1.9509
98.53
130.53
3
4.3611
1.9597
98.75
130.75
4
4.3318
1.9420
98.69
130.69
Mean ± SD
4.3346±0.0424
1.9511±0.0072
98.66±0.09
130.66±0.09
Appendix
Table 9: Generation quality before and after sparse 2D refinement across sampling seeds. Each seed uses 50,000 images. The final row reports the mean ± sample standard deviation across four runs. Token statistics refer to the final representation.
Figure 6: Distribution of selected 2D token counts K for reconstruction on the ImageNet validation set and generation with sampling seeds 2–4, with 50,000 images per panel. The vertical axis shows the percentage of images assigned each integer K . The fixed 32 1D tokens are excluded. Each panel spans the smallest to largest observed K , retaining zero-count bins within this range.
Tokenizer
Avg. content tokens
Bits per token index
Total bpp ↓
rFID ↓
PSNR (dB) ↑
TiTok S-128 [ 41 ]
128
12
0.02344
1.7222
17.72
TiTok BL-128 VQ [ 17 ]
128
13
0.02539
1.5063
19.83
One-D-Piece L-256 [ 23 ]
128
12
0.02344
1.4406
17.71
FlexTok d18-d28 [ 1 ]
128
16
0.03125
1.1422
16.76
DetailFlow-16 [ 19 ]
128
13
0.02539
1.1370
19.46
LlamaGen VQ-16 [ 33 ]
256
14
0.05469
2.1872
20.78
Appendix
Table 10: Reconstruction quality and estimated fixed-width bitrate on ImageNet 256×256 . Total bpp includes spatial side information. Token counts are rounded for display; bitrate uses unrounded counts.
2D rounds
Total rounds
gFID ↓
4
12
2.0314
8
16
1.9420
16
24
1.9204
32
40
1.9191
48
56
1.9242
64
72
1.9355
Appendix
Table 11: Effect of 2D sampling rounds with all other settings fixed. All settings use sampling seed 4.
Image tokenizers, from 2D grids to recent 1D sequences, typically encode every image with the same fixed number of tokens. Yet visual complexity is highly heterogeneous, so a uniform budget overspends on simple inputs and underserves complex ones. Existing elastic tokenizers expose variable-length reconstructions, but often leave token length as a deployment-time operating point, a search target, or an external prediction rather than an output of the tokenizer itself. In this work, we ask whether a discrete visual tokenizer can budget itself in one pass. Our central finding is that actionable elasticity requires a representation--allocation co-design: prefixes must remain decodable across budgets, and the tokenizer must learn which prefix each image needs. We propose AdaTok, a self-budgeting discrete 1D tokenizer. AdaTok combines Prioritized Representation Learning, which orders tokens with nested tail masking and resolves budget-dependent semantic shift through Multi-Head LoRA decoder heads, with Adaptive Token Allocation, which trains a lightweight deterministic-group GRPO policy over candidate budgets. Dynamic Pareto Weighting balances fidelity and efficiency during policy training without manual trade-off sweeps. On ImageNet-1K, AdaTok-Full reaches rFID 1.31 at 256 tokens, while AdaTok-Adaptive attains rFID 1.50 using only ~118 tokens on average, outperforming discrete 1D baselines at comparable budgets. In autoregressive image generation, the shorter adaptive representation yields ~2.1x throughput over a fixed 256-token decode, suggesting that visual token count can be learned as a content-conditioned output rather than set as a fixed hyperparameter.
Xiaocheng Lu, Yuxi Chen, Jie Zhang +5
The Hong Kong University of Science and Technology · The Hong Kong Polytechnic University
Visual generative models based on latent space have achieved great success, underscoring the significance of visual tokenization. Mapping images to latents boosts efficiency and enables multimodal alignment for scaling up in downstream tasks. Existing visual tokenizers primarily map images into fixed 2D spatial grids and focus on pixel-level restoration, which hinders the capture of representations with compact global semantics. To address these issues, we propose SemTok, a semantic one-dimensional tokenizer that compresses 2D images into 1D discrete tokens with high-level semantics. SemTok sets a new state-of-the-art in image reconstruction, achieving superior fidelity with a remarkably compact token representation. This is achieved via a synergistic framework with three key innovations: a 2D-to-1D tokenization scheme, a semantic alignment constraint, and a two-stage generative training strategy. Building on SemTok, we construct a masked autoregressive generation framework, which yields notable improvements in downstream image generation tasks. Experiments confirm the effectiveness of our semantic 1D tokenization. Our code will be open-sourced.
Yunpeng Qu, Kaidong Zhang, Yukang Ding +2
Tsinghua University, Beijing, China · Alibaba Group, Beijing, China
Continuous image tokenizers enable efficient visual generation, and those based on variational frameworks can learn smooth, structured latent representations through KL regularization. Yet this often leads to posterior collapse when using fewer tokens, where the encoder fails to encode informative features into the compressed latent space. To address this, we introduce MacTok, a Masked Augmenting 1D Continuous Tokenizer that leverages image masking and representation alignment to prevent collapse while learning compact and robust representations. MacTok applies both random masking to regularize latent learning and DINO-guided semantic masking to emphasize informative regions in images, forcing the model to encode robust semantics from incomplete visual evidence. Combined with global and local representation alignment, MacTok preserves rich discriminative information in a highly compressed 1D latent space, requiring only 64 or 128 tokens. On ImageNet, MacTok achieves a competitive gFID of 1.44 at 256×256 and a state-of-the-art 1.52 at 512×512 with SiT-XL, while reducing token usage by up to 64×. These results confirm that masking and semantic guidance together prevent posterior collapse and achieve efficient, high-fidelity tokenization.