Semantic One-Dimensional Tokenizer for Image Reconstruction and Generation

Yunpeng Qu; Kaidong Zhang; Yukang Ding; Ying Chen; Jian Wang

cs.CVMar 17, 2026

Semantic One-Dimensional Tokenizer for Image Reconstruction and Generation

Authors: Yunpeng Qu, Kaidong Zhang, Yukang Ding, Ying Chen, Jian Wang

Organizations: Tsinghua University, Beijing, China · Alibaba Group, Beijing, China

Abstract

Visual generative models based on latent space have achieved great success, underscoring the significance of visual tokenization. Mapping images to latents boosts efficiency and enables multimodal alignment for scaling up in downstream tasks. Existing visual tokenizers primarily map images into fixed 2D spatial grids and focus on pixel-level restoration, which hinders the capture of representations with compact global semantics. To address these issues, we propose SemTok, a semantic one-dimensional tokenizer that compresses 2D images into 1D discrete tokens with high-level semantics. SemTok sets a new state-of-the-art in image reconstruction, achieving superior fidelity with a remarkably compact token representation. This is achieved via a synergistic framework with three key innovations: a 2D-to-1D tokenization scheme, a semantic alignment constraint, and a two-stage generative training strategy. Building on SemTok, we construct a masked autoregressive generation framework, which yields notable improvements in downstream image generation tasks. Experiments confirm the effectiveness of our semantic 1D tokenization. Our code will be open-sourced.

Figures & tables

Figure 1: An overview of SemTok. ( a ) SemTok compresses 2D images into 1D discrete tokens through the encoder and reconstructs images through the decoder. A semantic alignment constraint is introduced at the encoder end for compact semantic representations. ( b ) To explore the semantic diversity of the latent space, SemTok adopts a two-stage training strategy: in Stage I, a diffusion-based decoder is used to predict images from noise; in Stage II, a one-step refiner is trained on image reconstruction tasks to enhance texture details.

Figure 2: Illustration of masked autoregressive modeling of image generation with SemTok. ( left ) During training, we randomly mask several tokens and utilize bidirectional attention to allow each token to see all tokens, thereby perceiving global semantics. ( right ) During inference, the model predicts multiple tokens simultaneously in a random order until the complete sequence is filled.

Tokenizer	Type	Quant	$\#$ Bits-per-pixel	$\#$ Token	$\#$ Codebook	rFID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Taming VQ-GAN-re ( Esser et al., 2021 )	2D	VQ	0.055	16×16	$2^{14}$	4.98	-	-	-
LlamaGen-16 ( Sun et al., 2024 )	2D	VQ	0.055	16×16	$2^{14}$	2.19	20.67	0.589	0.132
MaskBiT ( Weber et al., 2024 )	2D	LFQ	0.055	16×16	$2^{14}$	1.37	21.50	0.560	-
Cosmos DI-16x16 ( Agarwal et al., 2025 )	2D	FSQ	0.062	16×16	$\approx 2^{16}$	4.40	19.98	0.536	0.153
OpenMagViT-V2 ( Luo et al., 2024 )	2D	LFQ	0.070	16×16	$2^{18}$	1.17	21.63	0.640	0.111
VAR ( Tian et al., 2024 )	2D	VQ	0.125	680	$2^{12}$	0.99	22.12	0.624	0.109

Table 1: Reconstruction results of SOTA tokenizers on ImageNet

(256\times 256)

. *: indicates a continuous tokenizer trained on OpenImages. -re: indicates the use of rejection sampling.

Figure 3: Comparison of reconstructions from different tokenizers.

Figure 4: Generation results with different tokenizers.

Generator	Type	Params	gFID $\downarrow$	IS $\uparrow$	Pre $\uparrow$	Rec $\uparrow$
BigGAN ( Brock et al., 2018 )	GAN	112M	6.95	224.5	0.89	0.38
GigaGAN ( Kang et al., 2023 )	GAN	569M	3.45	225.5	0.84	0.61
LDM-4-G ( Rombach et al., 2022 )	Diff.	400M	3.60	247.7	-	-
DiT-L/2 ( Peebles and Xie, 2023 )	Diff.	458M	5.02	167.2	0.75	0.57
DiT-XL/2 ( Peebles and Xie, 2023 )	Diff.	675M	2.27	278.2	0.83	0.57
VDM++ ( Kingma and Gao, 2023 )	Diff.	2.0B	2.12	267.7	-	-

Table 2: Generation results with other methods on ImageNet (

256\times 256

). *: indicates generating images at resolution

384\times 384

and resizing back to

256\times 256

Tokenizer	Type	gFID $\downarrow$	IS $\uparrow$	Pre $\uparrow$	Rec $\uparrow$
VQGAN ( Esser et al., 2021 )	2D	4.62	254.1	0.84	0.49
FlowMo-Lo ( Sargent et al., 2025 )	1D	6.68	280.4	0.82	0.45
SemTok (Ours)	1D	2.77	293.1	0.78	0.60

Table 3: Generation results of different tokenizers on ImageNet (

256\times 256

Figure 5: Progressive generation results in raster scan order between 1D and 2D tokens.

Constraint		Reconstruction		Generation
Contrastive	Distill	rFID $\downarrow$	PSNR $\uparrow$	gFID $\downarrow$	IS $\uparrow$
✗	✗	1.08	21.74	3.83	271.4
✓	✗	0.97	21.86	2.87	276.8
✗	✓	1.02	21.79	3.55	264.9
✓	✓	0.88	22.19	2.77	293.1

Table 4: Ablation on the Semantic Alignment Constraint .

Figure 6: Feature Clustering with different loss functions.

Order	Reconstruction		Generation
Order	rFID $\downarrow$	PSNR $\uparrow$	gFID $\downarrow$	IS $\uparrow$
Raster	1.22	21.07	3.08	269.9
Global	0.88	22.19	2.77	293.1

Table 5: Ablation on the Masked Autoregressive Modeling vs. Next-token Prediction .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Depth	Width	Heads	# Params
Encoder	8	768	12	113M
Decoder	24	1536	24	2236M

Appendix

Table 8: Architecture Configuration of SemTok.

Size	Depth	Width	Heads	# Params
L	24	1024	16	318M
XL	36	1280	20	746M
XXL	24	2048	32	1.2B

Appendix

Table 9: Architecture Configuration of the AR model.

Quantizer	rFID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
LFQ ( Yu et al., 2023 )	1.32	20.78	0.564	0.163
BSQ ( Zhao et al., 2024 )	0.88	22.19	0.639	0.128

Appendix

Table 10: Ablation on the Quantizer .

$\mathcal{L}_{\textit{per}}$	rFID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
LPIPS ( Zhang et al., 2018 )	1.13	22.73	0.649	0.133
ResNet ( He et al., 2016 )	0.89	22.02	0.622	0.133
LPIPS + ResNet	0.88	22.19	0.639	0.128

Appendix

Table 11: Ablation on the Perceptual Loss .

Stage	rFID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Only Fine-tuning	1.80	20.54	0.556	0.177
Pre-training & Fine-tuning	0.88	22.19	0.639	0.128

Appendix

Table 12: Ablation on the Generative Training Strategy .

Figure 7: The influence of CFG scale under Linear Schedule .

Figure 8: The influence of CFG scale under Constant Schedule .

Groups of Codebook	gFID $\downarrow$	IS $\uparrow$	Pre $\uparrow$	Rec $\uparrow$
2	4.77	251.4	0.76	0.48
18 (Bitwise)	2.77	293.1	0.78	0.60

Appendix

Table 13: Ablation on the Bitwise Classifier .

Strategy	gFID $\downarrow$	IS $\uparrow$	Pre $\uparrow$	Rec $\uparrow$
Confidence-based	3.60	246.8	0.74	0.62
Random	2.77	293.1	0.78	0.60

Appendix

Table 14: Ablation on the Sampling Strategy .

Figure 9: Ablation on the Sampling Steps for token generation.

Figure 10: Ablation on the Top- $p$ Filtering for token generation.

Figure 11: Comparison of reconstructions from different tokenizers.

Figure 12: Visualization of generated images from our SemTok-based AR model across random ImageNet classes.

Explore similar work

MacTok: Robust Continuous Tokenization for Image Generation
Mar 31, 2026Hengyu Zeng, Xin Gao, Guanghao Li +5Image Tokenization Visual Token Compression
Fudan University
QuadTok: Quadtree Visual Tokenizer for Autoregressive Image Generation
Oct 7, 2026Yucheng Mao, Zeyuan Chen, Xiaojun Shan +4Visual Tokenization Autoregressive Image Generation
University of California, San Diego
End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer
May 1, 2026Wenda Chu, Bingliang Zhang, Jiaqi Han +4Image Tokenization Visual Tokenization
ByteDance Seed · California Institute of Technology · Stanford University

Mar 31, 2026J/K move · Enter open · S save

MacTok: Robust Continuous Tokenization for Image Generation

Hengyu Zeng, Xin Gao, Guanghao Li, Yuxiang Yan+4

Fudan University

Image Tokenization Visual Token Compression Posterior Collapse

Open paper AskPDF arXiv

Continuous image tokenizers enable efficient visual generation, and those based on variational frameworks can learn smooth, structured latent representations through KL regularization. Yet this often leads to posterior collapse when using fewer tokens, where the encoder fails to encode informative features into the compressed latent space. To address this, we introduce MacTok, a Masked Augmenting 1D Continuous Tokenizer that leverages image masking and representation alignment to prevent collapse while learning compact and robust representations. MacTok applies both random masking to regularize latent learning and DINO-guided semantic masking to emphasize informative regions in images, forcing the model to encode robust semantics from incomplete visual evidence. Combined with global and local representation alignment, MacTok preserves rich discriminative information in a highly compressed 1D latent space, requiring only 64 or 128 tokens. On ImageNet, MacTok achieves a competitive gFID of 1.44 at 256

\times

256 and a state-of-the-art 1.52 at 512

\times

512 with SiT-XL, while reducing token usage by up to 64

\times

. These results confirm that masking and semantic guidance together prevent posterior collapse and achieve efficient, high-fidelity tokenization.

Abstract

Explore similar work

MacTok: Robust Continuous Tokenization for Image Generation

QuadTok: Quadtree Visual Tokenizer for Autoregressive Image Generation

End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer