Organizations: Peking University · Meituan Longcat · Shanghai Jiao Tong University · Huazhong University of Science and Technology · The Chinese University of Hong Kong · Tsinghua University · MBZUAI
Discrete autoregressive models offer a scalable framework for image generation. However, they face a fundamental trade-off: long token sequences preserve representation quality but slow generation, while aggressive spatial compression reduces representation capacity. Based on product quantization, we show that, under a fixed nominal information budget, group size should scale quadratically with the per-axis spatial compression factor. Building on this principle, we propose GSTAR, a spatial--group co-scaling framework that enables 32× and even effective 64× compression while preserving strong reconstruction, understanding, and generation performance. Since larger groups introduce intra-group dependencies that must be modeled efficiently, we analyze their impact on inference latency and generation quality and introduce a semi-autoregressive decoder that combines parallel feature computation with lightweight causal modeling. With these designs, our highly compressed models perform competitively with leading models across five representative text-to-image benchmarks (GenEval, DPG-Bench, PRISM, LongText-Bench, and Qwen-Image-Bench). At 1024×1024 resolution, our model delivers a 20× speedup over prior discrete autoregressive models, generating each image in 2.5 seconds on a single NVIDIA H200. Together, these results show that highly compressed discrete representations can enable high-quality autoregressive generation at low latency.
Figures & tables
Figure 1: Overview of GSTAR. (a) Text-to-image generation samples spanning portraits, landscapes, typography, and structured layouts. (b) Image editing examples: given an original image and a text instruction, our model produces faithful edits while preserving unrelated content (original → edited). (c) Average benchmark score comparison, where our GSTAR-Pro-8B achieves competitive performance against leading proprietary, diffusion, and discrete autoregressive models. (d) Quality vs. latency at 1024×1024 resolution, where our GSTAR-Pro-8B and GSTAR-Flash-1.7B models achieve competitive quality with significantly lower latency.
Figure 2: Grouped tokenization and spatial–group co-scaling. (a) A semantic encoder maps an image to a spatially compressed latent grid, and group-wise quantization represents each position with G discrete codes. Lossless 2 × 2 packing converts the native 32× grid into an effective 64× interface, reducing the number of global autoregressive steps by four while increasing the codes per step from G to 4G. (b) Jointly scaling spatial compression and group size preserves the nominal information capacity per image while shortening the global sequence.
Figure 3: Comparison of intra-group prediction heads. Given the backbone context ht, (a) causal autoregressive decoding captures dependencies among codes but requires repeated, expensive head evaluations, while (b) parallel prediction head is fast but ignores these dependencies, risking inconsistent code combinations. (c) Our semi-autoregressive head addresses both limitations: a par- allel feature generator performs the heavy computation once, and a lightweight single-layer causal Transformer models dependencies during sequential sampling.
Model
Comp.
Params.
GenEval ↑
DPG ↑
PRISM ↑
LongText ↑
Qwen-Image ↑
Proprietary Models
Kolors 2.0
–
–
–
–
–
0.258
–
GPT Image 1
–
–
0.84
85.15
8.07
0.956
54.07
Seedream 3.0
–
–
0.84
88.27
7.62
0.896
–
Diffusion Models
HiDream-I1-Full
16×
17B
0.83
85.89
7.26
0.543
43.08
Table 1: Text-to-image generation results. PRISM is the English overall combined score (Qwen2.5-VL-72B judge), rescaled from 0–100 to 0–10. LongText reports the English subset; Qwen-Image denotes Qwen-Image-Bench overall (0–100).
G
rFID ↓
gFID ↓
Linear probe ↑ (%)
1
2.0624
2.1057
81.45
8
0.8525
6.3773
82.90
16
0.5357
5.7388
84.15
32
0.4325
17.05
84.55
Table 2: Effects of group size with K=16,384 .
G
GenEval ↑
DPG ↑
PRISM ↑
8
0.8060
82.49
5.30
16
0.8386
82.48
5.46
Table 3: Text-to-image group scaling. PRISM: combined score (0–10).
Table 7
Figure 5: Cumulative speedup breakdown for GSTAR-Flash-1.7B at 1024×1024 . Starting from the Causal AR head baseline (22.23 s), each stage—Semi-AR head, CUDA graph capture, fused top- k kernel, and SGLang serving—contributes multiplicative savings, reaching 8.9× end-to-end.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Resolution
Steps
Seq. length
Global batch
LR
PT
512px
400k
4,096
64
10−4
CT
1024px
100k
4,096
64
10−4
SFT
1024px / 2048px
30k
8,192
32
10−5
Appendix
Table 6: Training configurations shared by GSTAR-Pro-8B and GSTAR-Flash-1.7B. All stages use any-resolution training. SFT mixes the 1024px and 2048px resolution tiers at a ratio of 1:1.
Benchmark
Continuous
Discrete (f32g16)
MathVista [ 34 ]
45.7
44.5
RealWorldQA [ 52 ]
60.1
59.3
MMStar [ 10 ]
45.8
44.7
OCRBench [ 33 ]
676
665
TextVQA [ 45 ]
71.3
71.0
AI2D (NO_MASK) [ 27 ]
76.7
78.6
Appendix
Table 7: Multimodal understanding performance using continuous and discrete visual representations. Higher is better for all metrics. The average is computed over the eight listed benchmarks, with OCRBench divided by 10 to match the 0–100 scale of the other scores.
Compression
G
GenEval ↑
DPG ↑
PRISM ↑
32×
8
0.8060
82.49
5.30
64×
32
0.7885
80.44
5.13
Appendix
Table 8: Text-to-image generation with lossless latent packing. The configuration with 64× spatial compression and G=32 is obtained by losslessly packing the configuration with 32× spatial compression and G=8 , leaving tokenizer reconstruction unchanged. PRISM is the combined score on a 0–10 scale.
Tokenizer
Spatial compression
PSNR (dB) ↑
VQGAN [ 16 ]
16×
22.24
IBQ [ 44 ]
16×
23.39
VAR [ 47 ]
16×
23.43
Open-MAGVIT2 [ 35 ]
16×
23.98
Emu3.5 [ 13 ]
16×
23.82
GSTAR-Tokenizer (Ours) f32g16
32×
25.12
Appendix
Table 9: Image reconstruction quality on the ImageNet validation set at 512×512 resolution. We report the effective per-axis spatial compression factor; for VAR it refers to the finest latent scale. Higher PSNR is better.
Figure 6: Qualitative reconstruction examples. From left to right: ground truth, f32g16 ViT decoder, f32g16 refiner, f64g32 ViT decoder, and f64g32 refiner. Each row shows the same example across all methods. These visual examples are shown at 1024×1024 resolution, separately from the quantitative ImageNet evaluation at 512×512 .