Autoregressive (AR) image models have recently demonstrated remarkable generative capability, but their sequential nature results in significant inference latency. Existing training-free acceleration methods typically verify tokens independently, overlooking the strong co-occurrence patterns between adjacent visual tokens. This independence assumption often leads to contextual inconsistency and limits decoding efficiency. In this work, we introduce a novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated tokens within each decoding window. To construct such phrase units, we analyze token co-occurrence statistics from the training corpus and group frequently co-occurring tokens into semantically coherent visual phrases. During inference, the proposed phrase-level verification evaluates aggregated likelihood ratios over each phrase, allowing simultaneous acceptance of multiple tokens while preserving generation quality. Extensive experiments on autoregressive text-to-image generation show that our method significantly reduces the number of function evaluations (NFE) and achieves up to 30% faster decoding without compromising visual fidelity. Our findings reveal that modeling short-range token co-occurrence provides an effective and general principle for accelerating autoregressive inference.
Figures & tables
Figure 1: (a) Existing Token-Level Verification (e.g., SJD): Tokens are verified individually, often causing rejection of locally ambiguous tokens despite their contextual coherence, which breaks visual semantics.(b) Our Phrase-Level Verification (SJD-PV) :Contiguous tokens are grouped against a pre-constructed phrase library and verified jointly as coherent phrases This preserves coherent visual semantics to resolve local ambiguity, improving acceptance and accelerating AR image generation.
Figure 2: A toy example of the token phrase’s visual semantic. While the isolated patch for Token #6302 (top) presents an ambiguous texture, placing it within its contiguous sequence (bottom) exhibits a distinct zebra stripe structure. This validates that visual semantics are fundamentally defined by the collective behavior of neighboring tokens rather than stored independently.
Figure 3: Short-range token co-occurrence statistics. We compute token co-occurrence over consecutive positions in tokenized image sequences, revealing strong local correlations among specific visual tokens.
Configuration
Latency (↓)
NFE (↓)
Acceleration (↑)
FID (↓)
CLIP-Score (↑)
Latency
NFE
Parti-prompt
Lumina-mGPT ( Liu et al., 2024 )
79.37s
2392
1.00×
1.00×
–
32.091
Jacobi Decoding ( Song et al., 2021a )
82.21s
2300.0
0.97x
1.04×
–
32.091
SJD ( Teng et al., 2024 )
36.07s
1035.3
2.15x
2.31×
–
32.090
SJD + Ours
34.58s
941.4
2.29x
2.54x
–
32.098
Table 1: Quantitative evaluation on the MS-COCO 2017 and Parti-prompt.
Figure 4: Qualitative experiment. Our method on SJD and GSD shows on significant NFE acceleration while maintaining image quality.
Setting
Latency ↓
NFE ↓
CLIP ↑
w/o
35.96s
998.3
32.090
SJD-PV (Ours)
34.58s
941.4
32.098
Table 2: Ablation of Adaptive Neighborhood strategy
Setting
Latency ↓
NFE ↓
CLIP ↑
M=4k
37.02s
1005.6
32.072
M=8k
34.58 s
941.4
32.098
M=16k
34.60s
942.1
31.790
Table 3: Ablation of iterations M
Threshold ( τ )
Latency
NFE ↓
CLIP-Score ↑
τ=0.005
35.12s
965.2
32.095
τ=0.01
34.58s
941.4
32.098
τ=0.02
34.20s
928.5
32.012
τ=0.05
33.85s
915.0
31.650
Table 4: Ablation study on the neighborhood threshold τ .
Autoregressive (AR) models have demonstrated remarkable performance in generating high-fidelity images. However, their inherently sequential next-token prediction leads to significantly slower inference. Recent studies have introduced Jacobi-style decoding to accelerate autoregressive image generation. Extending the draft sequence initially improves efficiency, yet the acceleration quickly saturates as error propagation in the one-dimensional sequence hinders convergence. Observing that images exhibit strong local spatial correlations, we propose Parallel Jacobi Decoding (PJD), a training-free decoding approach that expands draft tokens in the two-dimensional spatial domain to enable efficient spatially parallel refinement. PJD adjusts the attention mask to mitigate error accumulation and improve convergence stability. Extensive experiments on diverse datasets show that PJD achieves 4.8x-6.4x acceleration across multiple autoregressive image generation models while maintaining competitive generation quality.
Speculative Jacobi Decoding (SJD) has emerged as a promising method for accelerating autoregressive image generation. Despite its potential, existing SJD approaches often suffer from the low acceptance rate issue of speculative tokens due to token selection ambiguity. Recent works attempt to mitigate this issue primarily from the relaxed token verification perspective but fail to fully exploit the iterative dynamics of decoding. In this paper, we conduct an in-depth analysis and make a novel observation that tokens whose probabilities increase are more likely to match the verification-accepted and correct token. Based on this, we propose a novel Speculative Jacobi Decoding with Verification Prediction (SJD-VP). The key idea is to leverage the change in token probabilities across iterations to guide sampling, favoring tokens whose probabilities increase. This effectively predicts which tokens are likely to pass subsequent verification, boosting the acceptance rate. In particular, our SJD-VP is plug-and-play and can be seamlessly integrated into existing SJD methods. Extensive experiments on standard benchmarks demonstrate that our SJD-VP method consistently accelerates autoregressive decoding while improving image generation quality.
Bingqi Shan, Baoquan Zhang, Xiaochen Qi +3
Harbin Institute of Technology, Shenzhen · Institute of Perceptual Intelligence, Pengcheng Laboratory, Shenzhen · SIFAR
Autoregressive models excel in visual generation by treating images as 1D sequences of discrete tokens, mirroring language modeling. However, this flattening discards the intrinsic 2D spatial locality of visual signals, creating severe computational bottlenecks during inference. We introduce Spatially Speculative Decoding (SSD), a framework that aligns the predictive objective with the natural geometry of images. Rather than predicting only the immediate next token in a 1D sequence, our model simultaneously predicts the adjacent horizontal token and the token directly below it. By capitalizing on this 2D spatial correlation, spatially speculative decoding overcomes the memory wall in visual inference. Our approach accelerates autoregressive image generation by up to 13.3x while maintaining high fidelity on DPG-Bench and GenEval. Our results suggest that respecting the underlying geometry of vision unlocks massive computational efficiencies, paving the way for real-time, high-resolution autoregressive generative models.