Speculative Jacobi Decoding (SJD) has emerged as a promising method for accelerating autoregressive image generation. Despite its potential, existing SJD approaches often suffer from the low acceptance rate issue of speculative tokens due to token selection ambiguity. Recent works attempt to mitigate this issue primarily from the relaxed token verification perspective but fail to fully exploit the iterative dynamics of decoding. In this paper, we conduct an in-depth analysis and make a novel observation that tokens whose probabilities increase are more likely to match the verification-accepted and correct token. Based on this, we propose a novel Speculative Jacobi Decoding with Verification Prediction (SJD-VP). The key idea is to leverage the change in token probabilities across iterations to guide sampling, favoring tokens whose probabilities increase. This effectively predicts which tokens are likely to pass subsequent verification, boosting the acceptance rate. In particular, our SJD-VP is plug-and-play and can be seamlessly integrated into existing SJD methods. Extensive experiments on standard benchmarks demonstrate that our SJD-VP method consistently accelerates autoregressive decoding while improving image generation quality.
Figures & tables
Figure 1: Illustration of the structural decoupling in existing SJD and our solution, SJD-VP. In (a), the SJD variants drafting step selects tokens based only on the sampling distribution of the current iteration, ignoring the verification history. As a result, the drafting step is decoupled from the verification step, often leading to rejected tokens. In contrast, (b), our SJD-VP actively leverages verification criterion as a prior by analyzing probability trajectories of tokens. By predicting verification preference and guiding the distribution to favor consecutive probability growth tokens, it achieves alignment between the drafting and verification steps.
Figure 2: Motivation Analysis on Verification-Accepted Tokens.
Figure 3: Motivation Analysis on Correct Tokens.
Configuration
Latency (↓)
NFE (↓)
Acceleration (↑)
FID (↓)
CLIP-Score (↑)
Latency
NFE
Parti-prompt
Lumina-mGPT ( Liu et al., 2024b )
79.37s
2392
1.00×
1.00×
–
32.09
Jacobi Decoding ( Song et al., 2021 )
82.21s
2300.0
0.97x
1.04×
–
32.09
SJD ( Teng et al., 2024 )
36.07s
1035.3
2.20x
2.31×
–
32.09
SJD-VP
27.00s
720.65
2.94x
3.32x
–
32.15
Table 1: Results of quantitative evaluation on the Parti-prompt and MS-COCO 2017 benchmarks.. The best results are highlighted in bold.
Figure 4: Qualitative experiment. Our method shows on average 1.3x NFE acceleration while maintaining image quality.
Table 6
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
γ
Latency
NFE
CLIP
1.0
28.75 s
781.6
32.12
0.8
27.00 s
720.7
32.15
0.6
27.51 s
737.8
32.11
0.4
27.92 s
752.0
32.09
0.2
28.04 s
756.9
32.12
Appendix
Table 6: Ablation of Decay Factor γ on PartPrompt.
Filter Top- k
Latency
NFE
CLIP
w/o Filter
32.19 s
904.8
32.09
k=70%
30.02 s
833.6
32.10
k=50%
28.47 s
759.0
33.10
k=30%
27.51 s
739.4
32.11
k=10%
27.00 s
720.7
32.15
Appendix
Table 7: Ablation of the Top- k filtering strategy on PartPrompt.
Speculative Jacobi Decoding (SJD) is an important approach for accelerating autoregressive image generation. Although SJD has shown superior performance, recent studies point out that it usually suffers from a token ambiguity issue during token verification but its reason can not be well explained. To figure out this reason, in this paper, we conduct a visualization analysis on vision token and find that different from text tokens, vision tokens generally corresponds to some local, small, and unclear vision details, which means only using single token is difficult to accurately express a certain semantic, thereby causing token ambiguity issue. To this end, we propose a novel Speculative Jacobi Decoding with Semantics Verification (called SJD-SV), for accelerating autoregressive image generation. The key idea is that leveraging the strong correction characters between tokens to recognize semantic-aware token subsequence and then instead of perform token-by-token verification, turning to perform verification on semantic-aware token subsequence level for accelerating image generation. In particular, our method is plug-in, which can be directly integrated into existing SJD and its variants. Extensive experiments on various datasets show that existing SJD methods achieve significant performance improvement after integrating our SJD-SV method.
Baoquan Zhang, Bingqi Shan, Shihao Fang +3
Shenzhen Key Laboratory of Internet Information Collaboration, Harbin Institute of Technology, Shenzhen · Pengcheng Laboratory, Shenzhen
Autoregressive (AR) image models have recently demonstrated remarkable generative capability, but their sequential nature results in significant inference latency. Existing training-free acceleration methods typically verify tokens independently, overlooking the strong co-occurrence patterns between adjacent visual tokens. This independence assumption often leads to contextual inconsistency and limits decoding efficiency. In this work, we introduce a novel training-free acceleration framework that performs phrase-level speculative verification, enabling the model to jointly validate multiple correlated tokens within each decoding window. To construct such phrase units, we analyze token co-occurrence statistics from the training corpus and group frequently co-occurring tokens into semantically coherent visual phrases. During inference, the proposed phrase-level verification evaluates aggregated likelihood ratios over each phrase, allowing simultaneous acceptance of multiple tokens while preserving generation quality. Extensive experiments on autoregressive text-to-image generation show that our method significantly reduces the number of function evaluations (NFE) and achieves up to 30% faster decoding without compromising visual fidelity. Our findings reveal that modeling short-range token co-occurrence provides an effective and general principle for accelerating autoregressive inference.
Autoregressive (AR) models have demonstrated remarkable performance in generating high-fidelity images. However, their inherently sequential next-token prediction leads to significantly slower inference. Recent studies have introduced Jacobi-style decoding to accelerate autoregressive image generation. Extending the draft sequence initially improves efficiency, yet the acceleration quickly saturates as error propagation in the one-dimensional sequence hinders convergence. Observing that images exhibit strong local spatial correlations, we propose Parallel Jacobi Decoding (PJD), a training-free decoding approach that expands draft tokens in the two-dimensional spatial domain to enable efficient spatially parallel refinement. PJD adjusts the attention mask to mitigate error accumulation and improve convergence stability. Extensive experiments on diverse datasets show that PJD achieves 4.8x-6.4x acceleration across multiple autoregressive image generation models while maintaining competitive generation quality.